17th March 2023

5 min read

AI Art Generator With Conditioning Using ControlNet

AI Art Generator With Conditioning Using ControlNet

You want to inspire your kid to learn yoga? One option is to connect yoga with one of the kid's favorite character such as Kung-fu Panda. It is now so easy to generate AI Art with conditions like that using state-of-the-art AI model ControlNet.
ControlNet was introduced in Adding Conditional Control to Text-to-Image Diffusion Models by Lvmin Zhang and Maneesh Agrawala. It introduces a framework that allows for supporting various spatial contexts that can serve as additional conditionings to Diffusion models such as Stable Diffusion.
Training ControlNet is as following:
1) Cloning the pre-trained parameters of a Diffusion model, such as Stable Diffusion's latent UNet (trainable copy), while also maintaining the pre-trained parameters separately (locked copy). It is done so that the locked parameter copy can preserve the vast knowledge learned from a large dataset, whereas the trainable copy is employed to learn task-specific aspects.
2) The trainable and locked copies of the parameters are connected via “zero convolution” layers which are optimized as a part of the ControlNet framework. This is a training trick to preserve the semantics already learned by frozen model as the new conditions are trained.
Example is below:
The Yoga poses to be used as conditions for ControlNet model

The Yoga poses to be used as conditions for ControlNet model

The sample prompt:
"a cute adorable kungfu panda character in japanese garden,physically based rendering,3d blender render, beautiful bright colors, 8k render,best quality, extremely detailed"
The conditions to the ControlNet can be simply added as parameter (in the example is "yogaposes"):
output = pipe(
    [prompt] * 4,
    yogaposes,    
    generator=generator,
    num_inference_steps=50,
)
And the generated images are created under that yoga poses condition that we specify for the generation process.
The generated Panda under the yoga poses conditions

The generated Panda under the yoga poses conditions

About AI Illustwitter, an amazing AI-powered Creative Design App: It so simple and quick to generate high quality, super resolution (8K, 8192x8192 pixels) images within seconds from text description on mobile devices. It can be used to generate AI artworks, nice images for design prototypes, illustrated images for social media posts, news.
There is no limit what kind of images can be generated from text using AI Illustwitter. The artwork can then be shared via AR-vatar and viewed in Augmented Reality.
In addition, you will enjoy the advanced AI-powered Image Editing features to named a few: Generative Fill, Generative Expand, Magic Move, AI Upscaler (upscale to 4X, 16X, and 64X to 8K 8192x8192 pixels images all in one go!), background remover, change background easily using text description and last but not least the magic AI Layering and automatic Repositioning in which intricate image is break down into separate layers and automatically reposition objects, transforming the way we approach image editing. AI Illustwitter is available to download from Apple Store and Google Play. Check it out!
Author: AI Illustwitter

Next article to read:

Texture Overlay

Whether you're a professional photographer or an enthusiast, bringing your images to life might require more than a dozen of image editing tools. That's where AI Illustwitter comes in, offering you the ability to transform your images in many creative and beautiful ways. AI Illustwitter Texture Overlay allows you to apply a wide variety of textures to your photos. Whether you want to add a vintage feel, a rustic charm, or a modern edge, the Texture Overlay tool gives you the flexibility to seamlessly layer textures over your images. This feature uses AI to intelligently blends the textures with your photo, preserving its original composition while enhancing its overall aesthetic. Read more about AI Illustwitter at: https://ai-illustwitter.art

7th December 2024

5 min read

Magic Hair

Want to see how you'd look with a bold new hair color...without the commitment? See yourself with fiery reds, cool violets, or platinum blondes is just a click away with AI Illustwitter Magic Hair tool. Read more about AI Illustwitter at: https://ai-illustwitter.art

5th December 2024

5 min read

PNG 2 SVG

In the ever-evolving world of digital design, efficiency and versatility are keys. One of the most common challenges designers face is converting raster images, like PNGs, into scalable vector graphics (SVGs). Fortunately, AI-powered tools are stepping in to simplify this process, enabling designers to unlock new creative possibiliti1es without sacrificing quality. Read more about AI Illustwitter at: https://ai-illustwitter.art

2nd November 2024

5 min read

Magic 3D

Whether you're creating content for social media, preparing e-commerce product's catalog, or creating marketing materials, a beautiful 3D animation video can significantly impact the perception of your product. The right animation video can catch user's attention for your product. With AI Illustwitter Magic 3D tool, now you can create beautiful 3D animation of your product from a single image...All with a simple click. Read more about AI Illustwitter at: https://ai-illustwitter.art

22nd October 2024

5 min read

Copyright © AI Illustwitter 2024