Enhanced Image Generation With Stable Diffusion and Roop.

In this post, we will go through the installation process and usage of stable diffusion for text-to-image or image-to-image generation.
text-to-image generation has to do with generating realistic images by using just text prompts, while image-to-image takes the game a little bit further by using a sample image in addition to test prompts to generate ultra realistic images that has features of the sample image.
Adding roop AI model to the picture further enhances the art by improving the quality of image generated to a perfection level. we will see how shortly, so let's started.

Install Stable Diffision

First, we need to install AUTOMATIC111 for stable diffusion on our system. i will be going through with the installation for mac, but not to worry windows and linux has similar process, you only need to use their respective package managers as AUTOMATIC111 runs solely on the terminal with a web ui on the browser.

Step 1: Install homebrew package manager on mac
If homebrew is not yet installed on your mac, please open a command line like iterm2 and run the commands below to install homebrew.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Step 2: install additional packages
After the installation completes, we need to install several other packages like python(if not installed), cmake, protobuf, rust, git, and wget. run the command below to install them.

brew install cmake protobuf rust python@3.10 git wget

Step 3: Clone AUTOMATIC111 Repo
To make AUTOMATIC111 work on the local environment which is our macbook, windows or linux, we need to get a copy of it saved locally. we can achieve that by cloning it. cd into the home directory (cd $HOME) and run the command below to do that.

cd $HOME && git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui

Step 4: Download the model file.
Once we have gotten a copy of AUTOMATIC111 for stable diffusion, next would be to download the model. please note that it is about 4GB of data, so you might want to make a coffee while it downloads.
v1-5-pruned-emaonly.ckpt

Step 5: Move models file to model directory
When download is completed, you need to move the model .ckpt file to stable-diffusion-webui/models/Stable-diffusion folder which is inside the AUTOMATIC111 we cloned.

After doing so, the folder will contain two files as above.

Step 6: Download Roop model
Download Roop inswapper model and place it same way as above but in the models/roop folder. when done the folder will look as below.
inswapper_128.onnx

Finally: Run AUTOMATIC111
To run AUTOMATIC111 and load the web ui in browser, enter the command below. please note that this might take a while on the first run as there are additional packages to be downloaded.

cd ~/stable-diffusion-webui
./webui.sh --precision full --no-half --skip-torch-cuda-test

please skip the additional options if you are using a GPU enabled device. just run

./webui.sh 

the image below on the terminal..

indicates that the webui has been loaded successfully.

finally we can start generating some cool images. let's enter a simple prompt "image of monalisa in a wonderland, photo realistic, colorful, hyper sensitive with birds and windy background".
enter the prompt in text2img box and click on generate button. then relax a bit while our image gets generated.

After some minutes, our image gets successfully generated as you can see. Did pretty well for a painting right?, it also took into consideration the elements in the prompt. but we can do much better.

Adding Roop to the picture
This is where Roop comes into play, to generate high quality content, we need to enable roop extension and use img2img to feed in a sample image we can use for the generation.
click on "extension tab", then "install from url". enter github repo link of roop sd-webui-roop and click "Install".

next click on img2img select an image, scroll down and click on roop. upload yet another image also make sure "enable" is checked then enter some prompt and click on generate. you will be amazed at the result. i generated the image below following the steps with a simple prompt "as a pirate".

Besides, the two great men in the source photos are Salva and Julien. my mentors at ALX/Holberton school. I don't even know if doing this is legal 🤭.

Conclusion
Roop with stable diffusion brings a whole new level to image generation with AI, there's a lot of options you can try to see which works best...like selecting a sampling method, batch count range, an upscaler and so on. I encourage you to check them out.
Also note that with AI there's always room for improvement and new things to learn.

Thanks for reading 🤗.