There's been a lot of buzz in r/StableDiffusion about a thing called ControlNET, which lets the user transfer a pose from one image to another. I finally got some time to check it out. I followed the video tutorial here on what to download and how to get started.
If you're new to it too and not getting the best results, know you're not alone and check out some results of my unicorn test.
I started with an AI-generated unicorn I liked. For the image with the pose to transfer, I found this photo of a running horse on Freepik then edited it to face the same direction as the prompt image and to eliminate the background wall, which was causing weird effects in my generated images.
MODULE: DEPTH
ControlNet Module: depth
ControlNet Model: control_sd15_depth [fef5e48e]
This is the "Annotator result" with a "Midas Resolution" of 1173:
A grid I made with Automatic 1111's "X/Y/Z Plot" script that shows the effects of different settings:
MODULE: HED
ControlNet Module: hed
ControlNet Model: control_sd15_hed [fef5e48e]
This is the "Annotator result" with a "HED Resolution" of 300:
Results:
Effects of changing HED Resolution:
Shared settings:
Prompt: a graceful unicorn with a pastel rainbow curly mane flowing in the wind and slender furry Clydesdale legs standing in a field at sunset, nature photography, 35mm, film grain
Negative prompt: floating horn, five legs, extra legs, bad anatomy, deformed, ugly, malnourished, starving, scrawny, 3D render, big butt, cow, tired eyes, dead eyes, bulky, fat, plastic, toy, doll, CGI, airbrushed, painting, digital art, cartoon, oversaturated shadows, oversaturated, saturated colors, blue legs
Hi there @pockymun ! Sure, let´s get down to business!
2. My OC is Kallipsa Akerilis, lady of the darkness. Yeah, Kalli is brutal woman that is great magician and also fighter. She served Evil, as one of it´s leaders. She then joined Leo, leader of Fantasm and Fantars in his quest to protect life. To my protagonist is mentor figure.
I love her, because she´s a strong character that despite everything that happened to her is unbroken. And despite her always having some secret agenda, Kalli does help in her own way. Also she has soft spot for certain people.
(I have her in one of my works, but that one .... It´s more for me.)
6. Now, don´t go for my head but thanks to few well written fanfics I started liking Kakashi/Obito. And Cor/Nyx those two just work together so well.
9. Okay,but there is so many XD .... Cor/Nyx is one of fanon things. But also that many writers and fans, include me, said "Fuck ya Enix, you didnt give us more of glaives interactions or informations about Galahd. So, we are making our own."
I started with what should've been a simple idea for an AI image-- a tattooed sphynx cat-- but it proved trickier and anticipated.
For my initial attempt, I used the "KKW-FANTAREAL V1.0" model. The interface I'm now using is "Automatic-1111".
I was seeing too much questionable anatomy and not enough tattoos.
In general, mixing tattoos and cats mostly got me tattoos OF cats... and whatever the hell that human-armed cat thing is.
After a while, I found the time to try out the "Instruct Pix2Pix" model, which is meant for uploading images and telling it how to change them (ex. "Add sunglasses"). I used my favorite of the previous sphynx images, hoping to quickly tell the computer to slap a cool dragon tattoo on the chest.
Here, I ran into some interesting problems. I'm suspecting a bias towards pictures of humans since trying to use a simple prompt such as "Add a dragon tattoo to the cat's chest" generated plenty of obscene-looking flesh monstrosities featuring the cat AS a tattoo instead of putting tattoos ON the cat. The combination of "chest" and "tattoo" in particular had me scrambling to fill up that negative prompt field to try purging human anatomy from the equation. I tried "shoulder", "under the neck", "below the neck", but that didn't get me the cool chest tats I wanted. Sometimes the cat's ears mutated into arms too.
These are some of the less horrific instances of cat-as-tattoo from this batch:
Even when the results were more reasonable, I still wasn't getting much of what I wanted:
So, I decided to just go back to FANTAREAL and inpainting. That proved much more effective:
I expanded my inpainting to include more:
Sometimes, some human BS still tried to sneak in...
All my Drawings so far, you may found them at
https://www.deviantart.com/lrdalucard or my instaram https://www.instagram.com/lrdalucard/
You may also find my social media on:
https://www.multi.link/lrdalucard