Posts

@Scarlet Ribbon
Okay, interesting.

In your opinion, what sampler is best to use when creating pony images in Pony Diffusion? On CivitAI, the default appears to be DPM++ 2M Karras.

@MareStare
Okay.

Another question - generally, how many steps for generation produce the best results?

@Zerowinger
You can use full sentences to describe the prompt with Pony Diffusion as well. Citing from their page on civitai the recommended prompt format:

score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, just describe what you want, tag1, tag2

where tag1, tag2 are simple words/word combinations similar to derpibooru tags like “unicorn, blushing, trio, duo”, etc

@Zerowinger
Different models are trained in different ways, leading to some models being better for natural language, and others better for tag-based prompting. Pony doesn’t completely fail with natural language prompting, but in my experience it performs much better with tag-based. If you add source_pony to your prompt, you can damn near just use Derpi/Tanta tags to get most of the results you’re looking for.

@MareStare
So basically, including that string is necessary for higher quality images then? What about the rest of the prompting? On Imagen, I’m used to using full sentences and phrases to describe exactly what I want the output to be, with Pony Diffusion it seems the go to format is to list each individual aspect as a prompt separated by a comma.

@Zerowinger
The score_* tags are specific to Pony Diffusion. Their original idea was that you’d be able to write score_7_up tag only (just a single tag), and you’d get an image based on the dataset of images of quality 7 or higher.

However, the way this was implemented during training was wrong, and completely broken. The developers discovered this bug only in the middle of training, at which point fixing that bug would be too expensive (they’d need to restart training from scratch again, which would cost them potentially several tens or even hundreds of thousands of dollars). So, they kept the bug, and made a guideline to include that lengthy score_9, score_8_up, ... etc. string at the start of the prompt to work around it.

There is more detail on this training fiasco in this article: https://civitai.com/articles/4248/what-is-score9-and-how-to-use-it-in-pony-diffusion

So, I tried out Pony Diffusion on Civitai to some success, and part of the prompt was copy-pasting the score_x score_up prompts that I had seen elsewhere. However, I’m a little confused as to exactly how those prompts work, the whole text to image format is very different to the style I’m familiar with.

Could I get some insider info on just exactly how this format in Pony Diffusion and similar checkpoints works?

@derp621
👍

It might be the case that part of the orignal prompt is what is messing up the cutiemark. In these situations ADetailer allows you to add a custom prompt and/or negative prompt to the cutiemark areas it detects

@FoalFucker
Unfortunately, ‘nothing’ isn’t specific enough for me to find anything useful with how my ‘script’ is organized. Even the characters are split across multiple files.

One of these days, I’ll convert my mess into a single JSON, but I just find it way easier to maintain as separate files. It is currently 740 separate wildcard files.

@Sunny
Because if I’m going to generate stuff that’s ‘not good enough’ to do all the manual editing on, but still interesting enough to share with my friends, I’d rather it look better than worse.

@Scarlet Ribbon
Nothing really, just anything that is specific to ponies/mlp/derpibooru, of anatomy, characters, copyrights, artists, etc. I am going to go through and do it myself for PDXL, I just figured I’d see if anyone had anything they thought was worth sharing.

@Sunny
With the Noob based models at least, it’ll get the cutie mark right some of the time, and if it gets it wrong, then ‘detailing’ it (masked inpainting) will usually get it right after a few tries if it understands the character. For more obscure characters this doesn’t work. For example, with >>34015 I had to manually edit the cutie mark in and then masked inpaint it a few times to blend it in.

@FoalFucker
What specifically are you looking for? I have extremely extensive wild card stuff, but it is primarily smut oriented and it is also a convoluted mess that is designed to work with a specific workflow that I maintain.

Almost everything with my prompter tag on this entire site uses that workflow.

Still, if you have specific things you’re seeking, I can probably fish out something more generically useful.

@Thoryn
I have the same GPU. I can generate a 1024x1024 image in Comfy UI in less than 15 seconds. I don’t know what was up with automatic1111, but I was getting similarly glacial performance on it.

Strongly recommend you just get rid of it and learn a different front end.

I don’t really see the point of an ‘ADetailer’ for cutie marks when you could just use an editing app to add the correct cutie mark manually. I get the idea of using an ADetailer for things like making horse pussy more correct / accurate, but why make a cutie mark-specific ADetailer LORA if the AI is going to keep messing up the cutie marks anyway? I can understand the usefulness of using an ‘ADetailer’ for cutie marks in 3D images, but it doesn’t seem as necessary for 2D ones.

@derp621
I do the same process as adetailer a lot, except for the automatic masking. The models themselves don’t actually change anything, they just automatically mask areas, which is then used for low-denoise only-masked inpainting.

Anyone else using ADetailer in there image generations?

The out of the box yolov8 face detection model is decent at detecting and improving faces, even pony/cartoon ones.

There are pony specific models as well:
https://civitai.com/models/979007/cutie-mark-adetailer
https://civitai.com/models/608504/adetailer-horse-pussy

@MareStare
Great idea! Img2img and inpainting are invaluable tools and it would be useful to describe them in detail. Things like denoising strength and the difference between “Whole picture” vs “Only masked” for the inpaint area are extremely important here.

@Thoryn
I used Photoshop for drawing the scribbles, and then moved the images to Forge UI for inpainting. I’m planning to describe some of my learnings and creative process in a shared guide website. I’ll post about it on tantabus discord and create a forum thread on tantabus when it’s more-or-less ready. I’d like to collect all the tips and tricks and organize them on a convenient medium for beginners study.

@MareStare
Really cool to see WIP steps like this and have them explained.

And your sketching abilities are way ahead of mine. :p

What program do you use for SD?
Do you handle all the painting in it, or do it elsewhere and move it over to SD?

Getting the composition as you want may be too big of a work for text2img. I recommend you to try inpainting with a colored scribble (coloring is important to make AI get the colors right).

For example, this is how I got Fluttershy inplanted into the scenery of this image:

Yeah, you can tell my drawing skills aren’t that good, but Zoinksnoob nailed Flutty almost immediately after I pasted it there and inpainted that area with a denoising strength of something like 0.7+. Sometimes it takes several iterations of drawing a scribble, then letting inpainting improve the detail, and then improve that more detailed version with some lighter scribble to get things exactly as you want.

Two times that I’ve done Tirek - one just as himself, and one in cosplay:

@Sunny
Yes, tried 3 or 4 of classroom LoRAs, and played around adjusting LoRA weights and prompts with them.

Typical problems I had with them, would be thatit insisted on the pony being at the very forefront / focus of the image and taking up half the image, very stretchy pony bodies, wrong size in relation to the room, messed up wall corners and such, pony bodies morphing into the desks, getting desks to look coherent (size, legs, also problem morphing into each other)

It can probably be fixed with extensive inpainting or something, but I should be able to get decent results with just prompting correctly, even without using LoRAs. (Most of my time on this has been spent without them, and I’ve gotten some half-decent results.)

Thinking I should probably look into ControlNet, but am uncertain if it will detect and adjust equine poses correctly.

@Thoryn
Have you tried using a LORA for a classroom?