Posts

Main thing with ComfyUI is that it involves making workflows where you connect together nodes, and takes a bit to get used to and you end up learning a little more of how stable diffusion actually works. Also, the UI will look out of date in a fair amount of videos and such on it because they overhauled it recently (and have been making things easier).

If you want an idea of what you are in for on that front, the first screenshot in my first post in this thread was a ComfyUI workflow.

Oh, and my main reason for suggesting Uni_pc as a sampler was just that one of the big things it’s good at is giving good images with less steps. It’s entirely possible A1111 might not have it, as it wasn’t around when it was originally made.

If you do go the ComfyUI route here’s their website and github:
https://www.comfy.org/
https://github.com/comfyanonymous/ComfyUI

If you scroll down on the second one, there’s a link to a portable version for windows.

@Lord Waite
I’ve had it run out of GPU memory before, during larger batches as well as too high resolution for upscaling, and in those instances it just stopped when running out during the batch, or failed at the beginning for the upscale, and gave an error saying out of memory in the status area. But it’s still certainly a possibility.

Right now, I’m unsure if I should pursue Stability Matrix some more (it looked really promising).. in the mean time, I’ll look at things like Forge and ComfyUI directly.

@Thoryn
Even before I went almost totally to ComfyUI, I went to SD Next and Forge and was going to those instead of A1111, so I can’t tell. I’ve been feeling like A1111 is getting out of date, really.

Could be it’s switching over to using the cpu for some reason, because generating on the cpu does take forever. Maybe the GPU was running out of memory…

@tyto4tme4l
Stability Matrix looked very promising, sadly it just freezes (and on one of the four attempts, leaked memory) for me.

With Automatic1111, I get random really long generation times every now and then. Can generate things “quickly” (ie 15 minutes) for hours, then suddenly it takes two hours to even reach 20% completion.. it even happens with only a different seed. It’s as if the model gets stuck, and there’s no step-bro to help it out.

Anyone else seeing this on Automatic? Seen it happen with others?

@tyto4tme4l
Yeah, IIRC, one of the points behind Forge was that it reworked things on A1111 to use code from ComfyUI for some of the backend, because ComfyUI is faster and better with memory.

I’ve got it, but I got used to ComfyUI, and while it might have a steep learning curve, it’s a lot more flexible for things once you know it. (Though, alright, inpainting is still going to be easier on a different UI.)

@Lord Waite
For the record, I’m using RTX 3070 TI with 8GB of video memory.

Good point on the fact that LoRAs also use some memory.. best to avoid if possible.

Don’t have the sampler named Uni_pc on my setup, so I kept it on DPM++ 2S a paired with Karras - as when I did a test with all the samplers (same prompt, seed etc), that was one of the fastest (Euler A was slightly faster and what I’ve been using when experimenting with Automatic a couple times before, but lately I have seen more instances of DPM++ 2S a in the wild, so figured I’d give it a try).

I copied your prompt exactly, and it actually gave a decent composition this time!
At 1024x1024 and 15 steps, it took 13 minutes… maybe this is passable for getting the composition, then I can use img2img to flesh things out? I should start experimenting with the pipeline soon, like only loading one or two LoRAs at a time (especially for kinks and concepts I know the model can’t do at well or at all), figure out grid-editing/prompting, inpainting etc.

score_9,score_8_up,score_7_up,score_6_up,score_5_up,score_4_up,
source_pony,rating_safe,
Flurry Heart lying under Christmas tree,ceiling,window,Christmas decorations on walls,presents,pony,filly,cute,star on top of tree,

Steps: 15, Sampler: DPM++ 2S a, Schedule type: Karras, CFG scale: 6, Seed: 40318147, Size: 1024x1024, Model hash: 67ab2fd8ec, Model: ponyDiffusionV6XL_v6StartWithThisOne, VAE hash: 95f26a5ab0, VAE: sdxl_vae.safetensors, Version: v1.10.1

@Thoryn
If you have a weak GPU, then how about trying out Forge WebUI? It looks almost exactly like A1111, but it should be much faster, especially on a weak GPU. I don’t know about newer versions, but I’m using a release from 02.2024 and it’s working great. I have GeForce 3060Ti with 8GB VRAM and I can generate four 1024x1024 pictures in slightly above one minute. Pretty much no OOM, errors or crashes.
https://github.com/lllyasviel/stable-diffusion-webui-forge/releases

There are also other UIs like ReForge or ComfyUI, I’d recommend testing different options to see what suits you best. Stability Matrix is great for installing and maintaining multiple UIs.
https://github.com/LykosAI/StabilityMatrix

@Thoryn
No problem.

512x512 was fine for 1.5 based models, just XL and newer has moved beyond that. Pony v5 was 2.1 based, so 768x768, and v4 and earlier was 1.5-, IIRC?

What I’ve got is a 3060 12GB, btw, which is probably about as low as you can go and still have 12GB. Though an 8GB card would work as well…

The lora does also add to the amount of memory used. While I don’t think the Flurry Heart one is needed, I will note that my first post was using the Wholesome MLP lora, which is a rather nice art style lora.

You could try changing the sampler to Uni_pc and normal, and lower the steps to, say, 12-14, and see if that speeds things up a bit for you.

@Lord Waite
Thanks for the tips. Have used 512x512 because I have basically the bare minimum of capable hardware (with that res, it With 75 prompt chunks, it usually takes me almost 5 for ~23-25 steps, and 10 minutes for 32-35.. and you’re saying I need to quadruple the res, oof..
(I promised myself not to throw more money at expensive GPUs as I am broke and have stopped gaming, but a 5090 starts to look more appealing the more I mess around with AI).

You’re correct that I’m using Automatic1111 by the way.
(Have pondered alternatives, as the cmd window spews errors left and right even on a fresh and up-to-date version, but it’s the devil I know right now..)

I will avoid LoRAs, clean out the negative prompts (and add things only as needed), up the res to 1024x1024 and do some testing.
Thanks again for all the input, really appreciate it.

No problem. And I can definitely see a couple potential issues.

First, 512x512 is not going to get good results with pony v6. It’s an XL based model, so generally speaking, we’re talking 1024x1024. Other good resolutions are 1152 x 896, 896 x 1152, 1216 x 832, 832 x 1216, 1344 x 768, 768 x 1344, 1536 x 640, & 640 x 1536. That’s basically what XL was trained on.

That negative prompt has way too much in it. “e621_p_low” isn’t actually a tag v6 knows. That was for older versions of pony, and was a precursor to the score tags. Usually, I’d say to start out with a very minimal negative prompt, and add things as needed, and also remove them if they aren’t working. Negative prompts that aren’t needed can actually make an image worse.

You do usually want to have a source tag and a rating tag after the scores. source_pony, source_furry, source_anime, and source_cartoon are the big source tags, then rating_safe, rating_questionable, and rating_explicit.

Not sure the lora is needed, either. You could probably remove it and just say “Flurry Heart”. Also, the point of “star on top of tree” was to try and get it to put the star at the top of the tree in the picture, and by extension, the rest of the tree.

I’d personally try something more like:
score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up,
source_pony, rating_safe,
Flurry Heart lying under Christmas tree, ceiling, window, Christmas decorations on walls, presents, pony, filly, cute, star on top of tree,

with no negative prompt and no lora, and go from there. Definitely keep in mind that the longer a prompt is, the less anything in it actually is weighted. The tokenizer can handle 75 (well, 77, but the other two are used internally) tokens, then after that, the prompt gets broken into 75 token chunks, and that’s the point they start meaning less individually.

(Looks like you’re using something A111 based, so there might be a token count at the top of the prompt entry box?)

@Lord Waite
Thanks for the tip.
I basically only get things like this, where the prompt acts like a toddler and doesn’t listen at all.

parameters

score_9,score_8_up,score_7_up,score_6_up,score_5_up,score_4_up,
Panoramic view of a spacious room with high ceiling,large tainted glass windows,Christmas decorations on walls,Flurry Heart_(Mlp),lying under Christmas tree,presents,<lora:Flurry Heart-Mlp-PonyXL:0.7>,pony,filly,cute,

Negative prompt: anthro,closeup,wip,sketch,blurry,disfigured,bad_hands,badly_drawn,bad_anatomy,watercolor,e621_p_low,thicc,thick,wide_hips,chubby,poofy,hyper,watermark,missing_tail,pillow,bed,couch,sofa,mattress,smiling,standing,sitting,scared,afraid,zoomed_in,cropped,signature,

Steps: 23, Sampler: DPM++ 2S a, Schedule type: Karras, CFG scale: 7, Seed: 582442736, Size: 512x512, Model hash: 67ab2fd8ec, Model: ponyDiffusionV6XL_v6StartWithThisOne, VAE hash: 95f26a5ab0, VAE: sdxl_vae.safetensors, Lora hashes: “Flurry Heart-Mlp-PonyXL: e75f8a2d04d3”, Version: v1.10.1

@Thoryn
I don’t know of any way to actually attach images. I generally upload them to https://postimages.org/ and then use the urls from there.

@Lord Waite
Still not seeing any success, no matter how much I emphasize the look or size of the room.
Is it possible to attach images here? Can only see URLs to enter, but I have nowhere to host them.

It does seem rather temperamental. I generally find it helps if you mention things the view needs to be bigger to accommodate. I had some success with this prompt on pony v6:
score_8, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up,
source_pony, rating_safe,
cute Pegasus Derpy sleeping under a tree, Christmas, presents, ornaments, star on top of tree, window, ceiling,
god rays,

“star on top of tree” and “ceiling” helped clue it in that it should be a further away shot.

(Generally, the format I use for pony v6 models is score tags, source, rating, a general description, then tags.)

Remember trying those two features a year or two ago, and not making it work. Guess I’ll just have to try again.

Another question though, when prompting do you guys use natural language describing what you envision, or comma,separated,keywords,like,this?
A mixture? Varies by model?
And to “save on tokens”, do you use more concise but lesser used words, or favor using more tokens for more words just in case the model wouldn’t understand the lesser used ones?

At this point, I would just put the tree in manually and img2img/inpaint.

Hello all, I’m hoping people can share their tips and tricks with regards to text-to-image prompting, as I am hitting a few hurdles that seem too silly to be a problem, yet they are.

My most basic issue at the moment, is that I’ve been trying to make some Christmas artwork, where the composition in my head is that there’d be a full view of the tree, with packages and a pony at the bottom of it. Sounds easy and very doable.

The biggest issue I’m having, is that no matter what I prompt, it always crops the tree and only shows the very bottom of it.. in the rare instance it gives me a seed with a full tree, it’s pot-plant sized.

I have tried to put christmas tree at the start of the prompt, majestic christmas tree, full view of christmas tree, big christmas tree etc, as well as putting things like zoomed in, cropped etc in negative prompt. I’ve also tried multiple aspect ratios. (1:1, 16:9 and 9:16)

Is there something obvious with my prompting that I’m doing incorrectly? Do I really need to get a LoRA for such a common thing?

Model: ponyDiffusionV6XL_v6StartWithThisOne

I just wanna put this out there, but PonyXL LORAs can be used in illustrious checkpoints as they’re both SDXL. Anthros or feral ponies won’t turn out as good unless that illustrious checkpoint is specialized for the body type, though I’ve used it myself on human/eqg based characters and the image quality does seem to be generally improved compared to other PonyXL checkpoints.

I like this tiny one :3
https://lykos.ai

Here you already install everything in one click and you can use the Stable Diffusion Forge and the Automatic1111. You can install the ComfyUI in the lykos too.

Nightcafe website has been my go to.

Invoke with a direct link to Krita.

Well, since if you want to see ponies by me, you can just click my profile. I will instead take the opportunity to post some none pony stuff I did:

SFW:
Nightelf Warrior
Good morning

Questionable:
Outdoor bath

NSFW:
Worgen

I’m curious which method/UI people are using for image generation. Is A1111 still widely used? Is ComfyUI more popular than Forge? How many are generating locally compared to online?

I’m actually in a hate/love relationship with Pony (SD XL) in general. In my honest opinion, SD 1.5 was so much better and generating detailed backgrounds. That’s why I, for quite a while actually, used both (like here). Back then I used Aniverse/Anitoon a lot in various versions and just used Pony to do the Characters.

But since it was a huge hassle and my pc crashed 50% of the time switching models one the go, I turned to PonyV6 full time. I used that an Autism for a while, got stuck on T-Ponynai3. It has this light “dirtyness” of left in sketch lines. And it can produce so much more detailed background than PonyV6 in my opinion. Here is an example. The light roughness makes it look a little more hand drawn in my opinion. Pics not done, so that is that.

I only use Loras for human chars. Since most people generated are either muscled dudes or near anorexic big boobed girls, I need them to stray away from that. For ponies I use non at all.

I tried out NoobAI-XL v-pred 1.0 yesterday, https://huggingface.co/Laxhar/noobai-XL-Vpred-1.0:

Wildcarded through a fair few combinations of artist styles when generating this image. Judging from that, it seems to have a better understanding of style and texture than the eps-pred version. It also seems to be slightly worse at fine detail, but that may just be a product of its better understanding of style. It needs further testing with artist tags that do highly detailed works to see if that is a limitation of the model.