0% found this document useful (0 votes)
507 views17 pages

Stable Diffusion Prompt Engineering

The document provides an introduction to generating images with Stable Diffusion through Dream Studio. It discusses the basics of prompting images and describes the different settings like width/height, CFG scale, steps, sampler, and seed. It provides examples of how changing these settings can impact the generated images and notes that experimentation is needed to find the best settings for different prompts. The goal is to help users understand the capabilities and possibilities of Stable Diffusion while acknowledging there are no perfect settings and the creative process involves trial and error.

Uploaded by

granpepper
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
507 views17 pages

Stable Diffusion Prompt Engineering

The document provides an introduction to generating images with Stable Diffusion through Dream Studio. It discusses the basics of prompting images and describes the different settings like width/height, CFG scale, steps, sampler, and seed. It provides examples of how changing these settings can impact the generated images and notes that experimentation is needed to find the best settings for different prompts. The goal is to help users understand the capabilities and possibilities of Stable Diffusion while acknowledging there are no perfect settings and the creative process involves trial and error.

Uploaded by

granpepper
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

1

Introduction

We will go over a variety of different subjects. Starting, of course, with the basics. This way you
can understand the process a bit better if you’re completely new to generating Stable Diffusion
art through Dream Studio. Later on you will see some examples of different image sizes, CFG
scales, prompts and more. To give you a little bit of an idea of what you can expect.

Some of the images in this guide have been created when Stable Diffusion still worked through
the Discord in beta 1, and others have been made through the website Dream Studio. This has
no difference, since both work with the same model. The Dream Studio is simply a new way of
interacting with the service.

Please keep in mind that, even though we go over settings and prompts in this guide, there is
no one answer that fits all. Many times you think you’ve found the perfect settings, e.g. CFG
scale, steps, image size… But then you write a different prompt and all of a sudden those
settings create very bad results. Even though this guide is meant to give you an idea of
possibilities with Stable Diffusion, and to some extent some tips and tricks. Please know that
we are all still learning, and it’s very much a case of experimentation to see what works best for
certain styles/prompts/outcomes.

(I am not affiliated with Stable Diffusion / Stability.ai, but simply a curious nerd who wanted to share his
experience and possibly provide some useful info. I hope you enjoy it.
Happy creating! - You can reach out to me if you got any questions or suggestions, MrH3RB#9625 on
discord, @MrH3RB on twitter, or @CaptainH3RB on reddit.)
2

Why Use Artificial Intelligence?

These tools are still quite new, and can even be a little scary at times. Will traditional artists lose
their jobs? Is art even art when a computer makes it? What’s the point even, type in some
words and the machine spits out an image…

Well, I don’t have all the answers. But yeh, it can be scary. No I don’t believe artists will lose
their jobs, if anything these new tools will bring the ability to adapt to new ways of creativity
that haven’t even been thought of before. And what is the point of all this? Creating with the
machine? Well it’s a lot more fun doing it, than it looks like when you just hear about these
things. We mostly see text to image tools like Stable Diffusion, DALL-E 2 and MidJourney. But
this is about so much more than just making images.

We are at a state in life where the ability to create is about to change massively. And that is just
the beginning. These AI systems are capable of much more which we will all witness soon
enough. But let’s not get ahead of ourselves.

Not everybody is able to create in the traditional way. Whether that is drawing or painting, or
making pottery or anything else. These new tools will make it so that millions and millions of
people all of a sudden have the ability to create what they have in mind, without a high barrier
of entry. Or maybe they just couldn’t because of a physical condition, and now there is this
new tool that brings the opportunity to still get their amazing thoughts into some form of
creativity.

I’m not here to preach, but I did want to write this little part into this guide. Because I strongly
feel that there is no need to compete. No need for either or, as in only traditional art or only ai
art can prevail. Since I’m someone who tried to learn traditional art, but can’t because of
physical conditions I can somewhat see both sides of the arguments being made online. And I
think we can all get along and create beautiful things together.
3

The Basics

Generating images in Dream Studio might seem complicated, with all the adjustable sliders. But
it isn’t actually that difficult.

Prompting your images

At the bottom of the screen you can fill in your prompt, after which you hit the “Dream”
button, and Stable Diffusion will start generating your image.
4
Generating images through Dream Studio works with a credit system.

The default settings will cost you 1 credit per generated image.

Default settings:

Changing the amount of steps and / or the Width / Height of your image will cost you
more credits. You can adjust the CFG, Sampler and seed without additional costs.

See the graph below for more information:


5

What are the different settings?

Let’s go over the settings briefly. Later on we will look at some generated images with different
settings to see how they affect the images in more detail.

Width & Height:

This one is pretty self explanatory, so we will move on to the next setting.

CFG Scale:

The CFG scale basically tells Stable Diffusion how closely it has to follow your prompt input. You
can see some examples of this later on. As default this is set to seven (7), which is fine for most
prompts. You can experiment with this, because with some prompts it might generate a much
better result when you put the scale a bit higher. Do note however that increasing the CFG scale
might cause artifacts in your images. This, again, does depend on your prompt. You can see
examples of this later on.
6
Steps:

Increasing the amount of steps tells Stable Diffusion that it should take more steps to generate
your final result which can increase the amount of detail in your image. Higher steps does not
always equal a better result. Especially when using a huge amount, like 100-150 or even higher.
Just like the CFG scale, this does depend on your prompt, but going too high with the steps can
cause artifacts in your final result.

Do note that increasing the amount of steps will cost you more credits per image. (see the graph
on the bottom of page four (4).)

Number of Images:

Increasing the number of images will create more images with the same prompt you have typed.

Sampler:

The sampler is what Stable Diffusion uses to decide how to generate your final result. You can
change this, but the differences between samplers are often very small. But again, it depends a
bit on your prompt.
7
Seed:

The seed determines the starting point of your image generation. Normally a seed is randomly
chosen when you hit the “Dream” button. But if you find something you like, you can use the
same seed to get the same image again. Do keep in mind that to get the exact same result as
before you need to not only use the same seed, but also the same settings and prompt!

Something that is fun about this seed system, is that you can change some small things about
your prompt while keeping the same seed, and thus generate very similar looking images. You’ll
see some examples of this later on.
8

Width & Height Examples


You can experiment with the width/height as much as you want.
Just keep in mind that changing this will change your composition, and costs extra credits.
9
Another note on the image size:

The AI has been trained on countless 512x512 images. Causing it to sometimes create doubles
when you try to generate an image that has a different size. Like a neck on top of a head, as the
image here:

The explanation from the developers has been that, because the AI has been trained on images
that were 512x512, it will try to generate your prompt in every 512x512 area. Meaning, if you
ask for a portrait it will try to create a portrait in every square with the above mentioned
dimensions. Whatever your prompt is, you might run into duplicates or odd images like the one
above when you go above the default image size.

This does not always happen, but it is something to be aware of. Especially because bigger
images cost more credits. (see the graph on the bottom of page four (4).)

Composition does change based on your image size, so if you have another composition in
mind it could be good to experiment with other sizes than the default 512x512.

If you simply want to generate bigger images for quality purposes I’d advise against that given
the extra cost. In that case it’s better to just create with the 512x512 settings and upscale the
image with a service like BigJPG, or a google colab notebook or whatever upscale service you
prefer. Many of these upscale services use AI, and have barely any quality loss, if any.
10

CFG Scales & Sampler Examples.

As said before, the CFG scale basically tells the AI how closely it has to follow your prompt.

Below you can see the same prompt with eight (8) different CFG scales through all the eight (8)
samplers.You can also find the comparison images here.

Something to note: All of these squirrel, still life and bottle images have been created with the discord bot when it
was still operational. The step cap was 50 at that time. Which is a little unfair to some samplers, because they work
best with much higher step amounts according to the developers.(Especially ddim and plms, hence why those results look
so bad in these comparisons. They apparently work best with around 1500 and 200 steps respectfully)
11
12
13

Generation Steps Examples

It might seem logical to always run at the maximum amount of steps, but this isn’t always a good
idea. Often you don’t see that much of a difference when running your steps higher than 70-100,
depending on your prompts. And in most cases, for simpel images, 50 is plenty. Link to image.
(CFG scale was set to 15)
14

Random vs Same Seed Examples


When you keep the seed random you will get random starting noise, resulting in different
images. But if you want to adjust an image you like, you can keep the same seed as explained
on page seven (7).
15

I’ve only changed two things, yet the images are quite different. (Removed “vibrant” from the
prompt, and lowered the CFG scale from 35 to 25.)

Experiment a bit with the seeds, and your prompts. You can get amazing results when you
change things around a bit. A tip, don’t change too much at once. The more variables you add,
the harder it is to actually find out what created the image you so enjoy.

Change one thing, hit Dream, reflect on the changes, and repeat. That is how you learn!
16
I hope this guide helped you create better art, or at least understand the tool a bit more. Below
are some other resources you can take a look at! If you have any questions, suggestions or
want me to add something to this guide, let me know.

Go create something: beta.dreamstudio.ai

Other Resources
- Stable Diffusion Akashic Records
- Lighting prompts comparison
- Stable Diffusion guide for non-artists.
- Stable Diffusion ultimate beginners guide.
- Dream Studio tips: Maximize your credits.
- DALL-E 2 prompt book

You might also like