In this tutorial we will walk you through one way you might approach training a custom generator using the Scenario webapp. This tutorial is a great starting point for beginners who are just getting used to training custom models.
In our recent Concept Art training guide, we went over three main categories your custom generator might fall into. We recommend you review that guide, but as a recap those categories are style, subject, and hybrid.
This illustration guide will be focused on style training. That means you should think of the custom generator you make from this as being a zoomed out look at the art world you are building. Illustration is one of the best all purpose regularization classes to use, as it is very general. Feel free to read more about regularization classes here.
Gritty Comic Style
This tutorial is going to focus on training an illustration custom generator on a gritty comic book style. We’ve provided a link to our dataset and welcome you in following along in our training. Or feel free to follow along using your own art style.
Curating the Dataset
When you take a look at the dataset provided, take a moment to notice - what is the similarity in each image, and also, what makes that image different from the rest of the dataset?
In this particular gritty comic book style, there are a few things that are similar throughout the set. You may find more than this, but what immediately jumps out are:
- The intensity of color and a distinct general color palette
- Weight of the linkwork and the movement of the lines
- Heightened ‘dark’ energy in the scenes - they are dramatic
- A fantasy aesthetic that’s hard to place in a particular time
What makes each image distinct and different is also important. What we notice here are the range and difference in:
- Featured subjects of each scene.
- The setting of the images.
- While the colors are consistent, they do not all have the exact same sample of the color theme.
- The composition, similarly, has shared aspects from image to image, but ranges significantly.
As a general rule the things that are shared throughout a finetuning dataset are prioritized in the training, and the things that differ are not. This isn’t a perfect rule - there are exceptions - however it is a good starting point and rule of thumb.
You will also notice that there are 16 images in this dataset. Although quantity is very important, particularly for style and concept training, it is equally important what you put in. A dataset with fewer better or more diverse images is better than a dataset with a lot of images that share too many similarities. The more overall style and subject consistency there is in your dataset, the less images you should use.
The minimum number of images you should use is five. However, we don’t recommend going below eight, as your images are more likely to underfit the fewer training steps you have. It will depend on what you are training, but somewhere between 10-30 tends to be ideal.
It is important to remember that you may need to refine and retrain your dataset, particularly when you first start using the program
Create Your Generator
Once you have your dataset, you’ll be ready to create your generator. You will need to go through the following steps:
- Go to Home > Create a Generator
- Upload your images to Add Training Images. If you need to crop them, do so at this step
- For this model we will not remove backgrounds from our images
- Click Next when you are ready
- You will not be prompted to choose a name and a regularization class. In this case we will be naming the model Drkcomic and picking Art Style > Illustration as our regularization class
- Click next. You will see the next page which talks about learning rate and text encoding. You do not need to change anything here, and you may launch the training program.
It should take anywhere from 20 minutes to around 2 hours for your generator to train. Check back in a little bit!
Test Your Generator
Once your training is complete, it’s time to test your outputs. Follow the guide we’ve created in our document on How to Prompt. If you are happy with your work, proceed to the next step. In some cases, you may find you want to adjust the outputs. You are always welcome to add or remove images and retry.If you are not happy with your output there are a few ways to proceed.
First, you may decide to use an imperfect generator with some extra prompts to create more nuanced images for your dataset. You can use these images to replace or add to your original dataset when you train a new model.
Your other option is to remove images from your original dataset that you see are showing up too often. It is easy to identify these if, when you generate images without additional prompts, you see one or two aesthetics or subjects from the dataset coming through multiple times. That indicates a need to prune and retrain.