Line art mode is a setting in ControlNet that extracts compositional information from a provided reference image.
Line art mode is intended to act as your ideal color assistant, identifying the general linework of an image and reimagining the color, shading, and style based on your prompt and model.
What are good examples of reference images and models?
Line art is at it’s best when used on cartoon, anime, or hand drawn style models. Although, as you will see in our examples, it is possible to get results from other styles as well. This is because line art was primarily trained on cartoon styles, and we have chosen this particular model due to the reminiscence between the color and shading of cartoon/anime and many game assets.
There are very few bad examples of reference images that a person can use. A great number of varying inputs work with line art mode, so long as designers know that unlike scribble mode, line art stay incredibly true to the original linework details in the majority of cases. It looks to amplify the linework present in an image and color it in. Sometimes it seeks to simplify the lines and edges for a more enhanced style.
Choosing the appropriate model and parameters
Line art is one of the more complex ControlNet tools. When it is leveraged with intention, it can be a powerful shading and coloration tool transforming a great number of images into complex and beautifully illustrated designs.
We recommend models that fall in the following categories for the best results:
- 2D hand drawn style game assets
- Painterly or oil painting style portraits and character design
- Anime and classic JRPG illustrations
- Watercolor style designs and environments
While other styled models most certainly can work with great success using Line art, we have found the most consistent results with these styles. Se for comparison below a 2D render style robot warrior and a oil painting portrait style illustration.
Leveraging influence rate for higher success
Although not all modes benefit greatly from adjusting the influence, we find that for some style this creates the best results. In particular, styles that are outside the typical scope of the line art training (3d, photorealism)
or circumstances when a designer is attempting to prioritize the style of their model over the style of the reference image.
As you can see, adjusting the influence between the 50-100% threshold can produce beautiful results, while still adhering to the integrity of the original linework.
Feature highlights
Every mode in ControlNet has different features. We’ve shared the primary feature highlights below.
Complex and nuanced coloration and shading
Line art mode is truly an illustrators best friend, particularly when working on tight deadlines with limited resources. It is intentionally designed to be able to understand the nuance of the image input it is given, and create a vibrant, color rich image.
Reskinning characters in a new style
It is common practice to reskin existing games and mechanics into new art styles, or for simpler platforms. Likewise, character and environment concept art might be originally imagined in one style, and quickly need to be redesigned in another.
Line art mode is useful for assisting this change in art direction. As you can see below, the less strength applied by the line art mode, the more additional details it may begin to add.
.png?width=688&height=313&name=grid%20(4).png)
Turning photorealistic or 3d into illustrated assets
Line art is also excellent at pulling from more complex images, such as user profile images, and re-imagining them as stylized portraits. It allows for the prompt an model to give more direction than a tool such as structure mode, while giving far more direction than scribble mode, depth mode or pose mode.
.png?width=688&height=313&name=grid%20(5).png)
Conclusion
Line art mode is ideal for specific use cases, and brings designers to the next level when leveraged correctly. To review, when you are using pose mode, take into consideration information about your refence image and model choice:
- Is your use case one that works well with this tool? (2d, avatars, portraits, anime, classic game art)
- The natural lines of the images will hold the most weight.
- Line art mode will draw out the boldest lines it sense from photorealistic or 3d images.
- Finished line art is better than simple scribble art, as it will stay very true to the linework it is presented with.
- It is ok to adjust the influence of the line art mode.
Thanks for reading, and enjoy creating with line art mode!