Diving Deeper into ControlNet
      Back to home
      1. Scenario Knowledge Center
      2. Advanced Image Creation Tools
      3. Diving Deeper into ControlNet

      Introducing Normal Map Mode

      Normal map mode is a setting in ControlNet that extracts compositional information from a provided reference image

      Normal maps are tools sometimes used in the 3D design process whose colors help to reference depth of field. Normal map mode work similarly to depth mode in identifying depth, shadow and distinct lines in an image; the difference being that normal map provides more foreground and background information and softer edges.

      What are good examples of reference images?

           Much like depth mode, normal mapping relies on information such a distinct foreground and background elements, shadow, light, and shading to create it’s mode map. These elements provide a blueprint of what normal map mode should highlight, and what it should keep the same. See some of our examples below:

      Examples of Good References

           Normal map mode is flexible, and responds well to images that detail a background, foreground, highlights, linework, and shadows. The source of light in normal map modes is also taken account of.

      3D artists can also use their own normal map images as inputs by selecting “Disable Mode Mapping” underneath the reference image upload box. This will stop the process of generating a normal map from the reference image, speeding up the image generation process.

      59-1

      Examples of bad references

           There are very few bad use cases or reference images for normal map mode, so long as you are trying to achieve results that align with what it does best. However, normal map mode tends to perform the most poorly with flat, unshaded illustrations. It’s not recommended to use particularly flat or unshaded images. A shaded vector image, for example, may work however its not ideal.

      60-1

      Feature highlights

           Every mode in ControlNet has different features. We’ve shared the primary feature highlights below.

      Very structure and depth guided nuance in composition

      Normal maps are very useful when attempting to maintain very complex and precise shadow, lighting and depth in a reference image. There is heavier influence placed on these specific qualities, and less on detailed linework. It could be thought of as a more complex a nuanced version of segmentation mode.

      34 (1)

      35

      Detailed face structure preservation

           One particular use of normal mapping that sets it apart from other models is the integrity it has towards detailed faces. Unlike structure mode, which retains significant face structure information but no depth, normal map mode is able to retain a similar level of structural coherence as well as depth of field when presented with realistic or 3d style portraits.

           Also, similarly to structure mode, in situations where this is a lot of geography to be mapped, normal map mode prioritizes the reference image coherence over style overrides of the model. This make it possible to use the mode with slightly overfit models if provided with the right reference image.

      In the case of images with complex depth, normal map may not pay as much attention to the model and prompt.

      38 (1)

      39

      Conclusion

      Normal map mode is versatile, and can act differently dependent on the complexity of the image input. As a reminder, a few important points to think about when you are choosing your reference image:

      • Depth of field
      • Details present in your reference image
      • Light source and shading in you reference image

      Thanks for reading, and enjoy creating with normal map mode!