This guide provides an overview of the IP Adapter's features and detailed instructions on its usage within Scenario.
IP-adapter (Image Prompt adapter) is a Stable Diffusion add-on for using images as prompts. You can use it to copy the style, composition, or a face in the reference image.
With this powerful tool you can create consistency and variations fast.
For example, I used the simple prompt: 'a woman with black hair' on our Platform Model: Creepy Nordic + Belgian + Semi-Realistic to produce the following image:
By dragging and dropping this image onto the 'Reference Image' area of the inference bar, and selecting 'Character Reference' for the Reference Image Mode, at Influence: 50 I simply type the prompt: 'blue eyes':
And just like that, we get a consistent result with the eye color change without any Sketching or use of a Canvas:
Here are some more prompt examples, remember you may need to adjust the influence to get the right balance between consistency and new added elements. The higher the influence the more adherence to the Reference Image will be maintained.
wearing glasses
chef
smiling
Don't forget about our Prompt Expressions for very fine control over the weight of your prompts
(blonde hair)+
By Selecting the Reference Image Mode ControlNet + Character you can further manipulate your results. Here we are using a picture of a dancing man, Character Mode for ControlNet at Influence: 50 and our Reference Image with Character Reference Influence: 50 with the prompt: 'woman dancing':
And here is the result:
IP Adapter is also useful with the LCM Scheduler. You can read about the LCM scheduler here: Using the LCM Scheduler
The LCM Scheduler generates much faster but can result in significant loss of quality.
In our tests, we have determined that LCM can be leveraged with the help of IP Adapters to maintain a consistent subject without sacrificing overall quality beyond an acceptable threshold. Outputs with LCM which previously only maintained 20-30% coherence as compared to the default scheduler, landed much closer to 70-80% consistency and fidelity.
-
As above, add your character image to Reference Image and switch to IP Adapter (Character Reference / Style Reference) Control. This works best with Influence set to around 50.
-
Set the LoRA to the LCM Scheduler and adjust the guidance to 1.5
This article was made using Character Reference. Using Style Reference will allow you to apply the style of a Reference Image to your output. This is particularly useful for color and texture matching.
Face Swap in Stable Diffusion Using IP Adapter & ControlNet