Run Flux Kontext using Huggingface Diffusers in your own GPUs

Run Flux Kontext Using GPU with 16G VRAM or Less

prompt: the scene is exactly the same, but add floral headwear with colorful flowers in her hair, and an elegant white lace collar around her neck

Previously, almost all Flux Kontext samples are from the API, everyone is expecting the releasing of the model weight. Unexpectly, Black forest Labs released the weights of Flux.1 Kontext Dev version hours ago.

black-forest-labs/FLUX.1-Kontext-dev · Hugging Face

If you go using the Python code provided in the above page, high chance you will get Out of Memory error, even with a GPU with 24G VRAM such as RTX 4090. And there is no sample code that enable you to run it with a 24G GPU yet.

After some experiements and trials, I figured out a way to run it in a NVidia GPU using less than 16G VRAM. Before using the code, make sure you have the latest Diffusers installed:

pip install -U git+https://github.com/huggingface/diffusers.git

And install torchao using the your cuxxx version:

pip install torchao --extra-index-url https://download.pytorch.org/whl/cu121 # full options are cpu/cu118/cu121/cu124