Memory Management

DeepFloyd IF Lab uses sequential loading of checkpoints to optimize memory usage for GPUs with lower than 24GB of VRAM.

The following memory layouts are available:

• I+II+III (24GB) - all stage models are loaded to memory.
• I/II+III (12GB) - stage I and stages II-II are sequentially reloaded on generation and upscaling.
• I/II/III (8GB) - all stages are sequentially reloaded, only one stage is loaded at time.

By default, the T5 encoder is loaded into the system RAM and uses CPU for computations. It is possible run it on GPU and partially load into VRAM with the --t5-on-gpu command line option, which may save some system RAM. This option implies sequential loading, all other models will be unloaded before the use of T5.