Advanced notebook-based web UI for DeepFloyd IF
- One-click installation on Windows and Linux.
- Support of all four default IF pipelines:
- Dream
- Style Transfer
- Super Resolution
- Inpainting
- PNGInfo
- Convenient batch generation workflow.
- Full control of IF stage parameters.
- JupyterLab environment.
- IF scripting directly in the notebook.
- 16GB of system RAM (32GB or more is recommended).
- 12GB of VRAM (24GB is recommended).
- 50GB of disk space (SSD is recommended).
- Make sure to have a Hugging Face account and be logged in.
- Accept the license on the model card of DeepFloyd/IF-I-XL-v1.0.
- Copy your Huggingface token to the clipboard to make it available for the installer.
Download the install.cmd script and place it in the directory where you want DeepFloyd IF Lab to be installed. Launch the script and wait until the browser window with the notebook opens and downloads all the necessary checkpoints. The installation may take around an hour with an average internet connection and should look as shown in this video.
- Install Git.
- Make sure that Python 3.10.9 is the default Python implementation in your system. Only Python 3.10 is supported.
- Download the install.sh script and place it in the directory where you want DeepFloyd IF Lab to be installed. Launch the script and wait until the browser window with the notebook opens. It may be necessary to open it manually at http://localhost:18888/lab and manually launch the first notebook cell if it does not run automatically.
Download the install.sh script and run it in the directory where you want DeepFloyd IF Lab to be installed. The script will also automatically install brew, python, and git.
Peak DeepFloyd IF Lab GPU memory usage for different sets of models and memory layouts, ±1GB.
| Model set | Stage produced | I+II+III (24GB) | I/II+III (12GB) | I/II/III (8GB) |
|---|---|---|---|---|
| IF-I-XL + IF-II-L | Stage II | 16GB | NA | NA |
| IF-I-XL + IF-II-L + SDx4 | Stage III | 22GB | 12GB | 12GB |
| IF-I-L + IF-II-L | Stage II | 9GB | NA | NA |
| IF-I-L + IF-II-L + SDx4 | Stage III | 12GB | 12GB | 8GB |
| IF-I-L + IF-II-M | Stage II | 7GB | NA | NA |
| IF-I-L + IF-II-M + SDx4 | Stage III | 12GB | 11GB | 7GB |
| IF-I-M + IF-II-M | Stage II | 7GB | NA | NA |
| IF-I-M + IF-II-M + SDx4 | Stage III | 12GB | 11GB | 7GB |
The UI attempts to apply the optimal settings depending on the available amount of VRAM. Please see this wiki page for more details.
| Dream | Style Transfer |
|---|---|
 |
 |
| Inpainting | Super Resolution |
|---|---|
 |
 |
The changelog could be found here.
Q: How can I save the generated images?
A: The images are saved automatically in the notebooks/outputs folder. On most browsers, it is possible to save the generated images through the native browser context menu, available when holding SHIFT while right-clicking on the image.
Q: It prints error messages several screens long. Is it normal?
A: Absolutely. DeepFloyd IF is an experimental library without a detailed user manual, and you are running it in Jupyter notebooks.
Q: It does not work, freezes, or crashes not displaying an error message. Are there any chances to make it work?
A: This may be anything, ranging from bugs to hardware incompatibility. Unfortunately, you are out of luck, because it is impossible to determine what it is exactly.
Q: It suddenly stopped working. What should I do?
A: Please try to restart the Jupyter Python kernel or the application.
Q: It does not work even after I have restarted the application. What should I do next?
A: Please delete the home/settings.json file and the entire venv folder.
Q: I have enough VRAM, but encounter memory errors. Do I need a system upgrade?
A: If your computer does not meet the recommended requirements, it works near the limits of available resources. To run DeepFloyd IF you need as much free VRAM and system RAM as possible. Only the recommended requirements allow to achieve more or less seamless experience.
Q: Can I run this UI on a 8GB GPU?
A: The UI may run on a 8GB GPUs with 12GB of system RAM and the swap of the same size, but it may require constant restarts due to the insufficient memory.
Q: My generations look like halftone prints that were shredded and glued back by the pieces. How can I improve them?
A: Please check the guidance level. It might be too high. As a last resort, there is an option to not pass the prompt to stage III. It is also possible to upscale the results of stage II using different upscaler.
Q: Despite all my efforts, when doing inpainting I can't reproduce the effect of disappearing hat demonstrated at DeepFloyd IF GitHub page. I always get a static image and it looks blurry. Is there a way to improve this?
A: The official demonstration of DeepFloyd IF inpainting is quite misleading. Inpainting always produces a static image, and it looks blurry because this is how DeepFloyd IF pipeline works. It reduces the source image to 64x64 pixels, inpaints there, and upscales it back. Probably there are bugs, or currently we do not know something that will allow us to obtain the same quality, as it was demonstrated.
Q: How do I create an inpainting mask?
A: Currently, DeepFloyd IF Lab has no ability to interactively create a mask just by painting on the source image. It is necessary to upload a black-and-white mask image along with the source image. It is possible to create a mask image by painting over the source image on separate layers in your favorite graphical editor, or by directly transforming the current selection/alpha channel into the corresponding black-and-white image. Some editors have a macro system that allows to perform such operations in a single keystroke. Please refer to your editor user manual.
Q: What are the advanced options for?
A: The advanced options allow to pass any supported argument values to the corresponding stages in the pipeline. For example, if you need to set aug_level to 0.2, specify aug_level=0.2 in one of these fields. The arguments are separated by commas.
Q: How this application is licensed?
A: This repository does not contain executable code derived from DeepFloyd IF and uses it as a library. It is licensed under BSD. Please remember that you may use DeepFloyd IF 1.0 only for personal research purposes due to its own license.