-
Notifications
You must be signed in to change notification settings - Fork 26.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement new paper: Dreambooth-StableDiffusion, Google Imagen based Textual Inversion alternative #914
Comments
I think we can already consume the resulting ckpt... but to create them you need to use the project you linked to. Not sure if it is worth integrating the creation part in this project when there is a dedicated repo for that. It also require a GPU with 35GB or VRAM or more... so most regular folks don't have access to that. So pretty much a non starter. https://github.com/JoePenna/Dreambooth-Stable-Diffusion/ |
Yeah no point for this gen UI, but for re training that is an awesome find. OP needs to create a comparison though between textual inversion and this to see if there are real advantages across many prompts and image sets. |
The comparison between TI and Dreambooth are a pretty sizable difference with the latter having a major advantage. Also, there's really no need to implement Dreambooth in this. It finetunes the entire model, meaning you simply just replace the default model with the trained one afterwards. There are no embeddings to use here. |
I would love to see the training part of the implementation put into the webui. There is a new memory tweak that just came out that allows the training to run on 24GB of VRAM which really opens things up to a lot of people: See here for setting it up with the memory optimizations: Another example of how well it works: |
Nice explanation of the paper, showing what's possible: |
Please!!!!!!!! |
It's down to 10GB VRAM requirements now. This would be an amazing feature. More info here: |
How much VRAM does this version require?
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: d8ahazard ***@***.***>
Sent: Tuesday, October 4, 2022 2:52:56 PM
To: AUTOMATIC1111/stable-diffusion-webui ***@***.***>
Cc: bmaltais ***@***.***>; Comment ***@***.***>
Subject: Re: [AUTOMATIC1111/stable-diffusion-webui] Implement new paper: Dreambooth-StableDiffusion, Google Imagen based Textual Inversion alternative (Issue #914)
Added a q&d port of the "Optimized-Dreambooth-SD" repo's version for training checkpoints via #1655<https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/1655>.
Still needs to be implemented and added to the UI, but the basic bit to do the work should be there.
—
Reply to this email directly, view it on GitHub<#914 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABZA34RHEC5W2ONAWCNWRFDWBR4IRANCNFSM6AAAAAAQT3OC3U>.
You are receiving this because you commented.Message ID: ***@***.***>
|
The PR here can be run with 8GB using the --medvram flag on launch, but it's VERY slow ATM. Testing with WSL and DeepSpeed to see if I can't make it faster. |
Potentially Related:
|
Is your feature request related to a problem? Please describe.
Only word embeddings are optimized in the current Textual Inversion implementation. But Dreambooth fine tunes the diffusion model as a whole. That's revolutionary.
Describe the solution you'd like
Add https://github.com/XavierXiao/Dreambooth-Stable-Diffusion
Describe alternatives you've considered
Additional context
The training images are obtained from the issue in the Textual Inversion repository, and they are 3 images of a large trash container. Regularization images are generated by prompt
photo of a container
. Regularization images are shown here:After training, generated images with prompt
photo of a sks container
:https://github.com/XavierXiao/Dreambooth-Stable-Diffusion/blob/main/assets/a-container-0038.jpg
Generated images with prompt
photo of a red sks container
:https://github.com/XavierXiao/Dreambooth-Stable-Diffusion/blob/main/assets/a-red-sks-container-0021.jpg
Generated images with prompt
a dog on top of sks container
:https://github.com/XavierXiao/Dreambooth-Stable-Diffusion/blob/main/assets/a-dog-on-top-of-sks-container-0023.jpg
The text was updated successfully, but these errors were encountered: