-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ModuleNotFoundError: No module named 'ChatUniVi.model.language_model.phi' #49
Comments
I added phi2 code, but the code seems to have bugs. See https://github.com/PKU-YuanGroup/Chat-UniVi/tree/main/ChatUniVi/model/language_model I hope this helps. But the code seems to have bugs. The phi2 model often hangs when attempting to amalgamate training images and videos. I guess this error comes from the deepspeed bug (microsoft/DeepSpeed#2223). |
Thank you very much, I'll try it first. By the way, IS the training of the following configuration correctly? the current training comes from the video training data collected by itself, using zero3_offload. deepspeed I found that if I follow finetune1, the loss hardly changes like the pink curve (set to true tune_mm_mlp_adapter), but with finetune2, the loss will shake violently at the beginning of training, is this normal? |
It's not normal. Typically, the LLM loss is expected to range between 0 and 5. However, I've noticed that your reported loss has peaked at 10, which suggests that there may be an issue affecting the training process. |
hi i would like to use phi because of the limit of gpu memory. But got the problem describe in the title. Can you share the phi.py file?
The text was updated successfully, but these errors were encountered: