is there a better guide for BLIP or CLIP pretraining?

[simplelifetime]

Thank you for your great work! I was wondering how should implement image-text pre-training for CLIP or BLIP, which seems unclear in the project readme files.

[dxli94]

Thanks for your interest in LAVIS.

• please refer to run_scripts/blip/train/pretrain.sh as the entry point for pretraining BLIP. The configuration file is lavis/projects/blip/train/pretrain_14m.yaml.
• before pretraining, you may want to prepare datasets. Please refer to https://github.com/salesforce/LAVIS/tree/main/lavis/datasets/download_scripts for downloading datasets.
• to better understand how training in LAVIS works, please refer to https://opensource.salesforce.com/LAVIS//latest/tutorial.training-example.html for an example.
• Your feedback is well received. We've noted it down and try to come up with more documentations on this.
• We have not tested CLIP pre-training.

Thanks!

[simplelifetime]

Thank you for your reply. And i notice that the notebook files broken in https://github.com/salesforce/LAVIS/tree/main/lavis/datasets/download_scripts/DownloadConceptualCaptions I'm not sure if it's my problem or it's actually broken

[dxli94]

Hi @simplelifetime, thanks a lot for well-spotting. I've removed incorrect license headers in the notebook. Can you pull and see whether it works as expected?

Thanks.

[simplelifetime]

It works fine now, thanks for your support!