Runtime error on inference.py

[jmstadt]

Hello, thank you for your excellent work.

I am trying to run inference.py but when I run line 34:

model.load_state_dict(checkpoint["model"], strict=False)

I get a runtime error:

RuntimeError: Error(s) in loading state_dict for Blip2T5:
	size mismatch for t5_model.shared.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).
	size mismatch for t5_model.encoder.embed_tokens.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).
	size mismatch for t5_model.decoder.embed_tokens.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).
	size mismatch for t5_model.lm_head.weight: copying a param with shape torch.Size([64868, 2048]) from checkpoint, the shape in current model is torch.Size([32128, 2048]).

Appreciate any thoughts or guidance. I loaded both the v2 checkpoint and the v2.1 checkpoint with the same result.

[jmstadt]

NVM, per stackoverflow:
model = torch.nn.DataParallel(model)
https://stackoverflow.com/questions/61909973/pytorch-load-incompatiblekeys
Running on CPU, that got me through that line of code.