Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Add ARM Support and Apple MPS Acceleration #594

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

crackerben99
Copy link

Add ARM Support and Apple MPS Acceleration.
Testing:

import torch
from PIL import Image

from lavis.models import load_model_and_preprocess
from lavis.processors import load_processor

raw_image = Image.open("xx.jpg").convert("RGB")

device = torch.device("mps")

captions = ["x", "xxx"]
model, vis_processors, text_processors = load_model_and_preprocess("blip2_image_text_matching", "pretrain", device=device, is_eval=True)
img = vis_processors["eval"](raw_image).unsqueeze(0).to(device)
for caption in captions:
    txt = text_processors["eval"](caption)
    itm_output = model({"image": img, "text_input": txt}, match_head="itm")
    itm_scores = torch.nn.functional.softmax(itm_output, dim=1)
    print(f'{caption}:  {itm_scores[:, 1].item():.3%}')

Copy link

Thanks for the contribution! Unfortunately we can't verify the commit author(s): guobentian <g***@s***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Salesforce Inc. Contributor License Agreement and this Pull Request will be revalidated.

Copy link

Thanks for the contribution! Before we can merge this, we need @chatgpt-1 to sign the Salesforce Inc. Contributor License Agreement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant