Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about how to detect events in video #36

Open
xiaokj37 opened this issue May 7, 2024 · 2 comments
Open

Question about how to detect events in video #36

xiaokj37 opened this issue May 7, 2024 · 2 comments

Comments

@xiaokj37
Copy link

xiaokj37 commented May 7, 2024

In the paper, author mentioned using DPC-KNN to segment the video to get multiple events, and then process each event separately.
I would like to ask where this part of the code is implemented, I don't seem to find it.

@jpthu17
Copy link
Member

jpthu17 commented May 22, 2024

I am sorry to reply to you so late because I have been busy with other projects recently.

The relevant code is below, we are currently trying to improve this part of the code and will update our code base in the near future.

cls_features = torch.mean(image_features, dim=1, keepdim=False).unsqueeze(0).clone()
token_dict = {'x': cls_features,
'token_num': cls_features.size(1),
'idx_token': torch.arange(cls_features.size(1))[None, :].repeat(
cls_features.size(0), 1),
'agg_weight': cls_features.new_ones(cls_features.size(0), cls_features.size(1),
1),
'mask': None}
down_dict, token_dict = self.get_model().ctm3(token_dict)
events = OrderedDict()
max_len = 0
for id, i in enumerate(down_dict["idx_token"][0].tolist()):
if i not in events:
events[i] = [id]
else:
events[i].append(id)
max_len = len(events[i]) if max_len < len(events[i]) else max_len

@xiaokj37
Copy link
Author

Thanks for your reply.
I am now trying to reproduce the results of the video event slicing given in the paper. You have given the visualization code for the image, is it possible to give the processing code for the video and the corresponding reference video.
I'm waiting for your early reply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants