torch.distributed kmeans #3876
Closed
torch.distributed kmeans #3876
+488
−43
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
Summary: The previous version required scipy to do the accumulation, which is replaced here with a nifty piece of numpy accumulation. This removes the need for scipy for non-sparse data. Differential Revision: D62884307
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
added a commit
to mdouze/faiss
that referenced
this pull request
Sep 20, 2024
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
mdouze
force-pushed
the
export-D63013820
branch
from
13622db
to
e4d594d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
added a commit
to mdouze/faiss
that referenced
this pull request
Sep 20, 2024
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
mdouze
force-pushed
the
export-D63013820
branch
from
e4d594d
to
0ff90d8
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
added a commit
to mdouze/faiss
that referenced
this pull request
Sep 20, 2024
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
mdouze
force-pushed
the
export-D63013820
branch
from
0ff90d8
to
de832e2
Compare
Summary: The contrib.torch subdirectory is intended to receive modules in python that are useful for similarity search and that apply to CPU or GPU pytorch tensors. The current version includes CPU clustering on torch tensors. To be added: * implementation of PQ Differential Revision: D62759207
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
added a commit
to mdouze/faiss
that referenced
this pull request
Sep 20, 2024
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
mdouze
force-pushed
the
export-D63013820
branch
from
de832e2
to
b4c442d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
added a commit
to mdouze/faiss
that referenced
this pull request
Sep 20, 2024
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
mdouze
force-pushed
the
export-D63013820
branch
from
b4c442d
to
6591d80
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
added a commit
to mdouze/faiss
that referenced
this pull request
Sep 20, 2024
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
mdouze
force-pushed
the
export-D63013820
branch
from
6591d80
to
c96f0f4
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
added a commit
to mdouze/faiss
that referenced
this pull request
Sep 20, 2024
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
mdouze
force-pushed
the
export-D63013820
branch
from
c96f0f4
to
fde0004
Compare
Summary: Pull Request resolved: facebookresearch#3876 Demo script for distributed kmeans. It provides a `DatasetAssign` object and shows how to run it with torch.distributed. Reviewed By: asadoughi, pankajsingh88 Differential Revision: D63013820
|
This pull request was exported from Phabricator. Differential Revision: D63013820 |
mdouze
force-pushed
the
export-D63013820
branch
from
fde0004
to
64d29f3
Compare
|
This pull request has been merged in 838612c. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Demo script for distributed kmeans. It provides a
DatasetAssignobject and shows how to run it with torch.distributed.Reviewed By: asadoughi, pankajsingh88
Differential Revision: D63013820