Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ED batch processing #102

Open
mickvanhulst opened this issue Jun 18, 2022 · 0 comments
Open

ED batch processing #102

mickvanhulst opened this issue Jun 18, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@mickvanhulst
Copy link
Collaborator

ED mini-batches currently consist of single documents. As a consequence of this, GPU utilization is limited as the number of mentions per document varies and can, at many times, be small. A potential line of improvement is to:

  1. Introduce an additional dimension for the number of documents that are processed per mini-batch. This would result in the following dimensions for a given mini-batch: (n_documents, n_mentions, n_features).
  2. This will require the dimensions to align across documents, meaning that padding will be required for the n_mentions. We need to investigate how much padding would roughly be required and what the variance is across the number of mentions per document.
  3. During training, it is essential that there is still randomness across the batches, so grouping documents by their number of mentions is assumed to be suboptimal. Now, during inference, this is no longer an issue. As such, if our goal is to improve inference (which I believe it is), we can actually group documents based on their number of mentions to reduce the amount of padding that is required.

Related to #90

@mickvanhulst mickvanhulst added the enhancement New feature or request label Jun 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant