Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚀 [Feature] Multi-threaded processing #45

Open
darkdragon-001 opened this issue May 15, 2021 · 1 comment
Open

🚀 [Feature] Multi-threaded processing #45

darkdragon-001 opened this issue May 15, 2021 · 1 comment
Labels

Comments

@darkdragon-001
Copy link
Collaborator

Problem to address
Modern systems provide multiple cores which are unused when the scripts run in a single thread.

Feature description
Process tasks concurrently to speed up processing of many laws significantly.

Implementation details
Use queue module.

It should be used optionally with a command-line flag.

@mk-pmb
Copy link
Contributor

mk-pmb commented Nov 14, 2023

If we can split workload accross different machines (virtual machines with one or very few cores), that would be even better in some scenarios. Maybe we can define some notion of todo range and then have each machine work on its assigned range?
If needed we can run a todo scanner first that then suggests ranges. A 3rd-party script specific to the cloud infrastructure can then use those suggestions to spin up the appropriate VMs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants