Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Up paperspace MAX_POLLS_FOR_UP_OR_STOP #3952

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

BitPhinix
Copy link

@BitPhinix BitPhinix commented Sep 17, 2024

Increases the paperspace MAX_POLLS_FOR_UP_OR_STOP from 8 to 16 minutes.

Spawning a larger number of CPU instances on paperspace can be quite slow. I've had to manually monkeypatch this limit several times to spawn a C8 clusters on paperspace. This is definitely set too low.

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

@concretevitamin
Copy link
Member

cc @asaiacai

@asaiacai
Copy link
Contributor

hi @BitPhinix , I added the paperspace integration. The PR lgtm at a glance, but do you have a minimal task.yaml that I can use to AB test the PR on my end. I am surprised since the CPU instances usually boot up pretty fast but it might be a question of scale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants