Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save task state info in the ctx for checkpoint #245

Merged
merged 4 commits into from
Aug 21, 2024

Conversation

superstar54
Copy link
Member

@superstar54 superstar54 commented Aug 20, 2024

The engine saves the task state in the extras so that it can be accessed from outside the engine. However, this will create a mismatch when the engine process is restored from a checkpoint.

This PR saves the task state information in the context so that it can be restored from a checkpoint. The state info is also stored separately in the extras of the WorkGraph process node for each task so that we can query it. For the process node, we serialize it so that it can also be saved into the extras.

In order to interact with the running WorkGraph Engine, we need to use rpc_send to send a custom message to the engine.

@superstar54 superstar54 linked an issue Aug 20, 2024 that may be closed by this pull request
@superstar54 superstar54 force-pushed the fix/242/restore_normal_function_task branch 2 times, most recently from 31d68ab to c0472bd Compare August 20, 2024 16:21
@superstar54 superstar54 force-pushed the fix/242/restore_normal_function_task branch 2 times, most recently from 70d6c54 to 44a9f05 Compare August 20, 2024 20:25
@codecov-commenter
Copy link

codecov-commenter commented Aug 20, 2024

Codecov Report

Attention: Patch coverage is 61.11111% with 7 lines in your changes missing coverage. Please review.

Project coverage is 79.48%. Comparing base (5937b88) to head (f1e4ff6).
Report is 40 commits behind head on main.

Files Patch % Lines
aiida_workgraph/engine/workgraph.py 61.53% 5 Missing ⚠️
aiida_workgraph/utils/control.py 33.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #245      +/-   ##
==========================================
+ Coverage   75.75%   79.48%   +3.73%     
==========================================
  Files          70       65       -5     
  Lines        4615     4807     +192     
==========================================
+ Hits         3496     3821     +325     
+ Misses       1119      986     -133     
Flag Coverage Δ
python-3.11 79.40% <61.11%> (+3.73%) ⬆️
python-3.9 79.44% <61.11%> (+3.70%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@superstar54 superstar54 force-pushed the fix/242/restore_normal_function_task branch from 44a9f05 to f1e4ff6 Compare August 21, 2024 04:31
@superstar54 superstar54 merged commit 531bcbb into main Aug 21, 2024
5 of 8 checks passed
@superstar54 superstar54 deleted the fix/242/restore_normal_function_task branch August 21, 2024 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Restore non-AiiDA process task from checkpoint
2 participants