Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable checkpointing in TensorFlow plugin (CPU only) #5334

Merged
merged 7 commits into from
Feb 27, 2024

Conversation

szkarpinski
Copy link
Collaborator

@szkarpinski szkarpinski commented Feb 23, 2024

Category:

New feature (non-breaking change which adds functionality)

Description:

This PR adds checkpointing support for TensorFlow plugin (only for CPU datasets). DALIDataset is now compatible with TF checkpointing . SaveInternal and RestoreInternal methods were implemented.

DALIDatasetWithInputs is not supported yet.

Additional information:

Affected modules and functionalities:

dali_dataset_op.cc

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: DALI-3750

@szkarpinski szkarpinski marked this pull request as ready for review February 23, 2024 15:12
@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13110161]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13110161]: BUILD FAILED

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13116159]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13116159]: BUILD FAILED

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13119222]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13119222]: BUILD PASSED

@szkarpinski szkarpinski merged commit 1d362af into NVIDIA:main Feb 27, 2024
7 checks passed
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants