Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use small videos in checkpointing tests #5305

Merged
merged 4 commits into from
Feb 12, 2024

Conversation

szkarpinski
Copy link
Collaborator

@szkarpinski szkarpinski commented Feb 2, 2024

Category:

Other -- test optimization

Description:

Checkpointing tests are taking some time to complete (especially with sanitizer). Video reader tests take 60% of the time spent in test_dali_checkpointing.

When testing checkpointing, we just want to see if the correct frames are produced after restoring. To do that, we don't need full resolution videos and small videos added in NVIDIA/DALI_extra#127 are fine.

In this PR I make checkpointing tests of video_reader and video_resize_reader use small videos. This results in ~15x speedup in those tests (which should result in ~2x total speedup for test_dali_checkpointing)

Additional information:

Affected modules and functionalities:

Checkpointing tests

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: N/A

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [12541620]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [12541620]: BUILD PASSED

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@szkarpinski szkarpinski changed the title [WiP] Experiments: make checkpointing tests faster Use small videos in checkpointing tests Feb 5, 2024
@szkarpinski szkarpinski marked this pull request as ready for review February 5, 2024 13:32
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [12757511]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [12757511]: BUILD PASSED

@szkarpinski szkarpinski merged commit 9fbbe5e into NVIDIA:main Feb 12, 2024
7 checks passed
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants