⚠️ OSV scanner integration

[another-rex]

What kind of change does this PR introduce?

Feature

• PR title follows the guidelines defined in our pull request documentation

What is the current behavior?

Currently the vulnerability check only checks if the HEAD commit hash has any vulnerability specified in OSV.dev. Most of the time this will return 10/10.

What is the new behavior (if this is a feature change)?**

This integrates the OSV-Scanner library, which will scan for manifest and SBOM files, retrieve the dependencies, and match those dependencies with OSV.dev database as well. (The commit hash check is still being done, though through the osvscanner library rather than directly setting up a client to use the OSV API).

• Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

Fixes #2162

Special notes for your reviewer

Currently the e2e tests fail because the Scorecard score has changed since osvscanner found 3 vulnerabilities. I can change the expected results to match, but this might cause more issues in the future as new vulnerabilities are discovered, causing tests to break even though there are no code changes. Not sure what the best way to resolve this.

Does this PR introduce a user-facing change?

Yes, TODO.

TODO!

[laurentsimon]

Thanks for this PR. I left one major comment about keeping clients.Vulnerabilities field.

[azeemshaikh38]

a challenge to adding the unit tests for this is /that it will be picked up as a vulnerability, even though it's part of the test data in the repository. Any ideas for a quick fix? Eventually we want to have project level ignore files for osv-scanner.

unit tests should be hermetic and should not be making any network calls. You can use the mocks generated for the interfaces to write any unit tests and it shouldn't affect any vulnerability finding.

[another-rex]

unit tests should be hermetic and should not be making any network calls. You can use the mocks generated for the interfaces to write any unit tests and it shouldn't affect any vulnerability finding.

That makes sense, what about e2e tests? It looks like a some repositories are set up in ossf-tests, can I set up a project with some lockfiles that contain known vulnerable dependencies?

[azeemshaikh38]

unit tests should be hermetic and should not be making any network calls. You can use the mocks generated for the interfaces to write any unit tests and it shouldn't affect any vulnerability finding.

That makes sense, what about e2e tests? It looks like a some repositories are set up in ossf-tests, can I set up a project with some lockfiles that contain known vulnerable dependencies?

Yes that's possible. Curious, does OSV-Scanner look at direct dependencies only or transitive dependencies too?

[laurentsimon]

unit tests should be hermetic and should not be making any network calls. You can use the mocks generated for the interfaces to write any unit tests and it shouldn't affect any vulnerability finding.

That makes sense, what about e2e tests? It looks like a some repositories are set up in ossf-tests, can I set up a project with some lockfiles that contain known vulnerable dependencies?

Yes that's possible. Curious, does OSV-Scanner look at direct dependencies only or transitive dependencies too?

https://github.com/ossf-tests/scorecard-check-osv-e2e

Send a PR and cc me on it - I don't receive notifications automatically for these repos

[codecov]

Codecov Report

Merging #2509 (cd4440e) into main (7206a2b) will decrease coverage by 0.73%.
The diff coverage is 34.61%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2509      +/-   ##
==========================================
- Coverage   40.69%   39.95%   -0.74%     
==========================================
  Files         115      122       +7     
  Lines        9609     9815     +206     
==========================================
+ Hits         3910     3922      +12     
- Misses       5419     5612     +193     
- Partials      280      281       +1     

[inferno-chromium]

@another-rex - Please try to finish this soon, so a scorecard release can be cut by end of week or coming Monday.

[another-rex]

unit tests should be hermetic and should not be making any network calls. You can use the mocks generated for the interfaces to write any unit tests and it shouldn't affect any vulnerability finding.

That makes sense, what about e2e tests? It looks like a some repositories are set up in ossf-tests, can I set up a project with some lockfiles that contain known vulnerable dependencies?

Yes that's possible. Curious, does OSV-Scanner look at direct dependencies only or transitive dependencies too?

OSV-Scanner also looks at transitive dependencies, since it scans the lockfile, which generally lists the transitive dependencies.

[naveensrinivasan]

@another-rex - Please try to finish this soon, so a scorecard release can be cut by end of week or coming Monday.

Is there a specific reason we want to cut a release?

[inferno-chromium]

@another-rex - Please try to finish this soon, so a scorecard release can be cut by end of week or coming Monday.

Is there a specific reason we want to cut a release?

I mean this is a very strong feature where scorecard was not looking at transitive deps before and can now can look at vulns transitively. By release, i just meant a minor release (not major announcement or anything for scorecard). OSV-scanner will go out next week, so it will be nice for people to try in scorecard too, not a big deal if you need more review time.

[naveensrinivasan]

@another-rex - Please try to finish this soon, so a scorecard release can be cut by end of week or coming Monday.

Is there a specific reason we want to cut a release?

I mean this is a very strong feature where scorecard was not looking at transitive deps before and can now can look at vulns transitively. By release, i just meant a minor release (not major announcement or anything for scorecard). OSV-scanner will go out next week, so it will be nice for people to try in scorecard too, not a big deal if you need more review time.

Yes, that makes sense! 👍

Thanks for the update. We need to get better at releases. To be honest, we need to improve our release process by calling out new features and breaking changes in the release notes.

[github-actions]

Integration tests success for
[f405955]
(https://github.com/ossf/scorecard/actions/runs/3654448698)

[github-actions]

Integration tests success for
[578624d]
(https://github.com/ossf/scorecard/actions/runs/3670958619)

[naveensrinivasan]

Integration tests success for
[f405955]
(https://github.com/ossf/scorecard/actions/runs/3654448698)

Can you please rebase with the main? Also, DCO is missing. Thanks

[github-actions]

Integration tests success for
[61e4be7]
(https://github.com/ossf/scorecard/actions/runs/3672092646)

[github-actions]

Integration tests success for
[981b438]
(https://github.com/ossf/scorecard/actions/runs/3680087135)

[github-actions]

Integration tests success for
[3e71603]
(https://github.com/ossf/scorecard/actions/runs/3680095638)

[github-actions]

Integration tests success for
[cd4440e]
(https://github.com/ossf/scorecard/actions/runs/3681023522)