Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds grype json output to the created github issue #1131

Merged

Conversation

joe-kimmel-vmw
Copy link
Contributor

see #1021

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
@joe-kimmel-vmw joe-kimmel-vmw force-pushed the grype-but-with-output branch 3 times, most recently from 41bfc57 to 64acefc Compare June 21, 2023 19:05
@joe-kimmel-vmw joe-kimmel-vmw marked this pull request as ready for review June 21, 2023 19:06
@joe-kimmel-vmw joe-kimmel-vmw requested a review from a team as a code owner June 21, 2023 19:06
@joe-kimmel-vmw
Copy link
Contributor Author

Example of the new issue output can be seen in my fork.

Note for exploratory purposes I had set the "failing" severity level to low so real output wouldn't include medium or low sev issues.

Note also: I'm not sure why some of the issues are duplicated but that seems to be in the scan output?

joe-kimmel-vmw#7

joe-kimmel-vmw#6

@joe-kimmel-vmw joe-kimmel-vmw changed the title probably adds grype json output to the created github issue adds grype json output to the created github issue Jun 22, 2023
@@ -92,7 +95,7 @@ jobs:
search_output=$(gh issue list --search "$title" --label "$label")

GITHUB_WORKFLOW_URL=https://github.com/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID
body="Latest lifecycle release v${{ steps.read-go.outputs.latest-release-version }} triggered CVE(s) from Grype. For further details, see: $GITHUB_WORKFLOW_URL"
body="Latest lifecycle release v${{ steps.read-versions.outputs.latest-release-version }} triggered CVE(s) from Grype. For further details, see: $GITHUB_WORKFLOW_URL json: $(cat ${{ steps.scan-image.outputs.json }} | jq '.matches[] | .vulnerability | {id, severity, description}' )"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this solution but I was thinking about a way to improve the visualisation of the issues' content and I thought that we might use a markdown table rather than a json output.

We can do something like

body: |
  Latest lifecycle release v${{ steps.read-versions.outputs.latest-release-version }} triggered CVE(s) from Grype. For further details, see: ${{ github.event.workflow_run.url }}

  | ID | Severity | Description |
  |----|----------|-------------|
  ${{ join('\n', steps.scan-image.outputs.json | fromjson | .matches[].vulnerability | "- | \(.id) | \(.severity) | \(.description) |") }}

We can stick with jq if it is better for you using jq -r '.matches[] | .vulnerability | [.id, .severity, .description] | @tsv'
and as output something like

Latest lifecycle release v1.0.0 triggered CVE(s) from Grype. For further details, see: https://github.com/owner/repo/actions/runs/123456789

| ID    | Severity | Description          |
|-------|----------|----------------------|
| CVE-1 | High     | Vulnerability 1      |
| CVE-2 | Medium   | Vulnerability 2      |
| CVE-3 | Low      | Vulnerability 3      |

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dlion I finally tried these suggestions -

the jq formatting didn't seem to work:
joe-kimmel-vmw#8

and i guess i don't understand the other suggestion with join, .. i just got

The workflow is not valid. .github/workflows/check-latest-release.yml (Line: 84, Col: 14): Unexpected symbol: '|'. Located at position 42 within expression: join('\n', steps.scan-image.outputs.json | fromjson | .matches[].vulnerability | "- | \(.id) | \(.severity) | \(.description) |")

IMO most of the value here is provided by having anything at all, whereas today we don't actually get any useful information from the scan. I appreciate that nice formatting is nice, but it's a cumbersome debug cycle because I have to hardcode part of the action and then push and run remotely for each iteration, so i'm not inclined to troubleshooting very much.. I'll leave this open / up overnight in case you want to tell me what i'm doing wrong in the most recent two commits..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I tried it with a wrong json output. 😅
I found online https://github.com/anchore/grype/blob/main/grype/presenter/json/test-fixtures/snapshot/TestJsonImgsPresenter.golden

So trying with this one:

{
 "matches": [
  {
   "vulnerability": {
    "id": "CVE-1999-0001",
    "dataSource": "",
    "severity": "Low",
    "urls": [],
    "description": "1999-01 description",
    "cvss": [
     {
      "version": "3.0",
      "vector": "another vector",
      "metrics": {
       "baseScore": 4
      },
      "vendorMetadata": {}
     }
    ],
    "fix": {
     "versions": [
      "the-next-version"
     ],
     "state": "fixed"
    },
    "advisories": []
   },
   "relatedVulnerabilities": [],
   "matchDetails": [
    {
     "type": "exact-direct-match",
     "matcher": "dpkg-matcher",
     "searchedBy": {
      "distro": {
       "type": "ubuntu",
       "version": "20.04"
      }
     },
     "found": {
      "constraint": ">= 20"
     }
    }
   ],
   "artifact": {
    "id": "96699b00fe3004b4",
    "name": "package-1",
    "version": "1.1.1",
    "type": "rpm",
    "locations": [
     {
      "path": "/foo/bar/somefile-1.txt"
     }
    ],
    "language": "",
    "licenses": [],
    "cpes": [
     "cpe:2.3:a:anchore:engine:0.9.2:*:*:python:*:*:*:*"
    ],
    "purl": "",
    "upstreams": [
     {
      "name": "nothing",
      "version": "3.2"
     }
    ],
    "metadataType": "RpmMetadata",
    "metadata": {
     "epoch": 2,
     "modularityLabel": ""
    }
   }
  },
  {
   "vulnerability": {
    "id": "CVE-1999-0002",
    "dataSource": "",
    "severity": "Critical",
    "urls": [],
    "description": "1999-02 description",
    "cvss": [
     {
      "version": "2.0",
      "vector": "vector",
      "metrics": {
       "baseScore": 1,
       "exploitabilityScore": 2,
       "impactScore": 3
      },
      "vendorMetadata": {
       "BaseSeverity": "Low",
       "Status": "verified"
      }
     }
    ],
    "fix": {
     "versions": [],
     "state": ""
    },
    "advisories": []
   },
   "relatedVulnerabilities": [],
   "matchDetails": [
    {
     "type": "exact-indirect-match",
     "matcher": "dpkg-matcher",
     "searchedBy": {
      "cpe": "somecpe"
     },
     "found": {
      "constraint": "somecpe"
     }
    }
   ],
   "artifact": {
    "id": "b4013a965511376c",
    "name": "package-2",
    "version": "2.2.2",
    "type": "deb",
    "locations": [
     {
      "path": "/foo/bar/somefile-2.txt"
     }
    ],
    "language": "",
    "licenses": [
     "MIT",
     "Apache-2.0"
    ],
    "cpes": [
     "cpe:2.3:a:anchore:engine:2.2.2:*:*:python:*:*:*:*"
    ],
    "purl": "",
    "upstreams": []
   }
  }
 ],
 "source": {
  "type": "image",
  "target": {
   "userInput": "user-input",
   "imageID": "sha256:ab5608d634db2716a297adbfa6a5dd5d8f8f5a7d0cab73649ea7fbb8c8da544f",
   "manifestDigest": "sha256:ca738abb87a8d58f112d3400ebb079b61ceae7dc290beb34bda735be4b1941d5",
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "tags": [],
   "imageSize": 65,
   "layers": [
    {
     "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
     "digest": "sha256:ca738abb87a8d58f112d3400ebb079b61ceae7dc290beb34bda735be4b1941d5",
     "size": 22
    },
    {
     "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
     "digest": "sha256:a05cd9ebf88af96450f1e25367281ab232ac0645f314124fe01af759b93f3006",
     "size": 16
    },
    {
     "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
     "digest": "sha256:ab5608d634db2716a297adbfa6a5dd5d8f8f5a7d0cab73649ea7fbb8c8da544f",
     "size": 27
    }
   ],
   "manifest": null,
   "config": null,
   "repoDigests": [],
   "architecture": "",
   "os": ""
  }
 },
 "distro": {
  "name": "centos",
  "version": "8.0",
  "idLike": [
   "centos"
  ]
 },
 "descriptor": {
  "name": "grype",
  "version": "[not provided]",
  "timestamp":""
 }
}

Which should be close to what we have

And the command is:

cat example.json | jq -r '.matches[].vulnerability | ["| \(.id) | \(.severity) | \(.description) |"][]' | awk 'BEGIN{print "| ID | Severity | Description |\n| --- | --- | --- |"} {print $0}'

It generates:

❯ cat example.json | jq -r '.matches[].vulnerability | ["| \(.id) | \(.severity) | \(.description) |"][]' | awk 'BEGIN{print "| ID | Severity | Description |\n| --- | --- | --- |"} {print $0}'

| ID | Severity | Description |
| --- | --- | --- |
| CVE-1999-0001 | Low | 1999-01 description |
| CVE-1999-0002 | Critical | 1999-02 description |

I removed any jq extension using just jq basic functionalities and awk.


If it doesn't work either, I agree with you that it doesn't worth more than a bunch of hours and that we can just print the json into the issue but let's try it first 😁

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's closer to being a table but it's not getting escaped for markdown joe-kimmel-vmw#9

i'll try one more time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added in triple-ticks so that the table would be interpreted as markdown in the issue but it didn't work - the issue failed to create because i guess somehow then the pipe didn't get interpreted correctly?

https://github.com/joe-kimmel-vmw/lifecycle/actions/runs/5404335134/jobs/9818362129

Run #!/usr/bin/env bash
label with name "cve" already exists; use `--force` to update its color and description
/home/runner/work/_temp/3e9ac2b3-28c9-49ba-8d98-e49302e3e0c6.sh: line 14: |: command not found
Error: Process completed with exit code 127.

i think at this point honestly printing it as valid json is pretty great anyhow in case somebody wants to paste that output into a script (or an editor/interface that makes it easy to collapse/filter), and much simpler. so i'm going to revert to what was previously working.

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

small refactor of gh action shell script and better issue name

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

sev threshold explicitly set to high

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com>
Copy link
Member

@dlion dlion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely there is room for improvement but since it doesn't have that much priority LGTM 👍🏼

@joe-kimmel-vmw
Copy link
Contributor Author

@natalieparellano wdyt?

Copy link
Member

@natalieparellano natalieparellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great improvement! Thanks for the PR @joe-kimmel-vmw!

@natalieparellano natalieparellano merged commit 1744539 into buildpacks:main Jul 6, 2023
7 checks passed
dlion pushed a commit to dlion/lifecycle that referenced this pull request Jul 7, 2023
* probably adds grype json output to the created github issue

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

* specify json output and use jq query to select summary

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

small refactor of gh action shell script and better issue name

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

sev threshold explicitly set to high

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

* grype release sev cutoff back to medium

Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com>

---------

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com>
Co-authored-by: Natalie Arellano <narellano@vmware.com>
dlion pushed a commit to dlion/lifecycle that referenced this pull request Jul 20, 2023
* probably adds grype json output to the created github issue

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

* specify json output and use jq query to select summary

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

small refactor of gh action shell script and better issue name

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

sev threshold explicitly set to high

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>

* grype release sev cutoff back to medium

Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com>

---------

Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com>
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants