Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add version range to mariner provider #585

Merged
merged 7 commits into from
May 28, 2024
Merged

feat: add version range to mariner provider #585

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
390b1a7
feat: add version range to mariner provider
willmurphyscode May 23, 2024
919e946
feat: account for version ranges in mariner parser
willmurphyscode May 24, 2024
4cf49c4
test: update remaining snapshots
willmurphyscode May 24, 2024
4b91ad0
ensure name exists before emitting fixed-in
willmurphyscode May 24, 2024
77c4098
generate version constraints correctly in vunnel
willmurphyscode May 24, 2024
72b96a9
pin xsdata at 22.12 and re-generate models
willmurphyscode May 28, 2024
2515b78
Merge branch 'main' of github.com:anchore/vunnel into feat-mariner-li…
willmurphyscode May 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/dependabot.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,7 @@ updates:
directory: "/"
schedule:
interval: daily
ignore:
# used to generate Mariner models.
# pin to keep from introducing needless drift in the models.
- dependency-name: "xsdata"
38 changes: 18 additions & 20 deletions poetry.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ orjson = "^3.8.6"
SQLAlchemy = ">= 1.4.46, < 2.0" # note: 1.4.x currently required for enterprise
mergedeep = "^1.3.4"
importlib-metadata = "^7.0.1"
xsdata = {extras = ["cli", "lxml", "soap"], version = ">=22.12,<25.0"}
xsdata = {extras = ["cli", "lxml", "soap"], version = "=22.12"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dev note: we might have to unpin this eventually to generate the models at a later date, but this pin keeps the diff less noisy until we want to tackle this issue

pytest-snapshot = "^0.9.0"
mashumaro = "^3.10"
iso8601 = "^2.1.0"
Expand Down
186 changes: 186 additions & 0 deletions schema/vulnerability/os/schema-1.0.1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "os-vulnerability",
"description": "represents vulnerability records for common linux distributions",
"properties": {
"Vulnerability": {
"type": "object",
"properties": {
"CVSS": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"base_metrics": {
"type": "object",
"properties": {
"base_score": {
"type": "number"
},
"base_severity": {
"type": "string"
},
"exploitability_score": {
"type": "number"
},
"impact_score": {
"type": "number"
}
},
"required": [
"base_score",
"base_severity",
"exploitability_score",
"impact_score"
]
},
"status": {
"type": "string"
},
"vector_string": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"base_metrics",
"status",
"vector_string",
"version"
]
}
]
},
"Description": {
"type": "string"
},
"FixedIn": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"Name": {
"type": "string"
},
"NamespaceName": {
"type": "string"
},
"VendorAdvisory": {
"type": "object",
"properties": {
"AdvisorySummary": {
"type": "array",
"items": {}
},
"NoAdvisory": {
"type": "boolean"
}
},
"required": [
"NoAdvisory"
]
},
"Version": {
"type": "string"
},
"VersionFormat": {
"type": "string"
},
"VulnerableRange": {
"type": ["string", "null"]
},
"Module": {
"type": ["string", "null"]
}
},
"required": [
"Name",
"NamespaceName",
"Version",
"VersionFormat"
]
}
]
},
"Link": {
"type": "string"
},
"Metadata": {
"type": "object",
"properties": {
"Issued": {
"type": "string"
},
"RefId": {
"type": "string"
},
"CVE": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"Name": {
"type": "string"
},
"Link": {
"type": "string"
}
},
"required": [
"Name"
]
}
]
},
"NVD": {
"type": "object",
"properties": {
"CVSSv2": {
"type": "object",
"properties": {
"Score": {
"type": "number"
},
"Vectors": {
"type": "string"
}
},
"required": [
"Score"
]
}
}
}
}
},
"Name": {
"type": "string"
},
"NamespaceName": {
"type": "string"
},
"Severity": {
"type": "string"
}
},
"required": [
"Description",
"FixedIn",
"Link",
"Metadata",
"Name",
"NamespaceName",
"Severity"
]
}
},
"required": [
"Vulnerability"
]
}
4 changes: 2 additions & 2 deletions src/vunnel/providers/mariner/model/generated.py
View file
Open in desktop
wagoodman marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -272,8 +272,8 @@ class Meta:
"type": "Attribute",
}
)
criterion: Optional[Criterion] = field(
default=None,
criterion: List[Criterion] = field(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the important change to the generated model. It might be worth backing out all. the required: True, since it's noisy and might lead to a failure to parse the whole OVAL file if there's one malformed record. (The intended behavior of Vunnel is to skip malformed records but carry on and still generate some data.)

My understanding is that right now there will always be exactly 2 items on the list, at least right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Vunnel parser is written to ignore the failure to parse individual definitions anyway: https://github.com/anchore/vunnel/blob/main/src/vunnel/providers/mariner/parser.py#L45

Based on some experimentation, the new required: True annotations are showing up because of bumping the xsdata package to version 23.5.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on offline discussion, we will pin xsdata package back to the version that generated these models to avoid introducing unnecessary drift in the models. (Putting required: True everywhere has no benefit on a field of type Optional, since given a populated model we have to check for None, and failing validation for this reason would be unfortunate, since the code defends against these Nones anyway.)

default_factory=list,
metadata={
"type": "Element",
}
Expand Down
Loading
Loading