-
Notifications
You must be signed in to change notification settings - Fork 685
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce a JSON build manifest #10761
Conversation
Skipping CI for Draft Pull Request. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My main concern is that the process of generating a data stream is complex and therefore there can be some difference between the manifest generated this way and the actual contents of data stream.
build-scripts/generate_manifest.py
Outdated
expected_filename = os.path.join(main_dir, expected_filename_template.format(id=rid)) | ||
if os.path.exists(expected_filename): | ||
contents.add(content_id) | ||
output_dict[rid] = list(contents) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For some reason, the list under the rule id is often empty, even if the remediation files exist.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't reproduce this, could you provide an example?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can reproduce it using ./build_product rhel9
. There is a large amount of examples basically you can pick any rule that has this list empty in the manifest, eg. package_sudo_installed, aide_periodic_cron_checking, partition_for_tmp, and you will find that in fact there exists a bash or ansible.
cmake/SSGCommon.cmake
Outdated
add_custom_command( | ||
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/manifest.json" | ||
COMMAND env "PYTHONPATH=$ENV{PYTHONPATH}" "${PYTHON_EXECUTABLE}" "${SSG_BUILD_SCRIPTS}/generate_manifest.py" --output "${CMAKE_CURRENT_BINARY_DIR}/manifest.json" --build-root "${CMAKE_CURRENT_BINARY_DIR}" | ||
DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/cpe-oval-unlinked.xml" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is the dependency cpe-oval-unlinked?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean: shouldn't this cmake command depend on a different target?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because the generator uses compiled artifacts. While most of artifacts are compiled at the beginning of the build, the per-rule OVALs are made available only after that step that is marked as a dependency.
I will add a comment to clarify that, it is indeed confusing.
Moreover, it doesn't appear to be working.
Yes, this manifest definitely reflects capabilities of the project. Although we are pretty sure that whatever is compiled makes its way to the datastream, it is not guaranteed. |
Collect built artifacts, and save the collection to a json file that represents the structure of the content available for the built product in a form that is more easy to process than the scanner-facing artifacts s.a. datastreams.
... to the manifest
This commit introduces a new script compare_versions.py which can show differences between two ComplianceAsCode versions. Lists added or removed rules, profiles, changes in profile composition and changes in remediations and platforms. For comparison, you can use git tags or ComplianceAsCode JSON manifest files directly. This new feature leverages the ComplianceAsCode JSON manifests, which have been introduced by ComplianceAsCode#10761
cmake/SSGCommon.cmake
Outdated
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/manifest-${PRODUCT}.json" | ||
COMMAND env "PYTHONPATH=$ENV{PYTHONPATH}" "${PYTHON_EXECUTABLE}" "${SSG_BUILD_SCRIPTS}/generate_manifest.py" --output "${CMAKE_CURRENT_BINARY_DIR}/manifest-${PRODUCT}.json" --build-root "${CMAKE_CURRENT_BINARY_DIR}" | ||
# The manifest requires compiled artifacts on right places and also per-rule OVAL. | ||
# It is not clear when those things assume their places, so manifest is compiled late |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indentation problem. It causes a CI fail in the cmakelint test.
It may take a lot of steps to get things in place
Code Climate has analyzed commit f215139 and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (50% is the threshold). This pull request will bring the total coverage in the repository to 53.5% (0.0% change). View more on Code Climate. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have generated and reviewed the manifest for RHEL 9 and the data in there seem to be correct.
This commit introduces a new script compare_versions.py which can show differences between two ComplianceAsCode versions. Lists added or removed rules, profiles, changes in profile composition and changes in remediations and platforms. For comparison, you can use git tags or ComplianceAsCode JSON manifest files directly. This new feature leverages the ComplianceAsCode JSON manifests, which have been introduced by ComplianceAsCode#10761
This commit introduces a new script compare_versions.py which can show differences between two ComplianceAsCode versions. Lists added or removed rules, profiles, changes in profile composition and changes in remediations and platforms. For comparison, you can use git tags or ComplianceAsCode JSON manifest files directly. This new feature leverages the ComplianceAsCode JSON manifests, which have been introduced by ComplianceAsCode#10761
This commit introduces a new script compare_versions.py which can show differences between two ComplianceAsCode versions. Lists added or removed rules, profiles, changes in profile composition and changes in remediations and platforms. For comparison, you can use git tags or ComplianceAsCode JSON manifest files directly. This new feature leverages the ComplianceAsCode JSON manifests, which have been introduced by ComplianceAsCode#10761
This commit introduces a new script compare_versions.py which can show differences between two ComplianceAsCode versions. Lists added or removed rules, profiles, changes in profile composition and changes in remediations and platforms. For comparison, you can use git tags or ComplianceAsCode JSON manifest files directly. This new feature leverages the ComplianceAsCode JSON manifests, which have been introduced by ComplianceAsCode/content#10761
Description:
Summarize product's built content in a JSON file.
The format isn't definitive, but the PR illustrates that it is easy to provide certain key information about content from compiled artifacts.
Rationale:
Although right now, a datastream can be used as a single source of truth, it is difficult to parse it. The JSON format is much better in this regard, and it allows e.g. machine-readable high-level comparison of two different content releases.
An accompanying tool able to compare contents of two versions of ComplianceAsCode will be added to this PR later.
Review Hints:
Build a product, and examine the
manifest.json
in the build directory.