Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tool for identifying the most used rules #11439

Merged
merged 10 commits into from
Feb 29, 2024

Conversation

Honny1
Copy link
Collaborator

@Honny1 Honny1 commented Jan 10, 2024

Description:

This PR adds a subcommand profile_tool.py that generates a list of rules with the number of uses in profiles in different formats.

Rationale:

It is known that many rules are common among profiles so we can infer that much less than 1825 rules are in fact used for RHEL, but we are including thousands of rules in the data stream because we don't know exactly what is needed or not.

We have many rules without Ansible remediation, some rules without Bash remediation and some few rules without OVAL check. It is great to close the gaps, but it would be smart to prioritize the most used rules.

It is hard to identify these most used rules and consequently optimize our efforts.

Review Hints:

To generate a list of the most used rules in the rhel9 benchmark you can run this command:

    $ ./build_product rhel9
    $ ./build-scripts/profile_tool.py most-used-rules  build/ssg-rhel9-xccdf.xml

Or you can run this command to get info about the whole project:

    $ ./build-scripts/profile_tool.py most-used-rules

Depends on: #11438

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Jan 10, 2024
Copy link

openshift-ci bot commented Jan 10, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@Honny1 Honny1 added the Infrastructure Our content build system label Jan 10, 2024
Copy link

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

@jan-cerny
Copy link
Collaborator

@Honny1 It would be great to sort the output by the count of the rules.

@Honny1 Honny1 force-pushed the most-used-rules branch 2 times, most recently from e92662f to 343816e Compare January 10, 2024 12:04
@Honny1
Copy link
Collaborator Author

Honny1 commented Jan 10, 2024

@jan-cerny The rules are listed in descending order.

@jan-cerny
Copy link
Collaborator

@Honny1 It doesn't sort for me

@Honny1
Copy link
Collaborator Author

Honny1 commented Jan 10, 2024

@jan-cerny Fixed!

@jan-cerny
Copy link
Collaborator

now it sorts for me, thanks

@marcusburghardt
Copy link
Member

@Honny1 , I saw the changes in #11438 are also incorporated here. Did I miss any change there but not here?

@Honny1
Copy link
Collaborator Author

Honny1 commented Jan 12, 2024

@marcusburghardt Yes, the changes from #11438 should be incorporated into this PR. I will rebase on the master after merging #11438.

@Honny1 Honny1 force-pushed the most-used-rules branch 5 times, most recently from e9e7849 to d915f63 Compare January 17, 2024 15:43
@Honny1 Honny1 marked this pull request as ready for review January 17, 2024 16:10
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Used by openshift-ci bot. label Jan 17, 2024
@Honny1 Honny1 force-pushed the most-used-rules branch 4 times, most recently from 2d19ce6 to cf9e5c4 Compare January 19, 2024 10:09
@Honny1 Honny1 force-pushed the most-used-rules branch 2 times, most recently from 60b5992 to 0761fe4 Compare January 29, 2024 09:45
@marcusburghardt marcusburghardt added this to the 0.1.73 milestone Jan 30, 2024
@jan-cerny
Copy link
Collaborator

/packit build

Copy link
Member

@marcusburghardt marcusburghardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested all the functions and they are working as expected. I have only some minor comments mainly about readability. After that I believe it would be good to be merged.

build-scripts/profile_tool.py Outdated Show resolved Hide resolved
ssg/build_profile.py Outdated Show resolved Hide resolved
utils/profile_tool/most_used_rules.py Outdated Show resolved Hide resolved
utils/profile_tool/profile.py Outdated Show resolved Hide resolved
utils/profile_tool/profile.py Outdated Show resolved Hide resolved
utils/profile_tool/profile.py Outdated Show resolved Hide resolved


class Profile:
def __init__(self, path, title):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered to also count the variables? I think it is easy to extend and this information might be useful as well.

If so, the next function can also be renamed from add_rule to add_rule_or_var, for example. However, it would be probably better to extend this in another PR.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it can easily be expanded if necessary.

utils/profile_tool/most_used_rules.py Outdated Show resolved Hide resolved
utils/profile_tool/most_used_rules.py Outdated Show resolved Hide resolved
Copy link

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:11439

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:11439

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:11439 make deploy-local

Add optional benchmark parameters.
Without parameters, the control files from the project will be used.
@Honny1 Honny1 force-pushed the most-used-rules branch 3 times, most recently from 7b536c5 to 2d96df9 Compare February 28, 2024 14:37
Copy link

codeclimate bot commented Feb 28, 2024

Code Climate has analyzed commit 18bfd53 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 83.3% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.8% (2.0% change).

View more on Code Climate.

@Honny1
Copy link
Collaborator Author

Honny1 commented Feb 29, 2024

/packit retest-failed

Copy link
Member

@marcusburghardt marcusburghardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @Honny1 . This capability will open space for good insights. Thanks

@marcusburghardt marcusburghardt merged commit cb599e6 into ComplianceAsCode:master Feb 29, 2024
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Infrastructure Our content build system
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants