Refactoring: Remove all references to prodtype (code/tests/docs) #11505

evgenyz · 2024-01-30T12:09:11Z

Description:

Pretty much what it says on the tin. After merging #11378 we don't have any use for prodtype as it always equals all.

This PR removes (and occasionally renames) build system code, tools and documentation related to prodtype.

Rationale:

We don't need the code that has no effect on the build process.

Review Hints:

There are notes in the PR I left to discuss or explain the changes.
Despite the fact that platforms (OVAL and remediation metadata) are somewhat related to prodtype, their improvement is considered out of scope for this PR.
Automatus tests are failing because they can't satisfy scenario requirements for packages. Which should not be a surprise as an attempt to test RHEL7 content in Fedora is pretty much doomed to fail.
Pay attention to 3-rd commit: a623596. It could start doing things we don't actually want.

Closes: #11382.

openshift-ci · 2024-01-30T12:09:15Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

github-actions · 2024-01-30T12:10:27Z

Start a new ephemeral environment with changes proposed in this pull request:

rhel8 (from CTF) Environment (using Fedora as testing environment)

Fedora Testing Environment

Oracle Linux 8 Environment

github-actions · 2024-02-06T22:51:51Z

This datastream diff is auto generated by the check Compare DS/Generate Diff

Click here to see the full diff

New content has different text for rule 'xccdf_org.ssgproject.content_rule_package_audit-audispd-plugins_installed'.
--- xccdf_org.ssgproject.content_rule_package_audit-audispd-plugins_installed
+++ xccdf_org.ssgproject.content_rule_package_audit-audispd-plugins_installed
@@ -4,6 +4,9 @@
 
 [description]:
 The audit-audispd-plugins package should be installed.
+
+[warning]:
+This package is not available in Red Hat Enterprise Linux 8 [rhel8]. The correct package is called audispd-plugins.
 
 [reference]:
 CCI-001851

New content has different text for rule 'xccdf_org.ssgproject.content_rule_service_ntp_enabled'.
--- xccdf_org.ssgproject.content_rule_service_ntp_enabled
+++ xccdf_org.ssgproject.content_rule_service_ntp_enabled
@@ -5,6 +5,9 @@
 [description]:
 The ntp service can be enabled with the following command:
 $ sudo systemctl enable ntp.service
+
+[warning]:
+The ntp package is not available in Red Hat Enterprise Linux 8. Please consider the chrony package instead together with the respective service_chronyd_enabled rule.
 
 [reference]:
 1

New content has different text for rule 'xccdf_org.ssgproject.content_rule_service_ntpd_enabled'.
--- xccdf_org.ssgproject.content_rule_service_ntpd_enabled
+++ xccdf_org.ssgproject.content_rule_service_ntpd_enabled
@@ -5,6 +5,9 @@
 [description]:
 The ntpd service can be enabled with the following command:
 $ sudo systemctl enable ntpd.service
+
+[warning]:
+The ntp package is not available in Red Hat Enterprise Linux 8. Please consider the chrony package instead together with the respective service_chronyd_enabled rule.
 
 [reference]:
 1

Some rules got it wrong.

Mab879

Overall, looks good.

Just more minor change that I'm not willing to block over.

Mab879 · 2024-02-09T21:28:56Z

tests/ssg_test_suite/common.py

    # Our local copy of env_yaml needs some properties from rule.yml
    # for completeness.
    local_env_yaml = dict()
    local_env_yaml.update(env_yaml)
    local_env_yaml['rule_id'] = rule.id_
    local_env_yaml['rule_title'] = rule.title
-    local_env_yaml['products'] = prodtypes
+    local_env_yaml["products"] = {product}


Suggested change

local_env_yaml["products"] = {product}

local_env_yaml['products'] = {product}

evgenyz · 2024-02-09T20:57:27Z

linux_os/guide/services/ftp/ftp_configure_vsftpd/ftp_configure_firewall/rule.yml

@@ -21,7 +21,7 @@ description: |-
 rationale: |-
    These settings configure the firewall to allow connections to an FTP server.

-    {{% if prodtype != "rhel7" %}}
+    {{% if product != "rhel7" %}}


There never was a Jinja variable named prodtype. It doesn't make sense.

Also, it might be that the author (@cipherboy) actually meant to do the opposite. I'm not sure.

\o hello @evgenyz -- Good question! It has been so long, I do not know. Likely I made a mistake and expected the build system to catch it...

evgenyz · 2024-02-09T20:57:56Z

linux_os/guide/services/ntp/service_ntp_enabled/rule.yml

 warnings:
    - general:
-        {{% if prodtype == "rhel7" %}}
+        {{% if product == "rhel7" %}}


There never was a Jinja variable named prodtype. It doesn't make sense.

evgenyz · 2024-02-09T20:58:11Z

linux_os/guide/services/ntp/service_ntpd_enabled/rule.yml

@@ -50,7 +50,7 @@ template:

 platform: package[ntp]

-{{% if prodtype in ["rhel8", "rhel9", "sle15"] %}}
+{{% if product in ["rhel8", "rhel9", "sle15"] %}}


There never was a Jinja variable named prodtype. It doesn't make sense.

evgenyz · 2024-02-09T20:58:19Z

linux_os/guide/system/auditing/package_audit-audispd-plugins_installed/rule.yml

@@ -34,9 +34,9 @@ template:
        pkgname@ubuntu1804: audispd-plugins
        pkgname@ubuntu2004: audispd-plugins

-{{% if prodtype in ["rhel7", "rhel8", "rhel9"] %}}
+{{% if product in ["rhel7", "rhel8", "rhel9"] %}}


There never was a Jinja variable named prodtype. It doesn't make sense.

evgenyz · 2024-02-09T20:59:55Z

shared/macros/10-oval.jinja

@@ -7,7 +7,7 @@ Generates the :code:`<affected>` tag for OVAL check using correct product platfo
 #}}
 {{%- macro oval_affected(products) %}}
 <affected family="unix">
-{{{ prodtype_to_platform(products)|indent(2) }}}
+{{{ product_to_platform(products)|indent(2) }}}


This was renamed to better reflect the intention. The product we use in our code is usually the product slug.

evgenyz · 2024-02-09T21:00:37Z

ssg/fixes.py



-def parse_platform(fix_contents):
+def find_platform_line(fix_contents):


Better reflects what it actually does.

evgenyz · 2024-02-09T21:02:28Z

ssg/utils.py

+
+    return set(map(lambda x: x.strip(), platform.split(',')))
+
+


It has been transferred from the rule_yaml module and renamed to reflect the scope of this new-old function, that is now used only to parse platforms.

evgenyz · 2024-02-09T22:19:59Z

Overall, looks good.

Just more minor change that I'm not willing to block over.

Could you please check the 3-rd commit and maybe provide some insight about the change: is it going to do a good thing or a bad thing?

Mab879 · 2024-02-09T22:40:22Z

Overall, looks good.
Just more minor change that I'm not willing to block over.

Could you please check the 3-rd commit and maybe provide some insight about the change: is it going to do a good thing or a bad thing?

They don't look actively harmful.

jan-cerny · 2024-02-12T08:25:29Z

docs/manual/developer/03_creating_content.md

-If a rule needs to be chosen only in some of products despite its `prodtype` we
-can use Jinja macros inside the controls file to choose products.


Here I would remove only "despite its prodtype".You can still use Jinja macros inside controls file to choose products in which the rule should appear. For example, you can add this to a control file:

{{% if product == "rhel9" %}} - partition_for_tmp {{% endif %}}

jan-cerny · 2024-02-12T08:27:13Z

docs/manual/developer/06_contributing_with_content.md

-fields in the content itself (e.g., if `rhel7` is not present in the
-`rhel7.xml` OVAL check platform specifier, it will be included in the
+content will not override the contents of `platform` field in
+the content itself (e.g., if `rhel7` is not present in the `rhel7.xml`


this is a great idea for work in future

Yep. I cleaned up connections between prodtype and platforms, but kept them out of the scope. It is a zoo there.

codeclimate · 2024-02-12T12:39:26Z

Code Climate has analyzed commit 08d3fb7 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 65.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 58.3% (-0.1% change).

View more on Code Climate.

jan-cerny

I have reviewed code. I also run Automatus locally.

openshift-ci bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Jan 30, 2024

evgenyz added Infrastructure Our content build system refactoring Improvement which, once completed, will enable the project to progress faster. labels Jan 30, 2024

evgenyz mentioned this pull request Jan 30, 2024

Replace rule/group prodtype alignment with explicit inclusion in a product profile/benchmark #11382

Closed

2 tasks

evgenyz force-pushed the remove-prodtype-step2 branch 2 times, most recently from 2bcb1bf to 380644e Compare February 6, 2024 22:44

evgenyz force-pushed the remove-prodtype-step2 branch 3 times, most recently from c20976d to 5ea8d4c Compare February 8, 2024 23:50

evgenyz added 3 commits February 9, 2024 00:50

Remove all references to prodtype (Code and Tests)

6d2b34c

Remove all references to prodtype (Docs)

eb3a99e

Replace bogus 'prodtype' with 'product' in Jinja expressions

a623596

Some rules got it wrong.

evgenyz force-pushed the remove-prodtype-step2 branch from 5ea8d4c to a623596 Compare February 8, 2024 23:51

evgenyz marked this pull request as ready for review February 9, 2024 21:04

openshift-ci bot removed the do-not-merge/work-in-progress Used by openshift-ci bot. label Feb 9, 2024

evgenyz requested review from jan-cerny, Mab879, vojtapolasek, mildas and ggbecker February 9, 2024 21:04

Mab879 approved these changes Feb 9, 2024

View reviewed changes

evgenyz commented Feb 9, 2024

View reviewed changes

Mab879 added this to the 0.1.73 milestone Feb 9, 2024

jan-cerny reviewed Feb 12, 2024

View reviewed changes

Minor fixes after removal of prodtype

08d3fb7

evgenyz force-pushed the remove-prodtype-step2 branch from 1dc8a6c to 08d3fb7 Compare February 12, 2024 12:16

jan-cerny approved these changes Feb 13, 2024

View reviewed changes

jan-cerny merged commit bee8cee into ComplianceAsCode:master Feb 13, 2024
38 of 43 checks passed

jan-cerny self-assigned this Feb 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactoring: Remove all references to prodtype (code/tests/docs) #11505

Refactoring: Remove all references to prodtype (code/tests/docs) #11505

evgenyz commented Jan 30, 2024 •

edited

Loading

openshift-ci bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Feb 6, 2024 •

edited

Loading

Mab879 left a comment

Mab879 Feb 9, 2024

evgenyz Feb 12, 2024

evgenyz Feb 9, 2024

cipherboy Feb 9, 2024

evgenyz Feb 9, 2024

evgenyz Feb 9, 2024

evgenyz Feb 9, 2024

evgenyz Feb 9, 2024

evgenyz Feb 9, 2024

evgenyz Feb 9, 2024 •

edited

Loading

evgenyz commented Feb 9, 2024

Mab879 commented Feb 9, 2024

jan-cerny Feb 12, 2024

evgenyz Feb 12, 2024

jan-cerny Feb 12, 2024

evgenyz Feb 12, 2024

codeclimate bot commented Feb 12, 2024

jan-cerny left a comment

	local_env_yaml["products"] = {product}
	local_env_yaml['products'] = {product}



		def parse_platform(fix_contents):
		def find_platform_line(fix_contents):

		If a rule needs to be chosen only in some of products despite its `prodtype` we
		can use Jinja macros inside the controls file to choose products.

Refactoring: Remove all references to prodtype (code/tests/docs) #11505

Refactoring: Remove all references to prodtype (code/tests/docs) #11505

Conversation

evgenyz commented Jan 30, 2024 • edited Loading

Description:

Rationale:

Review Hints:

openshift-ci bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024 • edited Loading

github-actions bot commented Feb 6, 2024 • edited Loading

Mab879 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

evgenyz Feb 9, 2024 • edited Loading

Choose a reason for hiding this comment

evgenyz commented Feb 9, 2024

Mab879 commented Feb 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codeclimate bot commented Feb 12, 2024

jan-cerny left a comment

Choose a reason for hiding this comment

evgenyz commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Feb 6, 2024 •

edited

Loading

evgenyz Feb 9, 2024 •

edited

Loading