Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring: Remove all references to prodtype (code/tests/docs) #11505

Merged
merged 4 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,6 @@ We use an OpenControl-inspired YAML rule format for input. Write once and
generate security content in XCCDF, Ansible, and others.

```YAML
prodtype: rhel7

title: 'Configure The Number of Allowed Simultaneous Requests'

description: |-
Expand Down
8 changes: 4 additions & 4 deletions controls/pcidss_4.yml
Original file line number Diff line number Diff line change
Expand Up @@ -845,7 +845,7 @@ controls:
title: When using remote-access technologies, technical controls prevent copy and/or
relocation of PAN for all personnel, except for those with documented, explicit
authorization and a legitimate, defined business need.
description: |-
description: |-
PAN cannot be copied or relocated by unauthorized personnel using remote-access
technologies. Storing or relocating PAN onto local hard drives, removable electronic
media, and other storage devices brings these devices into scope for PCI DSS.
Expand Down Expand Up @@ -2053,7 +2053,7 @@ controls:
- accounts_passwords_pam_tally2_unlock_time
- var_accounts_passwords_pam_tally2_unlock_time=1800
- cracklib_accounts_password_pam_retry

- id: 8.3.5
title: If passwords/passphrases are used as authentication factors to meet Requirement
8.3.1, they are set and reset for each user.
Expand Down Expand Up @@ -3004,8 +3004,8 @@ controls:
- base
status: automated
notes: |-
The selected rules might need updates in order to restrict their applicabilities or their
prodtypes to avoid conflicts.
The selected rules might need updates in order to restrict their platform applicability
to avoid conflicts.
rules:
# specific products should update the variable value when defining the profile based on
# this control file.
Expand Down
1 change: 0 additions & 1 deletion docs/manual/developer/01_introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ The issue description must explicitly state the date and time (in UTC) when the
The pull request removing the product should include the removal of
1. The product folder in `products/`
1. any Jinja templates that use the product
1. the product from all `prodtype`
1. any product specific checks or remediations
1. any product specific templates
1. The removal must be documented in the [user guide](../user/30_content_notes.md#deprecated-content).
Expand Down
11 changes: 1 addition & 10 deletions docs/manual/developer/03_creating_content.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,13 +180,7 @@ Rules from multiple locations can be used for a single Benchmark. There
is an optional key `additional_content_directories` for a list of paths
to some arbitrary Groups of Rules to be included in the benchmark. Note
that the additional directories cannot contain a benchmark file
(`benchmark.yml`), otherwise it fails to build the content. Of all the
rules collected only the following would become a part of the benchmark:

- rules that have the `prodtype` specified in correspondence with the
benchmark;

- rules that have no `prodtype` metadata.
(`benchmark.yml`), otherwise it fails to build the content.

<!-- -->

Expand Down Expand Up @@ -742,9 +736,6 @@ between XCCDF rules which directly implement the given controls (represented by
The `rules` and `related_rules` keys consist of a list of rule IDs and variable
selections.

If a rule needs to be chosen only in some of products despite its `prodtype` we
can use Jinja macros inside the controls file to choose products.
Comment on lines -745 to -746
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I would remove only "despite its prodtype".You can still use Jinja macros inside controls file to choose products in which the rule should appear. For example, you can add this to a control file:

{{% if product == "rhel9" %}}
    - partition_for_tmp
{{% endif %}}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.


After we finish our analysis, we will insert our findings to the controls file,
the file will look like this:

Expand Down
5 changes: 0 additions & 5 deletions docs/manual/developer/04_style_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,11 +226,6 @@ This section describes the style guide around the `rule.yml` files.
Rules sections must be in the following order, if they are present.

* `documentation_complete`
* `prodtype`
* Comma separated list
* No spaces between items
* Items must be in alphabetical order
* Required on all new rules
* `title`
* Must be one line
* Must be in [Title case](https://en.wikipedia.org/wiki/Title_case)
Expand Down
79 changes: 4 additions & 75 deletions docs/manual/developer/05_tools_and_utilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,6 @@ These sub-commands are:
- `duplicate_subkeys`: finds (but doesn't fix!) any rules with duplicated
`identifiers` or `references`.
- `sort_subkeys`: sorts all subkeys under `identifiers` and `references`.
- `sort_prodtypes`: sorts the products in prodtype.
- `add-cce`: automatically assign CCE identifiers to rules.

To execute:
Expand Down Expand Up @@ -156,7 +155,6 @@ Then based on the available pool you want to assign the CCEs, you can run someth
```

Note: Multiple rules can have the CCE at the same time by just adding space separated rule IDs.
Note: The rule should have the product assigned to the `prodtype` attribute or the `prodtype` should be empty.

Example for `sle15` product:

Expand All @@ -165,34 +163,6 @@ Example for `sle15` product:
```


### `utils/autoprodtyper.py` &ndash; automatically add product to `prodtype`

When building a profile for a new product version (such as forking
`ubuntu1804` into `ubuntu2004`), it is helpful to be able to build a
profile (adding in all rules that are necessary) and then attempt a
build.

However, usually lots of rules will lack the new product in its `prodtype`
field.

This is where `utils/autoprodtyper.py` comes in: point it at a product and
a profile and it will automatically modify the prodtype, adding this product.

To execute:

```bash
$ ./utils/autoprodtyper.py <product> <profile>
```

For example:

```bash
$ ./utils/autoprodtyper.py ubuntu2004 cis_level1_server
```

Note that it is generally good practice to commit all changes prior to running
one of these commands and then commit the results separately.

### `utils/refchecker.py` &ndash; automatically check `rule.yml` for references

This utility checks all `rule.yml` referenced from a given profile for the
Expand All @@ -218,52 +188,12 @@ product-independent.

Note that this utility does not modify the rule directories at all.

### `utils/mod_prodtype.py` &ndash; programmatically modify prodtype in `rule.yml`

`utils/mod_prodtype.py` is a command-based utility for modifying `rule.yml`
files. It supports the following sub-commands:

- `add`: add the given product(s) to the specified rule's prodtype.
- `list`: list computed and actual products in the specified rule's prodtype.
- `replace`: perform a pattern-match replacement on the specified rule's
prodtype.
- `remove`: remove the given product(s) from the specified rule's prodtype.

To execute:

```bash
$ ./utils/mod_prodtype.py <rule_id> <command> [...other arguments...]
```

For an example of `add`:

```bash
$ ./utils/mod_prodtype.py accounts_passwords_pam_tally2 add ubuntu2004
```

For an example of `list`:

```bash
$ ./utils/mod_prodtype.py accounts_passwords_pam_tally2 list
```

For an example of `replace`:

```bash
$ ./utils/mod_prodtype.py accounts_passwords_pam_tally2 replace ubuntu2004~ubuntu1604,ubuntu1804,ubuntu2004
```

For an example of `remove`:

```bash
$ ./utils/mod_prodtype.py accounts_passwords_pam_tally2 remove ubuntu1604 ubuntu1804 ubuntu2004
````

### `utils/mod_checks.py` and `utils/mod_fixes.py` &ndash; programmatically modify check and fix applicability

These two utilities have identical usage. Both modifies the platform/product
applicability of various files (either OVAL or hardening content), similar to
`utils/mod_prodtype.py` above. They supports the following sub-commands:
applicability of various files (either OVAL or hardening content). They support
the following sub-commands:

- `add`: add the given platform(s) to the specified rule's OVAL check.
**Note**: Only applies to shared content.
Expand Down Expand Up @@ -613,9 +543,8 @@ $ python utils/generate_profile.py -i benchmark.xlsx list
To generate a rule for a specific control:

```
$ python utils/generate_profile.py -i benchmark.xlsx generate --product-type ocp -c 1.1.2
$ python utils/generate_profile.py -i benchmark.xlsx generate -c 1.1.2
documentation_complete: false
prodtype: ocp
title: |-
Ensure that the API server pod specification file ownership is set to root:root
description: 'Ensure that the API server pod specification file ownership is set to
Expand Down Expand Up @@ -646,7 +575,7 @@ template: PLACEHOLDER
To generate an entire section:

```
$ python utils/generate_profile.py -i benchmark.xlsx generate --product-type ocp -s 1
$ python utils/generate_profile.py -i benchmark.xlsx generate -s 1
```

The `PLACEHOLDER` values must be filled in later, ideally when the rules are
Expand Down
39 changes: 8 additions & 31 deletions docs/manual/developer/06_contributing_with_content.md
Original file line number Diff line number Diff line change
Expand Up @@ -518,9 +518,9 @@ the use of unversioned products here (e.g., `rhel` applies to `rhel7`,
correct extension for content of that type (e.g., `.sh` for `bash`
content). Further, all of these directories are optional and will only
be searched for content if present. Lastly, the product naming of
content will not override the contents of `platform` or `prodtype`
fields in the content itself (e.g., if `rhel7` is not present in the
`rhel7.xml` OVAL check platform specifier, it will be included in the
content will not override the contents of `platform` field in
the content itself (e.g., if `rhel7` is not present in the `rhel7.xml`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a great idea for work in future

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. I cleaned up connections between prodtype and platforms, but kept them out of the scope. It is a zoo there.

OVAL check platform specifier, it will be included in the
build artifacts but later removed because it doesn't match the platform).
This means that any shared (or templated) checks won't be searched if
a product-specific file is present but has the wrong applicability;
Expand All @@ -542,31 +542,6 @@ For more information about these utilities, please see their help text.
To interact with `rule.yml` files and the OVALs inside a rule directory,
the following utilities are provided:

#### `utils/mod_prodtype.py`

This utility modifies the prodtype field of rules. It supports several
commands:

- `mod_prodtype.py <rule_id> list` - list the computed and actual
prodtype of the rule specified by `rule_id`.

- `mod_prodtype.py <rule_id> add <product> [<product> ...]` - add
additional products to the prodtype of the rule specified by
`rule_id`.

- `mod_prodtype.py <rule_id> remove <product> [<product> ...]` -
remove products to the prodtype of the rule specified by `rule_id`.

- `mod_prodtype.py <rule_id> replace <replacement> [<replacement> ...]` -
do the specified replacement transformations. A replacement
transformation is of the form `match~replace` where `match` and
`replace` are a comma separated list of products. If all of the
products in `match` exist in the original `prodtype` of the rule,
they are removed and the products in `replace` are added.

This utility requires an up to date JSON tree created by
`rule_dir_json.py`.

#### `utils/mod_checks.py`

This utility modifies the `<affected>` element of an OVAL check. It
Expand Down Expand Up @@ -597,8 +572,10 @@ OVAL with the following commands:

- `mod_checks.py <rule_id> replace <replacement> [<replacement ...]` -
do the specified replacement against the platforms in the shared
OVAL. See the description of `replace` under `mod_prodtype.py` for
more information about the format of a replacement.
OVAL. A replacement transformation is of the form `match~replace`
where `match` and `replace` are a comma separated list of products.
If all of the platforms in `match` exist in the original `platform`
of the rule, they are removed and the platforms in `replace` are added.

This utility requires an up to date JSON tree created by
`rule_dir_json.py`.
Expand Down Expand Up @@ -634,7 +611,7 @@ fixes with the following commands:

- `mod_fixes.py <rule_id> <lang> replace <replacement> [<replacement ...]` -
do the specified replacement against the platforms in the shared
fix. See the description of `replace` under `mod_prodtype.py` for
fix. See the description of `replace` under `mod_checks.py` for
more information about the format of a replacement.

This utility requires an up-to-date JSON tree created by
Expand Down
8 changes: 0 additions & 8 deletions docs/modules/utils.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,14 +89,6 @@ utils.mod\_fixes module
:undoc-members:
:show-inheritance:

utils.mod\_prodtype module
--------------------------

.. automodule:: utils.mod_prodtype
:members:
:undoc-members:
:show-inheritance:

utils.regexify\_banner module
-----------------------------

Expand Down
2 changes: 0 additions & 2 deletions docs/workshop/data/accounts_tmout/rule_yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
documentation_complete: true

prodtype: fedora,ol7,ol8,rhcos4,rhel7,rhel8,rhel9,rhv4,sle12,sle15,ubuntu2004

title: 'Set Interactive Session Timeout'

description: |-
Expand Down
8 changes: 2 additions & 6 deletions docs/workshop/lab3_profiles.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ The value is also automatically substituted into OVAL checks, Ansible^(R)^ Playb

. In order to learn more about the parameterization, find and review the variable definition file.
. Press `Ctrl+P` and paste the value `var_accounts_tmout.var`.
. Click in the shown item to open the file.
. Click in the shown item to open the file.

. The variable has multiple options, which you can see in the options list:
+
Expand All @@ -349,7 +349,7 @@ Later, you add a new pair--variable name and selector--into the profile and you

. To apply the variable parameterization, the variable and the selector have to be added to the `travel` profile.
. Press `Ctrl+P` and paste the value `products/rhel8/profiles/travel.profile`.
. Click in the shown item to open the file.
. Click in the shown item to open the file.
+
As with the rule IDs, the variable values also belong to the `selections` list in the profile.
However, the entry for a variable has the format `variable=selector`. So in this case, the format of the list entry is `var_accounts_tmout=5_min`.
Expand Down Expand Up @@ -856,10 +856,6 @@ Let's use those new controls in our `travel` profile.
We do it by replacing respective selections with security controls invocations.
A security control is identified by `policy id:control id`, and when we select it, all rules that are applicable for the product that we build get selected by the profile.

Security controls can contain rules that are built only for a limited set of products by means of restricting the rule's `prodtype`, which is an link:https://complianceascode.readthedocs.io/en/latest/manual/developer/06_contributing_with_content.html#rules[optional rule attribute].
Such rules are skipped when expanding security controls into profiles, so a profiles that are defined using the same security controls on different products can vary.
This may surprise content authors, but it is a useful way how to make security controls reusable despite product differences s.a. different names of packages.

So let's modify the profile definition and alter its `selections`.

----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description: |-
rationale: |-
These settings configure the firewall to allow connections to an FTP server.

{{% if prodtype != "rhel7" %}}
{{% if product != "rhel7" %}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There never was a Jinja variable named prodtype. It doesn't make sense.

Also, it might be that the author (@cipherboy) actually meant to do the opposite. I'm not sure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\o hello @evgenyz -- Good question! It has been so long, I do not know. Likely I made a mistake and expected the build system to catch it...

The first line allows initial connections to the FTP server port.
FTP is an older protocol which is not very compatible with firewalls. During the initial FTP dialogue, the client
and server negotiate an arbitrary port to be used for data transfer. The <tt>ip_conntrack_ftp</tt> module is used by
Expand Down
4 changes: 2 additions & 2 deletions linux_os/guide/services/ntp/service_ntp_enabled/rule.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@ template:
vars:
servicename: ntp

{{% if prodtype in ["rhel7", "rhel8", "rhel9", "sle15"] %}}
{{% if product in ["rhel7", "rhel8", "rhel9", "sle15"] %}}
warnings:
- general:
{{% if prodtype == "rhel7" %}}
{{% if product == "rhel7" %}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There never was a Jinja variable named prodtype. It doesn't make sense.

The <pre>ntp.service</pre> Systemd unit is not available in {{{
full_name }}}. <pre>ntpd.service</pre> should be used instead together with the respective <pre>service_ntpd_enabled</pre> rule..
{{% else %}}
Expand Down
2 changes: 1 addition & 1 deletion linux_os/guide/services/ntp/service_ntpd_enabled/rule.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ template:

platform: package[ntp]

{{% if prodtype in ["rhel8", "rhel9", "sle15"] %}}
{{% if product in ["rhel8", "rhel9", "sle15"] %}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There never was a Jinja variable named prodtype. It doesn't make sense.

warnings:
- general:
The <pre>ntp</pre> package is not available in {{{ full_name }}}. Please
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
documentation_complete: true

# applicable only to products that ship OpenSSH<8.2
# prodtypes: ???

title: 'Set SSH Client Alive Count Max to zero'

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ template:
pkgname@ubuntu1804: audispd-plugins
pkgname@ubuntu2004: audispd-plugins

{{% if prodtype in ["rhel7", "rhel8", "rhel9"] %}}
{{% if product in ["rhel7", "rhel8", "rhel9"] %}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There never was a Jinja variable named prodtype. It doesn't make sense.

warnings:
- general:
This package is not available in {{{ full_name }}}. The correct package
is called <pre>audispd-plugins</pre>. The rule <pre> is suggested as a replacement.
This package is not available in {{{ full_name }}} [{{{ product }}}]. The correct package
is called <pre>audispd-plugins</pre>.
{{% endif %}}
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
documentation_complete: true

# Set prodtypes if needed, otherwise let the rule to be applicable for all products by omitting the prodtype declaration

title: "Don't target root user in the sudoers file"

description: |-
Expand Down
Loading
Loading