Mark some scenarios as specific to SCE

[jan-cerny]

Description:

We introduce a new test scenarios header check that allows to mark test scenarios as specific to a single check engine type. For example, adding header check = sce to a test scenario marks this test scenario as specific only to SCE. If SCE check isn't available, such scenario will be skipped. If the check header isn't specified in a test scenario, the test scenario will work with any check type.

For SCE test scenarios we will also check if SCE support is installed in OpenSCAP in the back end and we will skip the test if SCE support isn't available there.

Rationale:

Fixes: #12030

Review Hints:

1. ./build_product -d rhel9
2. python3 tests/automatus.py rule --libvirt qemu:///system ssgts_rhel9 set_nftables_table
3. ADDITIONAL_CMAKE_OPTIONS="-DSSG_SCE_ENABLED=ON" ./build_product -d rhel9
4. repeat the automatus command in step 2
5. remove openscap-engine-sce from the virtual machine
6. repeat the automatus command in step 2

[github-actions]

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

[github-actions]

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:12052
This image was built from commit: 6602496

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:12052

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:12052 make deploy-local

[jan-cerny]

/test 4.15-images

[Mab879]

$  ADDITIONAL_CMAKE_OPTIONS="-DSSG_SCE_ENABLED=ON" ./build_product rhel9
$ python3 tests/automatus.py rule  --libvirt qemu:///system automatus_rhel9_4 set_nftables_table 
Setting console output to log level INFO
INFO - The data stream contains 3 Benchmarks
INFO - 0 - scap_org.open-scap_cref_ssg-rhel9-xccdf.xml
INFO - 1 - scap_org.open-scap_cref_rhel9-checks-sce-set_nftables_table.sh
INFO - 2 - scap_org.open-scap_cref_rhel9-checks-sce-ssh_keys_passphrase_protected.sh
INFO - Selected Benchmark is 0
INFO - To select a different Benchmark, use --xccdf-id-number option.
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/mburket/Developer/ComplianceAsCode/content/logs/rule-custom-2024-06-07-1527/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_set_nftables_table
ERROR - Script nftables_incorrect_family.fail.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in notchecked, instead of expected fail during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_set_nftables_table'.
ERROR - Script nftables_no_tables.fail.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in notchecked, instead of expected fail during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_set_nftables_table'.
ERROR - Script nftables_table_present.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in notchecked, instead of expected pass during initial stage 

I'm still getting some errors, after this fix. Is there something I'm missing?

[jan-cerny]

@Mab879 I think you're missing support for SCE in OpenSCAP running in your virtual machine. Check if openscap-engine-sce is installed in the VM and oscap --version contains the SCE capability.

This revealed a problem that this PR doesn't address the situation when the SCE check is going to be tested but OpenSCAP doesn't support SCE. We need to think how to address this situation.

[jan-cerny]

I have add logic that checks if the SCE tests are supported by OpenSCAP on the back end.

[codeclimate]

Code Climate has analyzed commit 6602496 and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category	Count
Complexity	1

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.4% (0.0% change).

View more on Code Climate.

[Mab879]

The new behavior works correctly.

Thanks.

[Mab879]

Waving the Rawhide tests as it appears to be failure in dnf.

Codeclimate finding can be waived for the moment.