Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test wrapper around SCAPVal tool #4327

Merged
merged 7 commits into from
May 9, 2019

Conversation

jan-cerny
Copy link
Collaborator

Runs SCAPVal tool on all built datastreams and then parses the results.

This wrapper will run in a special job in our Jenkins, which will help us to preserve SCAP 1.3 conformance.

Example usage:

python run_scapval.py --scap-version 1.3 \
--scapval-path /opt/scapval/scapval-1.3.2.jar \
--build-dir /home/jcerny/content/build

@jan-cerny jan-cerny added this to the 0.1.45 milestone May 3, 2019
@jan-cerny
Copy link
Collaborator Author

SCAP 1.3 conformance results (as of HEAD = 8cb2d0f):

  • ssg-fuse6-ds-1.3.xml fails on SRC-15
  • ssg-debian8-ds-1.3.xml fails on SRC-15
  • ssg-ol7-ds-1.3.xml fails on SRC-330
  • other *-ds-1.3.xml pass

See attached reports
reports.zip

@jan-cerny
Copy link
Collaborator Author

  • ssg-fuse6-ds-1.3.xml fails on SRC-15
    • there are 3 CPEs used in the benchamrk, but two of them are not defined in the CPE dictionary (they're commented out) and they miss OVALs
  • ssg-debian8-ds-1.3.xml fails on SRC-15
    • the Debian 8 CPE dictionary contains only Debian 8 CPE, but doesn't contain machine CPE or recently introduced package CPEs
  • ssg-ol7-ds-1.3.xml fails on SRC-330
    • OL7 profiles unselect "sap" profile instead of "sap" group. Looks like a bug in OpenSCAP in xsl/xccdf_1.1_to_1.2.xsl. Try to run this command that converts XCCDF 1.1 to XCCDF 1.2: xsltproc --stringparam reverse_DNS "org.ssgproject.content" --output /tmp/xccdf12.xml ~/openscap/xsl/xccdf_1.1_to_1.2.xsl build/ssg-ol7-xccdf.xml . You will see that <select idref="sap" selected="false"/> is incorrectly translated to <select idref="xccdf_org.ssgproject.content_profile_sap" selected="false"/>. A proper fix is to fix the XSLT templates in OpenSCAP. A workaround is to rename either the sap Group or rename sap OL7 profile so that the 2 names don't conflict.

@jan-cerny
Copy link
Collaborator Author

Fuse6 and Debian8 problems are addressed in PR #4329 .
OL7 problems are addressed in PR #4332 .

@ggbecker ggbecker self-assigned this May 9, 2019
@ggbecker
Copy link
Member

ggbecker commented May 9, 2019

Please rebase with master.

tests/run_scapval.py Outdated Show resolved Hide resolved
Runs SCAPVal tool on all built datastreams and then
parses the results.
This wrapper will run in a special job in our Jenkins,
which will help us to preserve SCAP 1.3 conformance.
Example usage:
python run_scapval.py --scap-version 1.3 \
--scapval-path /opt/scapval/scapval-1.3.2.jar \
--build-dir content/build
It would be useful to see the SCAPVAl stderr in the test output
to determine why SCAPVal failed.
@jan-cerny
Copy link
Collaborator Author

@ggbecker I have rebased and caught exception.

@scrutinizer-notifier
Copy link

The inspection completed: 4 new issues, 6 updated code elements

@ggbecker
Copy link
Member

ggbecker commented May 9, 2019

LGTM

@ggbecker ggbecker merged commit e7fb422 into ComplianceAsCode:master May 9, 2019
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants