Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCP4: Add rule to check ACS sensor deployed #11675

Merged
merged 1 commit into from
Apr 5, 2024

Conversation

Vincent056
Copy link
Contributor

Description:

Add rule acs_sensor_exists, so we can check that ACS sensor is being deployed on the cluster

Copy link

github-actions bot commented Mar 12, 2024

Start a new ephemeral environment with changes proposed in this pull request:

ocp4 (from CTF) Environment (using Fedora as testing environment)
Open in Gitpod

Fedora Testing Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

Copy link

github-actions bot commented Mar 12, 2024

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:11675
This image was built from commit: 397a3e6

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:11675

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:11675 make deploy-local

@yuumasato yuumasato self-assigned this Mar 13, 2024
controls/pcidss_4.yml Outdated Show resolved Hide resolved
applications/openshift/general/acs_sensor_exists/rule.yml Outdated Show resolved Hide resolved
applications/openshift/general/acs_sensor_exists/rule.yml Outdated Show resolved Hide resolved
@Vincent056
Copy link
Contributor Author

/test

Copy link

openshift-ci bot commented Mar 18, 2024

@Vincent056: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

  • /test 4.13-e2e-aws-ocp4-cis
  • /test 4.13-e2e-aws-ocp4-cis-node
  • /test 4.13-e2e-aws-ocp4-e8
  • /test 4.13-e2e-aws-ocp4-high
  • /test 4.13-e2e-aws-ocp4-high-node
  • /test 4.13-e2e-aws-ocp4-moderate
  • /test 4.13-e2e-aws-ocp4-moderate-node
  • /test 4.13-e2e-aws-ocp4-pci-dss
  • /test 4.13-e2e-aws-ocp4-pci-dss-node
  • /test 4.13-e2e-aws-ocp4-stig
  • /test 4.13-e2e-aws-ocp4-stig-node
  • /test 4.13-e2e-aws-rhcos4-e8
  • /test 4.13-e2e-aws-rhcos4-high
  • /test 4.13-e2e-aws-rhcos4-moderate
  • /test 4.13-e2e-aws-rhcos4-stig
  • /test 4.13-images
  • /test 4.14-images
  • /test 4.15-e2e-aws-ocp4-cis
  • /test 4.15-e2e-aws-ocp4-cis-node
  • /test 4.15-e2e-aws-ocp4-e8
  • /test 4.15-e2e-aws-ocp4-high
  • /test 4.15-e2e-aws-ocp4-high-node
  • /test 4.15-e2e-aws-ocp4-moderate
  • /test 4.15-e2e-aws-ocp4-moderate-node
  • /test 4.15-e2e-aws-ocp4-pci-dss
  • /test 4.15-e2e-aws-ocp4-pci-dss-node
  • /test 4.15-e2e-aws-ocp4-stig
  • /test 4.15-e2e-aws-ocp4-stig-node
  • /test 4.15-e2e-aws-rhcos4-e8
  • /test 4.15-e2e-aws-rhcos4-high
  • /test 4.15-e2e-aws-rhcos4-moderate
  • /test 4.15-e2e-aws-rhcos4-stig
  • /test 4.15-images
  • /test 4.16-e2e-aws-ocp4-cis
  • /test 4.16-e2e-aws-ocp4-cis-node
  • /test 4.16-e2e-aws-ocp4-e8
  • /test 4.16-e2e-aws-ocp4-high
  • /test 4.16-e2e-aws-ocp4-high-node
  • /test 4.16-e2e-aws-ocp4-moderate
  • /test 4.16-e2e-aws-ocp4-moderate-node
  • /test 4.16-e2e-aws-ocp4-pci-dss
  • /test 4.16-e2e-aws-ocp4-pci-dss-node
  • /test 4.16-e2e-aws-ocp4-stig
  • /test 4.16-e2e-aws-ocp4-stig-node
  • /test 4.16-e2e-aws-rhcos4-e8
  • /test 4.16-e2e-aws-rhcos4-high
  • /test 4.16-e2e-aws-rhcos4-moderate
  • /test 4.16-e2e-aws-rhcos4-stig
  • /test 4.16-images
  • /test e2e-aws-ocp4-cis
  • /test e2e-aws-ocp4-cis-node
  • /test e2e-aws-ocp4-e8
  • /test e2e-aws-ocp4-high
  • /test e2e-aws-ocp4-high-node
  • /test e2e-aws-ocp4-moderate
  • /test e2e-aws-ocp4-moderate-node
  • /test e2e-aws-ocp4-pci-dss
  • /test e2e-aws-ocp4-pci-dss-node
  • /test e2e-aws-ocp4-stig
  • /test e2e-aws-ocp4-stig-node
  • /test e2e-aws-rhcos4-e8
  • /test e2e-aws-rhcos4-high
  • /test e2e-aws-rhcos4-moderate
  • /test e2e-aws-rhcos4-stig
  • /test images

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-ComplianceAsCode-content-master-4.13-images
  • pull-ci-ComplianceAsCode-content-master-4.14-images
  • pull-ci-ComplianceAsCode-content-master-4.15-images
  • pull-ci-ComplianceAsCode-content-master-4.16-images
  • pull-ci-ComplianceAsCode-content-master-images

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@Vincent056
Copy link
Contributor Author

/test e2e-aws-ocp4-pci-dss

@@ -0,0 +1,2 @@
---
default_result: FAIL
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This goes back to @yuumasato's comment, but now that we have a manual remediation, we should be testing that it works in the second scan, right?


ocil_clause: 'ACS Sensor is not deployed'

{{% set jqfilter = '[ .items[] | select(.metadata.name == "sensor" and .metadata.labels["app.kubernetes.io/name"] == "stackrox") | .status.availableReplicas]' %}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable so long as it's safe to rely on sensor as the deployment name and stackrox as the label.

@dashrews78 does this look ok to you?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on the surface looks OK. There may be another caveat though. Just because sensor is deployed doesn't necessarily mean it is connected to a central. But that may be a more advanced workload type check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you know where I can check so we know it is being connected to the central?

Copy link

@SimonBaeumer SimonBaeumer Mar 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Luis could answer this, I've sent him this conversation.
My gut says it is sufficient to look into the Deployment status. A non-connecting sensor eventually will go to CrashLoopBackoff.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If ROX_PREVENT_SENSOR_RESTART_ON_DISCONNECT is enabled (which it should be by default), sensor should not go to CrashLoopBackoff. The only way to know for sure if sensor is connected with central is by checking logs, metrics, or central's UI.

I'm lacking some context here but I would say knowing if sensor is connected is not necessary because even if sensor is not connected to central, if sensor is deployed, it should flush all the resources to central upon reconnection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the input, I wonder if there is a way we can find it out through any of the kube api resources? metrics might not be easily accessed by CO at the moment.

@rhmdnd
Copy link
Collaborator

rhmdnd commented Mar 18, 2024

/test e2e-aws-ocp4-pci-dss

I don't think this tested anything new from the patch since we're not including the new rule in a profile, yet.

Add rule acs_sensor_exists, so we can check that ACS sensor is being deployed on the cluster
Copy link

openshift-ci bot commented Apr 1, 2024

@Vincent056: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/4.13-images 397a3e6 link true /test 4.13-images

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Copy link

codeclimate bot commented Apr 1, 2024

Code Climate has analyzed commit 397a3e6 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.3% (0.0% change).

View more on Code Climate.

@BhargaviGudi
Copy link
Collaborator

Verification failed with 4.16.0-0.nightly-2024-03-31-180021 + compliance-operator.v1.4.0 + rhacs-operator.v4.4.0

  1. Install Co
  2. Install rhacs-operator
$ oc get pods -n rhacs-operator 
NAME                                                 READY   STATUS    RESTARTS   AGE
rhacs-operator-controller-manager-6f647b746b-jsh4d   1/1     Running   0          6h27m
  1. Create ssb
$ oc compliance bind -N test profile/upstream-ocp4-pci-dss-4-0 profile/upstream-ocp4-pci-dss-node-4-0
Creating ScanSettingBinding test
$ oc get scan
NAME                                    PHASE   RESULT
upstream-ocp4-pci-dss-4-0               DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-master   DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-worker   DONE    NON-COMPLIANT
$ oc get rules | grep acs
[bgudi@bgudi-thinkpadt14sgen2i openshift]$ oc get cr
NAME                                                              STATE
upstream-ocp4-pci-dss-4-0-api-server-encryption-provider-cipher   NotApplied
upstream-ocp4-pci-dss-4-0-audit-profile-set                       NotApplied
[bgudi@bgudi-thinkpadt14sgen2i openshift]$ oc get ccr -l compliance.openshift.io/automated-remediation=,compliance.openshift.io/check-status=FAIL
NAME                                                              STATUS   SEVERITY
upstream-ocp4-pci-dss-4-0-api-server-encryption-provider-cipher   FAIL     medium
upstream-ocp4-pci-dss-4-0-audit-profile-set                       FAIL     medium

Observation 1: upstream-ocp4-pci-dss-4-0-acs-sensor-exists is failed even though acs was installed on the cluster

$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    FAIL     medium
$ oc describe ccr upstream-ocp4-pci-dss-4-0-acs-sensor-exists
Name:         upstream-ocp4-pci-dss-4-0-acs-sensor-exists
Namespace:    openshift-compliance
Labels:       compliance.openshift.io/check-severity=medium
              compliance.openshift.io/check-status=FAIL
              compliance.openshift.io/scan-name=upstream-ocp4-pci-dss-4-0
              compliance.openshift.io/suite=test
Annotations:  compliance.openshift.io/rule: acs-sensor-exists
API Version:  compliance.openshift.io/v1alpha1
Description:  Ensure that Advanced Cluster Security (ACS) Sensor is deployed
Red Hat Advanced Cluster Security (ACS) for Kubernetes provides comprehensive security for containerized environments. It offers deep visibility into deployed resources across Kubernetes clusters, enabling teams to detect vulnerabilities in all images, manage compliance, and enforce security policies. By integrating ACS into the Kubernetes environment, organizations can automate security checks and configurations, ensuring that every deployed application is scanned and secured according to best practices and organizational policies. Sensor is the service responsible for analyzing and monitoring the cluster. Sensor listens to the OpenShift Container Platform or Kubernetes API and Collector events to report the current state of the cluster. Sensor also triggers deploy-time and runtime violations based on RHACS Cloud Service policies. In addition, Sensor is responsible for all cluster interactions, such as applying network policies, initiating reprocessing of RHACS Cloud Service policies, and interacting with the Admission controller.
Id:    xccdf_org.ssgproject.content_rule_acs_sensor_exists
Kind:  ComplianceCheckResult
Metadata:
  Creation Timestamp:  2024-04-01T16:07:23Z
  Generation:          1
  Managed Fields:
    API Version:  compliance.openshift.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:description:
      f:id:
      f:metadata:
        f:annotations:
          .:
          f:compliance.openshift.io/rule:
        f:labels:
          .:
          f:compliance.openshift.io/check-severity:
          f:compliance.openshift.io/check-status:
          f:compliance.openshift.io/scan-name:
          f:compliance.openshift.io/suite:
        f:ownerReferences:
          .:
          k:{"uid":"3ac6fa8a-461e-41d7-96ce-4f13aef69f27"}:
      f:rationale:
      f:severity:
      f:status:
    Manager:    compliance-operator
    Operation:  Update
    Time:       2024-04-01T16:07:23Z
  Owner References:
    API Version:           compliance.openshift.io/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  ComplianceScan
    Name:                  upstream-ocp4-pci-dss-4-0
    UID:                   3ac6fa8a-461e-41d7-96ce-4f13aef69f27
  Resource Version:        208234
  UID:                     841d3b17-a19b-4789-ad5b-adb3cfb61601
Rationale:                 ACS provides a method to continuously monitor and protect the Kubernetes environment against vulnerabilities and misconfigurations. This ensures that the infrastructure and applications are compliant with security standards and regulations, reducing the risk of security breaches.
Severity:                  medium
Status:                    FAIL
Events:                    <none>

Observation 2: upstream-ocp4-pci-dss-4-0-acs-sensor-exists rule is not listed in oc get rules
@Vincent056 Could you please help me check this issue? Thanks

@BhargaviGudi
Copy link
Collaborator

Verification passed with 4.16.0-0.nightly-2024-04-01-213440 + compliance-operator.v1.4.0 + rhacs-operator.v4.4.0

  1. Install CO and rhacs-operator
  2. Create ssb
$ oc compliance bind -N test profile/upstream-ocp4-pci-dss-4-0 profile/upstream-ocp4-pci-dss-node-4-0
Creating ScanSettingBinding test
$ oc get suite 
NAME   PHASE   RESULT
test   DONE    NON-COMPLIANT
$ oc get scan
NAME                                    PHASE   RESULT
upstream-ocp4-pci-dss-4-0               DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-master   DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-worker   DONE    NON-COMPLIANT
$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    FAIL     medium
  1. Apply manual remediation
$ sh applications/openshift/general/acs_sensor_exists/tests/ocp4/e2e-remediation.sh
+ echo 'Mimicking the behavior of a deployed scanner'
Mimicking the behavior of a deployed scanner
+ oc apply -f ocp-resources/e2e/acs-sensor-install.yaml --server-side=true
namespace/stackrox serverside-applied
deployment.apps/sensor serverside-applied
+ sleep 30
+ echo 'waiting for gitops deployment to exist'
waiting for gitops deployment to exist
++ oc wait -n stackrox --for=condition=Available --timeout=300s deployment/sensor
+ '[' -z 'deployment.apps/sensor condition met' ']'
+ echo 'waiting for gitops deployment to be ready'
waiting for gitops deployment to be ready
+ oc wait -n stackrox --for=condition=Available --timeout=300s deployment/sensor
deployment.apps/sensor condition met
$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    FAIL     medium
$ oc compliance rerun-now scansettingbinding test
Rerunning scans from 'test': upstream-ocp4-pci-dss-4-0, upstream-ocp4-pci-dss-node-4-0-master, upstream-ocp4-pci-dss-node-4-0-worker
Re-running scan 'openshift-compliance/upstream-ocp4-pci-dss-4-0'
Re-running scan 'openshift-compliance/upstream-ocp4-pci-dss-node-4-0-master'
Re-running scan 'openshift-compliance/upstream-ocp4-pci-dss-node-4-0-worker'
$ oc get suite
NAME   PHASE   RESULT
test   DONE    NON-COMPLIANT
$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    PASS     medium

@BhargaviGudi
Copy link
Collaborator

/unhold

@BhargaviGudi
Copy link
Collaborator

/lgtm

runtime violations based on RHACS Cloud Service policies. In addition, Sensor is
responsible for all cluster interactions, such as applying network policies,
initiating reprocessing of RHACS Cloud Service policies, and interacting with
the Admission controller.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor note is that we can probably add a link to either stackrox or ACS documentation as a way for reader to find more information.

cce@ocp4: CCE-86171-6

references:
pcidss: Req-6.3.2,Req-11.3.1.1,Req-11.5.1.1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to incorporate versions here before we release PCI-DSS v4.0.0.

Copy link
Collaborator

@rhmdnd rhmdnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

We can cleanup the documentation reference in a follow up. It also makes sense to do the versioning for the profile references separately from this change.

@rhmdnd
Copy link
Collaborator

rhmdnd commented Apr 5, 2024

I believe all @yuumasato's feedback was resolved.

@rhmdnd rhmdnd requested a review from yuumasato April 5, 2024 20:26
@rhmdnd rhmdnd dismissed yuumasato’s stale review April 5, 2024 20:27

We've incorporated Yuuma's feedback into the latest PR.

@rhmdnd rhmdnd merged commit 12735ee into ComplianceAsCode:master Apr 5, 2024
41 of 45 checks passed
@yuumasato yuumasato added this to the 0.1.73 milestone Apr 8, 2024
@Mab879 Mab879 added the Update Rule Issues or pull requests related to Rules updates. label May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Update Rule Issues or pull requests related to Rules updates.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants