update code for for large networkpolicy and egress firewall rule

[liqcui]

Type of change

• Refactor
• [ Yes ] New feature
• Bug fix
• Optimization
• Documentation Update

Description

To simulate a customer zero trust environment. We deny all traffic of ingress and egress for network policy and egress firewall, adding a whitelist to open essential ports.

For network policy, create two type workloads to send network traffic.
pod to pod traffic in the same namespace - node density heavy, create two pods, one pod is postgres DB, another is client to insert data into database continuously.
across namespace traffic, it will query dns to prometheus pod hostname from others namespace.
For egress firewall, adding allow/deny rules for egress firewall. We create one pod to send network traffic from the OCP network to the internet continuously.

Creating 10500 pods, 20k networkpolicy, 200k egress firewall rule(acl) first, then create 9 additional new ns with 20k networkpolicy, 200k egress firewall rule, then delete those additional ns to simulate customer maybe remove unuseless network policy.

After all pods/network policy/egress firewall created and ready, wait for 90 minutes to create a new namespace and add new network-policy, create new pods again to check OVN init-sync time.

Related Tickets & Documents

• Related Issue #
  https://issues.redhat.com/browse/OCPQE-17154
• Closes #

Checklist before requesting a review

• [ Yes ] I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.

Testing

• Please describe the System Under Test.

master: 16 vCPUs 64 GiB 

worker: 8 vCPUs 32 GiB | x86 | 100 worker node

• Please provide detailed steps to perform tests related to this code change.

  https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/scale-ci/job/liqcui-e2e-benchmarking-multibranch-pipeline/job/kube-burner/373/console

  The key variable to set pods number, networkpolicy and egressfirewall rules

  export POD_RPLICAS=25. # Create 75 pods each namespace

  export NETWORKPOLICY_RPLICAS=75. # Create 150 networkpolicy each namespace

  export EGRESS_FIREWALL_POLICY_TOTAL_NUM=600 # Create 600 egress firewall each namespace

  export QPS=50

  export BURST=100

  export MAX_WAIT_TIMEOUT=5h

  export JOB_TIMEOUT=8h

• How were the fix/results from this change verified? Please provide relevant screenshots or results.

https://docs.google.com/document/d/1X3YNRXmPB1boDeEk0OCZdZPoYdA2B0q2yYrDzJWBcaw/edit

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: liqcui
Once this PR has been reviewed and has the lgtm label, please assign morenod for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Hi @liqcui. Thanks for your PR.

I'm waiting for a cloud-bulldozer member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[liqcui]

/assign @qiliRedHat @paigerube14 Please review the PR when you are available, thanks!

[liqcui]

/cc @qiliRedHat @paigerube14

[krishvoor]

@liqcui Can you please sign-off the PR?

[liqcui]

@krishvoor Sorry, I don't quite understand what's your mean, you mean the case isn't suitable for merge into e2e-benchmarking code repository now? right?

[krishvoor]

@liqcui we recently added DCO a mandatory check which needs users to sign commits first this could be performed via git commit -s

[liqcui]

@liqcui we recently added DCO a mandatory check which needs users to sign commits first this could be performed via git commit -s
Thank you, I have sign-off the PR now!

[liqcui]

/cc @mohit-sheth

[qiliRedHat]

@liqcui This sentence seems not completed.

[liqcui]

@qiliRedHat Thank you for taking time to review my pr, I have updated the README, please review again, if still have something that didn't describe clearly. thanks!

[qiliRedHat]

Could you be more specific about the exact pod name?
Could you describe when the OVN DB sync will happen, is that right after applying all the network policies and egress firewalls? How long it usually take with the default settings (the default WAIT_OVN_DB_SYNC_TIME setting). What will happen if this is set too short?

[liqcui]

@qiliRedHat I will update the pod name since the script did a minor change. got the pod name from new testing.

[liqcui]

Update the pod name now!

[qiliRedHat]

@liqcui I searched and found 11 'amadeus', please remove or replace them.

[liqcui]

Fixed in new code, thanks!

[qiliRedHat]

I didn't find the case-engress-firewall.yaml file.

[liqcui]

This file will be automatically generated by https://github.com/liqcui/e2e-benchmarking/blob/large-networkpolicy-egress/workloads/kube-burner/common.sh#L195