Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add condition to only stream flow logs from production VPCs to S3 #8001

Merged
merged 1 commit into from
Sep 20, 2024
Merged

Add condition to only stream flow logs from production VPCs to S3 #8001

merged 1 commit into from
Sep 20, 2024

Conversation

dms1981
Copy link
Contributor

@dms1981 dms1981 commented Sep 20, 2024

A reference to the issue / Description of it

#7607

How does this PR fix the problem?

As only production VPCs should be in scope, this adds a conditional check to ensure only production VPCs have logs streamed to S3

How has this been tested?

Tested with local plan

Deployment Plan / Instructions

Deploy through CI

Checklist (check x in [ ] of list items)

  • I have performed a self-review of my own code
  • All checks have passed
  • I have made corresponding changes to the documentation
  • Plan and discussed how it should be deployed to PROD (If needed)

Additional comments (if any)

{Please write here}

@dms1981 dms1981 requested a review from a team as a code owner September 20, 2024 14:22
@github-actions GitHub Actions
Copy link
Contributor

Trivy Scan Success

Show Output ```hcl

Trivy will check the following folders:
terraform/environments/core-vpc

Running Trivy in terraform/environments/core-vpc
2024-09-20T14:24:06Z INFO [db] Need to update DB
2024-09-20T14:24:06Z INFO [db] Downloading DB... repository="ghcr.io/aquasecurity/trivy-db:2"
2024-09-20T14:24:08Z INFO [vuln] Vulnerability scanning is enabled
2024-09-20T14:24:08Z INFO [misconfig] Misconfiguration scanning is enabled
2024-09-20T14:24:08Z INFO Need to update the built-in policies
2024-09-20T14:24:08Z INFO Downloading the built-in policies...
74.86 KiB / 74.86 KiB [-----------------------------------------------------------] 100.00% ? p/s 0s2024-09-20T14:24:08Z INFO [secret] Secret scanning is enabled
2024-09-20T14:24:08Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-09-20T14:24:08Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.55/docs/scanner/secret#recommendation for faster secret detection
2024-09-20T14:24:08Z INFO [terraform scanner] Scanning root module file_path="."
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="aws_iam_role.member-delegation" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="aws_iam_role_policy.member-delegation" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.dns-zone" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.dns_zone_extend" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.dns_zone_extend_private" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.resource-share" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.vpc" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.vpc_attachment" value="cty.NilVal"
2024-09-20T14:24:08Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.vpc_nacls" value="cty.NilVal"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z INFO [terraform executor] Ignore finding rule="aws-sns-enable-topic-encryption" range="monitoring.tf:9-15"
2024-09-20T14:24:09Z INFO Number of language-specific files num=0
2024-09-20T14:24:09Z INFO Detected config files num=4
trivy_exitcode=0

</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>

```hcl

*****************************

Checkov will check the following folders:
terraform/environments/core-vpc

*****************************

Running Checkov in terraform/environments/core-vpc
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
2024-09-20 14:24:12,188 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-pagerduty-integration?ref=0179859e6fafc567843cd55c0b05d325d5012dc4:None (for external modules, the --download-external-modules flag is required)
2024-09-20 14:24:12,188 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-aws-data-firehose?ref=2e58c8fd0b43ca8461dfd0c8cc5f43a1a9c49987:None (for external modules, the --download-external-modules flag is required)
2024-09-20 14:24:12,188 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-member-vpc?ref=c0475531b5c9f45a78a94fea441cd9ce91ad3c59:None (for external modules, the --download-external-modules flag is required)
2024-09-20 14:24:12,189 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/terraform-aws-observability-platform-tenant?ref=fbbe5c8282786bcc0a00c969fe598e14f12eea9b:None (for external modules, the --download-external-modules flag is required)
terraform scan results:

Passed checks: 169, Failed checks: 0, Skipped checks: 36


checkov_exitcode=0

CTFLint Scan Success

Show Output 
*****************************

Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.9.1)
tflint will check the following folders:
terraform/environments/core-vpc

*****************************

Running tflint in terraform/environments/core-vpc
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

Trivy Scan Success

Show Output 
*****************************

Trivy will check the following folders:
terraform/environments/core-vpc

*****************************

Running Trivy in terraform/environments/core-vpc
2024-09-20T14:24:06Z	INFO	[db] Need to update DB
2024-09-20T14:24:06Z	INFO	[db] Downloading DB...	repository="ghcr.io/aquasecurity/trivy-db:2"
2024-09-20T14:24:08Z	INFO	[vuln] Vulnerability scanning is enabled
2024-09-20T14:24:08Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-09-20T14:24:08Z	INFO	Need to update the built-in policies
2024-09-20T14:24:08Z	INFO	Downloading the built-in policies...
74.86 KiB / 74.86 KiB [-----------------------------------------------------------] 100.00% ? p/s 0s2024-09-20T14:24:08Z	INFO	[secret] Secret scanning is enabled
2024-09-20T14:24:08Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-09-20T14:24:08Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.55/docs/scanner/secret#recommendation for faster secret detection
2024-09-20T14:24:08Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="aws_iam_role.member-delegation" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="aws_iam_role_policy.member-delegation" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.dns-zone" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.dns_zone_extend" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.dns_zone_extend_private" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.resource-share" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.vpc" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.vpc_attachment" value="cty.NilVal"
2024-09-20T14:24:08Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.vpc_nacls" value="cty.NilVal"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T14:24:09Z	INFO	[terraform executor] Ignore finding	rule="aws-sns-enable-topic-encryption" range="monitoring.tf:9-15"
2024-09-20T14:24:09Z	INFO	Number of language-specific files	num=0
2024-09-20T14:24:09Z	INFO	Detected config files	num=4
trivy_exitcode=0

@dms1981 dms1981 added this pull request to the merge queue Sep 20, 2024
Merged via the queue into main with commit e738be2 Sep 20, 2024
12 checks passed
@dms1981 dms1981 deleted the fix/7607-scope-flow-logs-only-to-production branch September 20, 2024 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SteveLinden SteveLinden SteveLinden approved these changes

@sukeshreddyg sukeshreddyg sukeshreddyg approved these changes

Assignees
No one assigned
Labels
core networking
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dms1981 @SteveLinden @sukeshreddyg