Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Send flow logs to S3 #7995

Merged
merged 1 commit into from
Sep 20, 2024
Merged

Send flow logs to S3 #7995

merged 1 commit into from
Sep 20, 2024

Conversation

dms1981
Copy link
Contributor

@dms1981 dms1981 commented Sep 20, 2024

A reference to the issue / Description of it

#7607

How does this PR fix the problem?

Adds statement to build flow log that outputs to S3 bucket in core-logging.

Cleans up formatting of terraform for module.vpc a bit.

How has this been tested?

Tested with local plan and CI pipeline

Deployment Plan / Instructions

Deploy through CI

Checklist (check x in [ ] of list items)

  • I have performed a self-review of my own code
  • All checks have passed
  • I have made corresponding changes to the documentation
  • Plan and discussed how it should be deployed to PROD (If needed)

Additional comments (if any)

{Please write here}

@dms1981 dms1981 requested a review from a team as a code owner September 20, 2024 12:41
@github-actions GitHub Actions
Copy link
Contributor

Trivy Scan Success

Show Output ```hcl

Trivy will check the following folders:
terraform/environments/core-vpc

Running Trivy in terraform/environments/core-vpc
2024-09-20T12:43:27Z INFO [db] Need to update DB
2024-09-20T12:43:27Z INFO [db] Downloading DB... repository="ghcr.io/aquasecurity/trivy-db:2"
2024-09-20T12:43:29Z INFO [vuln] Vulnerability scanning is enabled
2024-09-20T12:43:29Z INFO [misconfig] Misconfiguration scanning is enabled
2024-09-20T12:43:29Z INFO Need to update the built-in policies
2024-09-20T12:43:29Z INFO Downloading the built-in policies...
74.86 KiB / 74.86 KiB [-----------------------------------------------------------] 100.00% ? p/s 0s2024-09-20T12:43:29Z INFO [secret] Secret scanning is enabled
2024-09-20T12:43:29Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-09-20T12:43:29Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.55/docs/scanner/secret#recommendation for faster secret detection
2024-09-20T12:43:29Z INFO [terraform scanner] Scanning root module file_path="."
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="aws_iam_role.member-delegation" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="aws_iam_role_policy.member-delegation" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.dns-zone" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.dns_zone_extend" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.dns_zone_extend_private" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.resource-share" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.vpc" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.vpc_attachment" value="cty.NilVal"
2024-09-20T12:43:29Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.vpc_nacls" value="cty.NilVal"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-sns-enable-topic-encryption" range="monitoring.tf:9-15"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO [terraform executor] Ignore finding rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z INFO Number of language-specific files num=0
2024-09-20T12:43:30Z INFO Detected config files num=4
trivy_exitcode=0

</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>

```hcl

*****************************

Checkov will check the following folders:
terraform/environments/core-vpc

*****************************

Running Checkov in terraform/environments/core-vpc
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
2024-09-20 12:43:33,209 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-pagerduty-integration?ref=0179859e6fafc567843cd55c0b05d325d5012dc4:None (for external modules, the --download-external-modules flag is required)
2024-09-20 12:43:33,209 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-aws-data-firehose?ref=2e58c8fd0b43ca8461dfd0c8cc5f43a1a9c49987:None (for external modules, the --download-external-modules flag is required)
2024-09-20 12:43:33,210 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/modernisation-platform-terraform-member-vpc?ref=c0475531b5c9f45a78a94fea441cd9ce91ad3c59:None (for external modules, the --download-external-modules flag is required)
2024-09-20 12:43:33,210 [MainThread  ] [WARNI]  Failed to download module github.com/ministryofjustice/terraform-aws-observability-platform-tenant?ref=fbbe5c8282786bcc0a00c969fe598e14f12eea9b:None (for external modules, the --download-external-modules flag is required)
terraform scan results:

Passed checks: 169, Failed checks: 0, Skipped checks: 36


checkov_exitcode=0

CTFLint Scan Success

Show Output 
*****************************

Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.9.1)
tflint will check the following folders:
terraform/environments/core-vpc

*****************************

Running tflint in terraform/environments/core-vpc
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0

Trivy Scan Success

Show Output 
*****************************

Trivy will check the following folders:
terraform/environments/core-vpc

*****************************

Running Trivy in terraform/environments/core-vpc
2024-09-20T12:43:27Z	INFO	[db] Need to update DB
2024-09-20T12:43:27Z	INFO	[db] Downloading DB...	repository="ghcr.io/aquasecurity/trivy-db:2"
2024-09-20T12:43:29Z	INFO	[vuln] Vulnerability scanning is enabled
2024-09-20T12:43:29Z	INFO	[misconfig] Misconfiguration scanning is enabled
2024-09-20T12:43:29Z	INFO	Need to update the built-in policies
2024-09-20T12:43:29Z	INFO	Downloading the built-in policies...
74.86 KiB / 74.86 KiB [-----------------------------------------------------------] 100.00% ? p/s 0s2024-09-20T12:43:29Z	INFO	[secret] Secret scanning is enabled
2024-09-20T12:43:29Z	INFO	[secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-09-20T12:43:29Z	INFO	[secret] Please see also https://aquasecurity.github.io/trivy/v0.55/docs/scanner/secret#recommendation for faster secret detection
2024-09-20T12:43:29Z	INFO	[terraform scanner] Scanning root module	file_path="."
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="aws_iam_role.member-delegation" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="aws_iam_role_policy.member-delegation" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.dns-zone" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.dns_zone_extend" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.dns_zone_extend_private" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.resource-share" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.vpc" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.vpc_attachment" value="cty.NilVal"
2024-09-20T12:43:29Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="module.vpc_nacls" value="cty.NilVal"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:31-50"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-iam-no-policy-wildcards" range="iam.tf:48"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-sns-enable-topic-encryption" range="monitoring.tf:9-15"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	[terraform executor] Ignore finding	rule="aws-cloudwatch-log-group-customer-key" range="../../modules/r53-resolver-logs/main.tf:13-17"
2024-09-20T12:43:30Z	INFO	Number of language-specific files	num=0
2024-09-20T12:43:30Z	INFO	Detected config files	num=4
trivy_exitcode=0

@dms1981 dms1981 added this pull request to the merge queue Sep 20, 2024
Merged via the queue into main with commit 2cf0277 Sep 20, 2024
12 checks passed
@dms1981 dms1981 deleted the feature/7607-start-vpc-logging-to-s3 branch September 20, 2024 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ASTRobinson ASTRobinson ASTRobinson approved these changes

Assignees
No one assigned
Labels
core networking
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dms1981 @ASTRobinson