Can't find explanation why do I need to provide S3 bucket name

[IrmantasMarozas]

Describe what happened:
We just upgraded to the latest version to take advantage of forwarding log group tags for the logs that of the log group that triggered the function. Log group tags are not appearing in DataDog (even though logs are somehow forwarded) and function keeps failing due to missing s3 bucket. We needed this functionality to be able to differentiate between logs from different services.

Maybe I'm misunderstanding what DD_FETCH_LOG_GROUP_TAGS parameter does...

[ERROR] ValueError: Required parameter bucket_name not set
Traceback (most recent call last):
  File "/var/task/datadog_lambda/wrapper.py", line 124, in _call_
    self.response = self.func(event, context, **kwargs)
  File "/var/task/lambda_function.py", line 92, in datadog_forwarder
    metrics, logs, trace_payloads = split(transform(enrich(parse(event, context))))
  File "/var/task/parsing.py", line 96, in parse
    return normalize_events(events, metadata)
  File "/var/task/parsing.py", line 757, in normalize_events
    for event in events:
  File "/var/task/parsing.py", line 473, in awslogs_handler
    formatted_tags = account_cw_logs_tags_cache.get(logs["logGroup"])
  File "/var/task/cache.py", line 431, in get
    self._refresh()
  File "/var/task/cache.py", line 199, in _refresh
    tags_fetched, last_modified = self.get_cache_from_s3()
  File "/var/task/cache.py", line 174, in get_cache_from_s3
    cache_object = s3_client.Object(DD_S3_BUCKET_NAME, self.CACHE_FILENAME)
  File "/var/runtime/boto3/resources/factory.py", line 474, in create_resource
    client=self.meta.client)(*args, **kwargs)
  File "/var/runtime/boto3/resources/base.py", line 119, in _init_
    'Required parameter {0} not set'.format(identifier))

Describe what you expected:
I expected the function to continue working as before just this time attach to the logs the tags of the CloudWatch Log Group that triggered the forwarder function.

Steps to reproduce the issue:

Not sure how to provide the steps, I've deployed function using terraform by downloading latest zip file from releases page in GitHub.

resource "aws_lambda_function" "lambda" {
  function_name = local.lambda_function_name
  filename         = "aws-dd-forwarder-3.44.0.zip"
  source_code_hash = filebase64sha256("aws-dd-forwarder-3.44.0.zip")
  role                = aws_iam_role.iam_for_lambda.arn
  handler          = "lambda_function.lambda_handler"
  runtime          = var.runtime

  environment {
    variables = {
       DD_API_KEY_SECRET_ARN = aws_secretsmanager_secret.dd_api_key.arn
       DD_ENHANCED_METRICS = false
       DD_FETCH_LOG_GROUP_TAGS = true
    }
  }

  depends_on = [
    aws_iam_role_policy_attachment.lambda,
    aws_cloudwatch_log_group.lambda,
  ]
}

[IrmantasMarozas]

Okay, I think I found why S3 bucket is used #540 , to my understanding it's to reduce the number of requests to AWS API?

[abrahamvarricatt]

Your situation is remarkably similar to what I just working with.

A couple of points that stand out to me,

• using version 3.44.0
  This took me a good while to puzzle out - but version 3.44.0 does not support the feature to forward log group tags. At time of typing, version 3.66.0 does support the feature. Please keep in mind, when you upgrade that you will also need to update the runtime from python3.7 to python3.8

• using Booleans in terraform environment variables
  This was a nasty one! Your terraform code,

environment {
    variables = {
       DD_API_KEY_SECRET_ARN = aws_secretsmanager_secret.dd_api_key.arn
       DD_ENHANCED_METRICS = false
       DD_FETCH_LOG_GROUP_TAGS = true
    }
  }

records those true/false values as Boolean. That means if you check the lambda function environment variables, they get recorded as 1/0. BUT the lambda function is looking for a string named true. See here =

datadog-serverless-functions/aws/logs_monitoring/base_tags_cache.py

Line 94 in bf8892b

return os.environ.get("DD_FETCH_LOG_GROUP_TAGS", "false").lower() == "true"

If you add quotes in your terraform code to look like,

environment {
    variables = {
       DD_API_KEY_SECRET_ARN = aws_secretsmanager_secret.dd_api_key.arn
       DD_ENHANCED_METRICS = "false"
       DD_FETCH_LOG_GROUP_TAGS = "true"
    }
  }

that should be enough to get the LogGroup tags to get attached.

And (as far as version 3.66.0 is concerned) - you do not have to attach an S3 bucket to get the LogGroup Tags to be attached. At least, it works for me. Not sure how it affects lambda costs. 😕

Hope this helps!

[RaphaelAllier]

Hello,

The variable DD_S3_BUCKET_NAME is used to reference a bucket to use for caching the various tags we may attach to logs in the Forwarder, for instance log group tags. This logic is defined here

If you intend on attaching those tags, we implemented a cache to prevent hitting rate limits.
I'm doing some community issues clean up and closing this one, feel free to reopen it if you're still encountering issues