Some aws resources don't work correctly with tags with empty/null string as a value

[gwohletz]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v0.12.31

• provider.aws v3.51.0

Affected Resource(s)

• aws_launch_template
• aws_autoscaling_group
• aws_iam_role
• aws_iam_instance_profile
• maybe others?

Terraform Configuration Files

code on these resources that specifies tags with a value of "" will endlessly show a need to update when planning, if instead these values are changed to "1" (or some non empty value) things work as expected.

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

  name = "masters.XXX"
  role = aws_iam_role.XXX.name
  tags = {
    "KubernetesCluster"                                     = "XXX"
    "Name"                                                  = "XXX"
    "environment"                                           = "XXX"
    "kubernetes"                                            = ""
    "kubernetes.io/cluster/kube-us-west-2-beta.addepar.com" = "owned"
  }
}

similar for the other resource types, the key is having "" as the value of the tag.  From what I can tell when you apply the plan the tags with the empty string values silently fail resulting in the next plan run thinking they still need to be updated

Debug Output

  ~ resource "aws_iam_instance_profile" "nodes-XXX" {
        arn         = "XXX"
        create_date = "2020-08-07T20:25:08Z"
        id          = "nodes.XXX"
        name        = "nodes.XXX"
        path        = "/"
        role        = "nodes.XXX"
      ~ tags        = {
            "KubernetesCluster"                                     = "XXX"
            "Name"                                                  = "nodes.XXX"
            "environment"                                           = "XXX"
          + "kubernetes"                                            = ""
            "kubernetes.io/cluster/XXX" = "owned"
        }
      ~ tags_all    = {
            "KubernetesCluster"                                     = "XXX"
            "Name"                                                  = "nodes.XXX"
            "environment"                                           = "XXX"
          + "kubernetes"                                            = (known after apply)
            "kubernetes.io/cluster/XXX" = "owned"
        }
        unique_id   = "XXX"
    }

Panic Output

Expected Behavior

After applying the plan subsequent plans should not show changes

Actual Behavior

same list of changes are listed

Steps to Reproduce

terraform plan -out=planfile; terraform apply; terraform plan

Important Factoids

References

• #0000

[gwohletz]

Correction the only resources that required setting the tags to have a non-empty value were:

aws_iam_instance_profile
aws_iam_role
aws_launch_template

The repeated updates of the aws_autoscaling_group resource was being triggered by the aws_launch_template being perceived to need an update, it actually handles the empty string tags correctly.

[anGie44]

Hi @gwohletz, thank you for raising this issue. I can definitely reproduce this behavior even with a completely unrelated resource like aws_appconfig_application. It's quite odd that we're seeing (known after apply) for that empty string value, but as a workaround in the meantime, have you already tried using a null instead or does that not create the effect you were looking for with the empty string? I found that it seems to correctly suppress the diff e.g.

resource "aws_launch_template" "test" {
  name = "tf-acc-test-1234"

  tags = {
    "kubernetes"                                            = null
    "kubernetes.io/cluster/kube-us-west-2-beta.addepar.com" = "owned"
  }
}

[gwohletz]

The tf code in question is actually autogenerated by KOPS (see: kubernetes/kops#12071 ) what I have done for now is to hand edit the autogenerated aws_launch_template code and change "" to "1" (as shown in the kops bug report i just referenced).

BTW, I have confirmed that when you "terraform apply" a plan that includes these blank value tags, the tags don't actually get created at all, despite the fact that the "terraform apply" returns success.

With regards to using null as the tag value:

I did some experiments with setting the tag value to null as you mention above, but what I found was setting an existing tag to null causes the tag to actually be deleted entirely as opposed to being created with no value. If I add additional tags with null as the value and run plan tf claims "No changes. Infrastructure is up-to-date." even though the tags do not exist.

To summarize, with "" plan thinks it needs to create the tags but then silently fails to do so, with null it removes the tag if it exists, or takes no action if it does not

[gwohletz]

bump in case you missed my answer to your question above

[hcsu]

I resolved this by deleting the launch template on AWS Web Console then re-created them on Terraform.

[gwohletz]

what does "anGie44 removed their assignment" mean? Has this issue been abandoned.

[johnsonaj]

This issue has been resolved in #30793 and merged to main in #31392. Will be released in v5.0.0

[github-actions]

This functionality has been released in v5.0.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.