I believe that there has been regression with v4.22.0 of the provider in terms of how credentials chain behaves when assume_role_with_web_identity{} is present in provider config.

[gdavison]

Originally posted by @azec-pdx in #25680 (comment)

@azec-pdx, please note that issues related to a released feature should be created as new issues. We only saw this because someone else linked to your comment.

---

I believe that there has been regression with v4.22.0 of the provider in terms of how credentials chain behaves when assume_role_with_web_identity{} is present in provider config.

Behavior before v4.22.0

I have a Terraform module with these files:

versions.tf

terraform {
  required_version = ">= 1.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.0"
    }
  }
}

providers.tf

provider "aws" {
  region              = var.region
  allowed_account_ids = [var.account_id]
  # Web Identity Role Federation only used in CI/CD
  assume_role_with_web_identity {
    role_arn           = var.ci_cd_role_ARN
    session_name       = var.ci_cd_sts_session_name
    duration           = var.ci_cd_sts_session_duration
    web_identity_token = var.ci_cd_web_identity_token
  }

  default_tags {
    tags = {
      # redacted
    }
  }
}

devops-account-globals.yaml (this is my top level YAML config that feeds TF root module variables)

terraform:
  vars:
    # Defines the order of labels which constitute name for all resources stacked:
    # {namespace}-{tenant}-{environment}-{stage}-{name}-{attributes}
    label_order:
     - "namespace"
     - "tenant"
     - "environment"
     - "stage"
     - "name"
     - "attributes"
    account_id: "REDACTED"
    ci_cd_role_ARN: "REDACTED"
    ci_cd_sts_session_name: "Terraform-CI-CD"
    ci_cd_sts_session_duration: "1h"
    # To be provided as TF_VAR_ci_cd_web_identity_token through CI/CD.
    # Setting empty string value breaks credentials chain lookup when OIDC IdP is not used (e.g.engineer workstations and
    # using Administrator SAML-federated IAM Role)
    ci_cd_web_identity_token: "FAILSAFE"

Before v4.22.0 with having ci_cd_web_identity_token set to value FAILSAFE, provider would attempt to 1st get STS session using STS: AssumeRoleWithWebIdentity and once that fails (because obviously "FAILSAFE" is not valid JWT token, intentionally), provider would proceed with other mechanisms to get AWS credentials. In scenario where I run terraform commands from my developer workstation, it would use my already persisted STS session saved in AWS_ environment variables.

Behavior after v4.22.0

All files above unchanged.
After upgrade to v4.22.0, with having ci_cd_web_identity_token set to value FAILSAFE, provider would attempt to 1st get STS session using STS: AssumeRoleWithWebIdentity and once that fails, provider doesn't even attempt further to continue with other methods to find AWS credentials (credentials chain stops).
I get the error:

Error: error configuring Terraform AWS Provider: IAM Role (REDACTED) cannot be assumed with web identity token.
│
│ There are a number of possible causes of this - the most common are:
│   * The web identity token used in order to assume the role is invalid
│   * The web identity token does not have appropriate permission to assume the role
│   * The role ARN is not valid
│
│ Error: failed to retrieve credentials, operation error STS: AssumeRoleWithWebIdentity, exceeded maximum number of attempts, 1, https response error StatusCode: 400, RequestID: 78672982-935a-4758-bd50-0f8a4c96f301, InvalidIdentityToken: The ID Token provided is not a valid JWT. (You may see this error if you sent an Access Token)

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[gdavison]

Thanks for reporting this issue, @azec-pdx. This is expected behaviour. Prior to v4.22, the parameter assume_role_with_web_identity was accepted but not hooked up, so it was ignored. #25681 fixes this, so that the provider now tries to assume the role. Since the parameters are invalid, the authentication fails.

One option for selecting different authentication methods based on variables, using a dynamic block, is as follows:

locals {
  web_identity_token = var.ci_cd_web_identity_token == "" ? null : var.ci_cd_web_identity_token
}

provider "aws" {
  # other parameters

  dynamic "assume_role_with_web_identity" {
    for_each = local. web_identity_token[*]
    content {
      role_arn           = var.ci_cd_role_ARN
      session_name       = var.ci_cd_sts_session_name
      duration           = var.ci_cd_sts_session_duration
      web_identity_token = local. web_identity_token
    }
}

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.