Latest version (1.6.0) breaks the vault_aws_secret_backend_role resource

[atrepca]

Summary

The latest version of the provider - 1.6.0 - breaks the vault_aws_secret_backend_role resource; using the previous version - 1.5.0 - works as expected.

Terraform Version

Terraform v0.11.13

Affected Resource(s)

• vault_aws_secret_backend_role

Terraform Configuration Files

Tested using the example from the docs:

resource "vault_aws_secret_backend_role" "test" {
  backend = "aws"
  name    = "test"
  credential_type = "assumed_role"

  policy_document = <<EOT
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iam:*",
      "Resource": "*"
    }
  ]
}
EOT
}

Debug Output

Error: Error applying plan:

1 error(s) occurred:

* vault_aws_secret_backend_role.test: 1 error(s) occurred:

* vault_aws_secret_backend_role.test: error creating role "test" for backend "aws": Error making API request.

URL: PUT https://REDACTED:8200/v1/aws/roles/test
Code: 500. Errors:

* 1 error occurred:

* either policy or arn must be provided

Important Factoids

Installing version 1.5.0 of the provider and using the code below works fine:

resource "vault_aws_secret_backend_role" "test" {
  backend = "aws"
  name    = "test"

  policy = <<EOT
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iam:*",
      "Resource": "*"
    }
  ]
}
EOT
}

[tyrannosaurus-becks]

Hi @atrepca ! Thanks for opening this ticket.

I can't duplicate this locally but I suspect it has to do with the version of Vault I'm testing against. What version of Vault are you on?

[tyrannosaurus-becks]

Actually, I found that I can duplicate a failure on version 11.6 of Vault. Does #329 fix this for you?

[atrepca]

Ugg, had an old Vault server, upgraded from 0.10.0 to 1.1.0 since this was opened and the 1.6.0 provider works fine now. Closing this, sorry for the trouble and thanks for looking!

[tyrannosaurus-becks]

Thanks @atrepca ! Actually, though, we definitely want the code to maintain backwards compatibility so in a perfect world it would work for both of us. Going to leave this open until we can get into that state.

[tyrannosaurus-becks]

Hmmmmmm, actually, in coding up a fix, I may be incorrect, but it looks like if I support previous versions of Vault, that's different from supporting previous versions of the schema.

The Terraform path for deprecating and replacing attributes includes adding a deprecation warning, and mapping uses of the old value to the new value that's replacing it. That's how the code is currently, so it's working in that respect.

If I instead break from that and preserve old styles of providing attributes, that will cause both versions to work but I won't be following a proper deprecation path.

I think you were right in the first place. I think we should follow normal Terraform provider paradigms and continue to support the current version of Vault. Those wanting to point at past versions of Vault can stick with past versions of the Terraform Vault Provider.

I think there's no more work here to be done.

[fatmcgav]

@tyrannosaurus-becks Is there some kind of version compatibility matrix available?

As I came across this issue after testing the latest 1.7.0 provider release, which threw a load of errors such as:

Error: module.aws_secret_backends.module.aws_elastic_test.vault_aws_secret_backend_role.raw_policy[1]: "credential_type": required field is not set

As a point of principle, I'd also consider that a breaking change, as it's changing the API that existing TF code conforms to... However that is an aside...

Those wanting to point at past versions of Vault can stick with past versions of the Terraform Vault Provider.

This is unfortunate, as it's taken what should've been a simple "Bump the Terraform provider version and profit" change to a "First, upgrade the Vault server to this latest version before you can do anything..."

[tyrannosaurus-becks]

Hi @fatmcgav ,

Generally we're trying to keep the current version of the Terraform Vault Provider in tune with the current version of Vault. When the Vault API changes, we generally can't support the current Vault API while also supporting past Vault APIs and following best practices around deprecations and removals. It's definitely a trade-off. We do err on the side of following best practices and keeping up with current Vault.

Definitely would recommend not setting the provider to auto-update versions unless you're keeping up with present Vault as well. Apologies for the inconvenience!

[patoarvizu]

I'm still a bit confused. I understand the need for following the best practices, but I'd like to understand what this means in terms of upgrading.

Let's say I'm on Vault 1.0 and terraform-provider-vault 1.6 and I want to create a vault_aws_secret_backend_role resource but use the new parameters on the endpoint. Are you saying that the provider should first mark the breaking arguments as Deprecated for at least 1 release (I'm assuming 1.7 at this point), then the new parameters to the endpoint would be available as arguments of the resource in the following version of the provider (1.8)? Or can the new arguments be exposed on the same version that the deprecations happen?

In other words, if I wanted to have parity between the provider and the Vault API, would I have to wait a minimum of two releases, or can it happen on the next one?

I hope that made sense.

[tyrannosaurus-becks]

Hi @patoarvizu ! Best practices for deprecations, removals, and renames are here. Best practices for versioning are here. We follow both wherever possible.