Organization accounts attribute is null for data source: aws_organizations_organization while using child AWS account for AWS provider

[marshall7m]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform: 0.14.8
AWS-Provider: 3.35.0

Affected Resource(s)

• aws_organizations_organization

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

data "aws_organizations_organization" "this" {}

output "data_org" {
  value = data.aws_organizations_organization.this.accounts[*].id
}

Debug Output

Panic Output

Expected Behavior

data_org = {
  "accounts" = [
      {
          arn = "arn:aws:organizations::111111111111:account/o-1234567/22222222222"
          email = "baz@bar.com"
          id = "22222222222"
          name = "child_account"
      }
  ]
  "arn" = "arn:aws:organizations::111111111111:organization/o-1234567"
  "aws_service_access_principals" = toset([])
  "enabled_policy_types" = toset([])
  "feature_set" = "ALL"
  "id" = "o-1234567"
  "master_account_arn" = "arn:aws:organizations::111111111111:account/o-1234567/111111111111"
  "master_account_email" = "foo@bar.com"
  "master_account_id" = "111111111111"
  "non_master_accounts" = tolist(null) /* of object */
  "roots" = tolist(null) /* of object */
}

Actual Behavior

Outputs:

data_org = {
  "accounts" = tolist(null) /* of object */
  "arn" = "arn:aws:organizations::111111111111:organization/o-1234567"
  "aws_service_access_principals" = toset([])
  "enabled_policy_types" = toset([])
  "feature_set" = "ALL"
  "id" = "o-1234567"
  "master_account_arn" = "arn:aws:organizations::111111111111:account/o-1234567/111111111111"
  "master_account_email" = "foo@bar.com"
  "master_account_id" = "111111111111"
  "non_master_accounts" = tolist(null) /* of object */
  "roots" = tolist(null) /* of object */
}

Steps to Reproduce

1. terraform apply

Important Factoids

• Verified that there's an AWS organization defined in the management AWS account via the AWS console

UPDATE:

• Tried using the child AWS account for the AWS provider but resulted in the output defined under Actual Behavior
• Tried using the master AWS account that hosts the organization and it works as defined under Expected Behavior

References

[bill-rich]

Hi @marshall7m! I tried to reproduce this issue, but was not able to when using the account that owns the organization. Can you provide an organization config that would reproduce the issue?

[marshall7m]

Hi @marshall7m! I tried to reproduce this issue, but was not able to when using the account that owns the organization. Can you provide an organization config that would reproduce the issue?

Hi @bill-rich! I'm not sure what attributes you're looking for from the organization config, but here's a redacted output from running aws organizations describe-organization

{
    "Organization": {
        "Id": "o-1234567",
        "Arn": "arn:aws:organizations::111111111111:organization/o-1234567",
        "FeatureSet": "ALL",
        "MasterAccountArn": "arn:aws:organizations::111111111111:account/o-1234567/111111111111",
        "MasterAccountId": "111111111111",
        "MasterAccountEmail": "mastermail@private.com",
        "AvailablePolicyTypes": [
            {
                "Type": "SERVICE_CONTROL_POLICY",
                "Status": "ENABLED"
            }
        ]
    }
}

I also updated my original post as the problem is only in regards to using the data.aws_organization_organization with the AWS provider configured with a child AWS account.

[cmd-werner-diers]

Hello,

While using child/member AWS account from my organization I have the same issue:

• Terraform: 0.15.0
• Provider : hashicorp/aws v3.47.0

data "aws_organizations_organization" "org" {}

output "org" {
  value = data.aws_organizations_organization.org
}

$terraform apply
Acquiring state lock. This may take a few moments...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:

Terraform will perform the following actions:

Plan: 0 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + org = {
      + accounts                      = null
      + arn                           = "arn:aws:organizations::12345678910:organization/o-myorgId"
      + aws_service_access_principals = []
      + enabled_policy_types          = []
      + feature_set                   = "ALL"
      + id                            = "o-myorgId"
      + master_account_arn            = "arn:aws:organizations::12345678910:account/o-myorgId/12345678910"
      + master_account_email          = "master_account_email@example.com"
      + master_account_id             = "12345678910"
      + non_master_accounts           = null
      + roots                         = null
    }

Note that this works if the execution is performed in the organization master account. If this is the expected behaviour, it would be great to update the documentation to highlight it.

[cob16]

This behaviour is documented in the resource but it does not fit my usecase

According to the code, accounts will always be null unless run in the root AWS account due to this if statement

terraform-provider-aws/aws/data_source_aws_organizations_organization.go

Line 159 in 915f5f5

if aws.StringValue(org.Organization.MasterAccountId) == meta.(*AWSClient).accountid {

looking at the docs

This operation can be called only from the organization's management account or by a member account that is a delegated administrator for an AWS service.

When running inside an account that is a delegated administrator I can run aws organizations list-accounts as expected.

Given this, I think the current behaviour is a bug as it is not expected. As returning null give users a false sense of security that the data has been fetched

IMO a better behaviour would be to make the API call and return an access denied error as this can then be worked around by the user.

[cob16]

After writing up this pr to remove the check from the code I realised that this could be a breaking change for those who are using this data source to get other attributes such as master_account_id etc.

It would be good to get some feedback from the maintainers on how this should be fixed. IMO to keep backwards compatibility it might be better to move this API call out to a new data-source and deprecate the account field 🤷

I am happy to make the PR if I can get some direction here

[rnikoopour]

Hi All,

I'm running into this same issue and would like to implement a solution. Should we look to modify this same resource or should we look to provide a aws_organizations_delegated_administrator data resource that just exposes a full list of account?

It really seems like the intent of the aws_organizations_organization is to have it be used from the root account.

[cob16]

Given it has been a year now @rnikoopour and there are no dissenting opinions it does seem that a new resource is the way to go. 😄 I think It might just be worth seeing what we can get merged 🤷

[ravenium]

Just stumbled into this today when writing PermissionSet assignments. I can get around it by explicitly writing every last mapping but would be a nice shortcut to iterate upon.

[danowensdaylight]

You can use the aws_organizations_organizational_unit_descendant_accounts data source to list all the descendant accounts of the root account.

[gilbert-hsu]

You can use the aws_organizations_organizational_unit_descendant_accounts data source to list all the descendant accounts of the root account.

aws_organizations_organizational_unit_descendant_accounts need OU id, which chlid account can't get from aws_organizations_organization

Does any data source's ouput has OU id ?

[maunzCache]

You can use the aws_organizations_organizational_unit_descendant_accounts data source to list all the descendant accounts of the root account.

aws_organizations_organizational_unit_descendant_accounts need OU id, which chlid account can't get from aws_organizations_organization

Does any data source's ouput has OU id ?

Not as of 4.63.0 . I have at least not found any in the organizations "package". I tried them all today with terragrunt plan and none seemed to work as i get the following error:

ERROR: listing Organizations Account for Parent (xxxxxxxxxxxx): InvalidInputException: You provided a value that does not match the required pattern.
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "..."
  },
  Message_: "You provided a value that does not match required pattern.",
  Reason: "INVALID_PATTERN:PARENT_ID"
}

[maunzCache]

Given it has been a year now @rnikoopour and there are no dissenting opinions it does seem that a new resource is the way to go. 😄 I think It might just be worth seeing what we can get merged 🤷

Actually i don't think that a new resource is necessary. From what @cob16 wrote this is a bug in the way that the resource does not check if the account is a delegated administrator. So to make this resource backwards compatible adding the additional check would be sufficient.

Edit: Just noticed that all API calls that would help check if the account is delegated need the same permissions as the ListAccounts call. So it would be easier to just remove that if conditions as it yields no benefit. If the API call is not allowed then the error should be propagated by the provisioner. So i see the following options:

1. Make it backwards compatible: Remove the if condition but catch the AccessDenied error and return tolist(null). This may lead to other false-positive sideffects.
2. Make a breaking change: Only remove the if condition and propagate the error to the user.

Edit 2: After thinking about it even more i am pro "new resource". The current datasource combines the DescribeOrganization (and other Describe* calls) and ListAccounts API calls. Splitting this into two new resources one for each API call gives me a better gut feeling. It will eliminate the hazzle of breaking existing code while maintaining a cleaner provisioner. You need to explicitly call the ListAccount API in that way so you should know that the AWS account needs access to it.

Edit 3: Sorry for expanding the comment but this is my first time looking into this project. I just noticed that the current provider may violate the contribution guidelines as it does not fit to the API resource described at https://docs.aws.amazon.com/organizations/latest/APIReference/API_Organization.html . Even though the guideline does not mention that a resource has to be 1:1 a representation, it suggests naming data sources according to the data type. Thus an account(s) data source is valid.

[obeliskm]

for others caught out by not being able to enumerate a list of org accounts in a delegated admin account, I've ended up doing this to work around it:

in CI, before tf is run:

aws organizations list-accounts | jq 'walk(if type=="object" then with_entries(.key|=ascii_downcase) else . end)' > /tmp/aws-org-list-accounts
aws organizations list-roots    | jq 'walk(if type=="object" then with_entries(.key|=ascii_downcase) else . end)' > /tmp/aws-org-list-roots

then in tf

locals {
  account_list = jsondecode(file("/tmp/aws-org-list-accounts")).accounts
  parent_id     = jsondecode(file("/tmp/aws-org-list-roots")).roots[0].id
}

it's a bit of pain having to call out to the CLI, hopefully this will get resolved in some fashion soon.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.