ec2: cross-app usage of VPC

[pahud]

❓ General Issue

The Question

I am trying to build a sample to pass vpcId across stacks. It's fine if I pass vpcId as the stack property, however, the reality is, the vpc stack may be built by the infra team with native cloudformation templates and export the vpcId in the Outputs and the App team may build application with CDK in that VPC. All the App team knows is the export name and have to build an app stack in that VPC.

In TypeScript I can simply get the vpcId as String by cdk.Fn.importValue('ExportedVpcId')

import cdk = require('@aws-cdk/core');
import ec2 = require('@aws-cdk/aws-ec2');
import ecs = require('@aws-cdk/aws-ecs');
import ecsPatterns = require('@aws-cdk/aws-ecs-patterns');
import { ContainerImage } from '@aws-cdk/aws-ecs';
import { countResources } from '@aws-cdk/assert';
import { Vpc } from '@aws-cdk/aws-ec2';

export interface MyStackProps extends cdk.StackProps {
  vpc?: ec2.Vpc
  vpcId?: string
}

export class InfraStack extends cdk.Stack {
  readonly vpc: ec2.Vpc

  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    this.vpc = new ec2.Vpc(this, 'SharedVPC', {
      natGateways: 1,
    })

    new cdk.CfnOutput(this, 'vpcId', {
      value: this.vpc.vpcId,
      exportName: 'ExportedVpcId'
    })

  }
}

/**
 * This Fargate stack will be created within the ec2.Vpc we pass in from another stack
 */
export class FargateStack extends cdk.Stack {
  readonly vpc: ec2.Vpc

  constructor(scope: cdk.Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    this.vpc != props.vpc

    const cluster = new ecs.Cluster(this, 'Cluster', {
      vpc: this.vpc
    })

    const svc = new ecsPatterns.ApplicationLoadBalancedFargateService(this, 'FargateService', {
      cluster,
      image: ContainerImage.fromRegistry('nginx')
    })

    new cdk.CfnOutput(this, 'ServiceURL', {
      value: `http://${svc.loadBalancer.loadBalancerDnsName}/`
    })
  }
}

/**
 * This Fargate stack will be created within the vpcId we pass in from another stack
 */
export class FargateStack2 extends cdk.Stack {
  readonly vpc: string

  constructor(scope: cdk.Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    this.vpc != props.vpcId

    const cluster = new ecs.Cluster(this, 'Cluster', {
      vpc: Vpc.fromLookup(this, 'Vpc', {
        vpcId: this.vpc
      })
    })

    const svc = new ecsPatterns.ApplicationLoadBalancedFargateService(this, 'FargateService', {
      cluster,
      image: ContainerImage.fromRegistry('nginx')
    })

    new cdk.CfnOutput(this, 'ServiceURL', {
      value: `http://${svc.loadBalancer.loadBalancerDnsName}/`
    })
  }
}

And

/**
 * create our infra VPC
 */
const infra = new InfraStack(app, 'InfraStack', { env });

/**
 * create our Fargate service in the VPC from InfraStack
 */
const svc = new FargateStack(app, 'FargateServiceStack', {
    env,
    vpc: infra.vpc
})

/**
 * we can get the vpcId from the exported value from InfraStack
 */
const svc2 = new FargateStack2(app, 'FargateServiceStack2', {
    env,
    vpcId: cdk.Fn.importValue('ExportedVpcId')
})

However, in Python I got this error:

jsii.errors.JavaScriptError: 
  Error: All arguments to Vpc.fromLookup() must be concrete (no Tokens)

from aws_cdk import core, aws_ec2, aws_ecs, aws_ecs_patterns


class CdkPyCrossStackInfraStack(core.Stack):

    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        self.vpc = aws_ec2.Vpc(self, 'Vpc', nat_gateways=1)
        core.CfnOutput(self, 'vpcId', value=self.vpc.vpc_id, export_name='ExportedVpcId')


class CdkPyCrossStackFargateStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, vpc: aws_ec2.Vpc, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        self.vpc = vpc

        cluster = aws_ecs.Cluster(self, 'Cluster', vpc=self.vpc)
        svc = aws_ecs_patterns.ApplicationLoadBalancedFargateService(
            self, 'FargateService',
            cluster=cluster,
            image=aws_ecs.ContainerImage.from_registry('nginx'))

        core.CfnOutput(self, 'ServiceURL', value='http://{}/'.format(svc.load_balancer.load_balancer_full_name))


class CdkPyCrossStackFargateStack2(core.Stack):
    def __init__(self, scope: core.Construct, id: str, vpcId: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        cluster = aws_ecs.Cluster(self, 'Cluster', vpc=aws_ec2.Vpc.from_lookup(self, 'Vpc', vpc_id=vpcId))

        svc = aws_ecs_patterns.ApplicationLoadBalancedFargateService(
            self, 'FargateService',
            cluster=cluster,
            image=aws_ecs.ContainerImage.from_registry('nginx'))

        core.CfnOutput(self, 'ServiceURL', value='http://{}/'.format(svc.load_balancer.load_balancer_full_name))

infra_stack = CdkPyCrossStackInfraStack(app, "cdk-py-xstack-infra", env=AWS_ENV)
f1 = CdkPyCrossStackFargateStack(app, "cdk-py-xstack-fargate-svc1", env=AWS_ENV, vpc=infra_stack.vpc)
f2 = CdkPyCrossStackFargateStack2(app, "cdk-py-xstack-fargate-svc2",
                                  env=AWS_ENV,
                                  vpcId=core.Fn.import_value('ExportedVpcId')
                                  )


app.synth()

It looks like cdk.Fn.importValue() in TypeScript returns String but core.Fn.import_value() in Python returns Token.

Not sure if this is a valid issue. Any guidance is highly appreciated.

Environment

• CDK CLI Version: 1.8.0
• Module Version: 1.8.0
• OS: OSX Mojave
• Language: Python

Other information

https://gitter.im/awslabs/aws-cdk?at=5d7f7b8636461106bb29e96e

[ghost]

I think I'm having this issue, but in Typescript.

This code fails:

new elb.ApplicationLoadBalancer(this, "loadBalancer", {
  internetFacing: true,
  idleTimeout: cdk.Duration.seconds(30),
  vpc: ec2.Vpc.fromLookup(this, 'vpcId',  {
   vpcId: cdk.Fn.importValue("VPCID")
  }),
})

Returns:

"All arguments to Vpc.fromLookup() must be concrete (no Tokens)"

Where this code works:

new elb.ApplicationLoadBalancer(this, "loadBalancer", {
  // http2Enabled: false,
  internetFacing: true,
  idleTimeout: cdk.Duration.seconds(30),
  vpc: ec2.Vpc.fromLookup(this, 'vpcId',  {
   vpcId: 'vpc-99999999' 
 }),
})

This is CDK 1.12.0

cdk --version
1.12.0 (build 923055e)

[gaddam1987]

I also face same issue when I try to use Fn.importValue("vpcId") in java cdk api

[quappinger]

Same here on TypeScript with CDK 1.12.0
Are there any updates on this issue?

[EllSeni]

I am also experiencing this issue. Is there another way we are meant to pass the vpc id between stacks? I want to avoid having to add it in manually.

[RomainMuller]

That's a problem in all languages, as in it is not (currently) possible to pass deploy-time values across environments that aren't co-located (same account AND region).

[rix0rrr]

And this use case can never really work. The reason is that we need to know more about a VPC than its VPC ID: we also need to know all of the subnets, routing tablets, etc, because there are a lot of things people want to do to VPCs and many of the things require detailed knowledge about the VPC layout.

The only way to get this going right now is for you infrastructure team to deploy the VPC completely, then use fromLookup() to find it in every region you intend to deploy to. I recommend using tags to identify the VPC.

[jishi]

And this use case can never really work. The reason is that we need to know more about a VPC than its VPC ID: we also need to know all of the subnets, routing tablets, etc, because there are a lot of things people want to do to VPCs and many of the things require detailed knowledge about the VPC layout.

The only way to get this going right now is for you infrastructure team to deploy the VPC completely, then use fromLookup() to find it in every region you intend to deploy to. I recommend using tags to identify the VPC.

But how come it works if you hard-code the VPC-id, but doesn't work when importing it? It would still not know anything about the remaining configuration?

[mikepietruszka]

Is there any traction on this? As it stands it appears that exporting values from one stack to another simply does not work. I'm facing the same issue in Python.

[thomasgtaylor]

@maschinetheist As a work around, I used the boto3 library to pull in the export.

cf = boto3.client("cloudformation")
vpc_id = next(export['Value'] for export in cf.list_exports()['Exports'] if export['Name'] == 'export_name_here')

Hopefully though, this will be fixed.

[mikepietruszka]

@maschinetheist As a work around, I used the boto3 library to pull in the export.

cf = boto3.client("cloudformation")
vpc_id = next(export['Value'] for export in cf.list_exports()['Exports'] if export['Name'] == 'export_name_here')

Hopefully though, this will be fixed.

Thank you!

I was able to make some progress as well, but without using exports:

class VPCStack(core.Stack):
    def __init__(self, app: core.App, id: str, **kwargs):
        super().__init__(app, id, **kwargs)

        self.vpc = aws_ec2.Vpc(self, "VPC", nat_gateways=1) 
        

class ConsumingStack(core.Stack):
    def __init__(self, app: core.App, id: str, vpc: aws_ec2.Vpc, **kwargs) -> None:
        super().__init__(app, id, **kwargs)

        # Update VPC routes
        self.vpc = vpc
        private_subnets = self.vpc.private_subnets
        for subnet in private_subnets:
            # print(subnet.route_table.route_table_id)
            route_table_stack = ec2.CfnRoute(
                self, str("MainRouteTable" + random.randint(0, 254)),
                route_table_id=subnet.route_table.route_table_id,
                destination_cidr_block="10.0.0.0/16"
                transit_gateway_id="tgw-123456"
            )


vpcstack = VPCStack(app, "VPCStack")
consumingstack = ConsumingStack(app, "ConsumingStack", vpc=vpcstack.vpc) # Import VPC construct from vpcstack
cdk.synth()

Of course this will work within an app (across stacks) but not cross-app. For that I will try out boto3.

[sheridansmall]

Having the same issue reading DistributionID from CloudFront to pass to stacks using another region. So we are using aws_cloudformation.CustomResource which calls a lambda to read the values we need. The same principal should work for VPC-id but it is a bit like using a sledge hammer to crack a nut.

[jones-chris]

I encountered this issue when getting the VPC ID from a CfnMapping object that is defined in the same stack:

mapping = core.CfnMapping(
            self, 'Mapping',
            mapping={
                'dev': {
                    'vpc': 'vpc-123456'
                },
                'test': {
                     'vpc': 'vpc-789012'
                 },
                 'prod': {
                     'vpc': 'vpc-345678'
                 }
            }

vpc = aws_ec2.Vpc.from_lookup(
                    self, 'Vpc',
                    vpc_id=mapping.find_in_map(env, 'vpc')
)

And I get this error:

jsii.errors.JavaScriptError: 
  Error: All arguments to Vpc.fromLookup() must be concrete (no Tokens)

But, hard coding the vpc works fine:

vpc = aws_ec2.Vpc.from_lookup(
              self, 'Vpc',
              vpc_id='vpc-123456'
)

[joehillen]

@jones-chris That mapping will only show up when cdk generates the cloudformation. You should use an if-statement or switch-statement.

[michaelday008]

I'm facing the same issue in typescript in CDK 1.31.0.

FYI, in this case, the VPC to be looked up is in the same account and region as the stack being deployed, and I'm passing the correct credentials for this account and region to the synthesis.

Below fails.

const vpcExportName = 'VPC-devops';
const vpcID = cdk.Fn.importValue(vpcExportName);
const codebuildVPC = ec2.Vpc.fromLookup(this, 'VPC', {
        vpcId: vpcID
      });  

below succeeds

const vpcID = '1234556';
const codebuildVPC = ec2.Vpc.fromLookup(this, 'VPC', {
        vpcId: vpcID
      });  

What is the exact problem here? Is the issue that the cross stack reference is not looked up at synthesis time, but expected to be looked up only at deploy time?

Is there a way that something can be implemented so that these cross stack references could be looked up at synthsis time?

const vpcID = cdk.Fn.importValueImmediate(vpcExportName);
or
const vpcID = cdk.Fn.importValueForSynthesis(vpcExportName);

It's a big pain to go through all my CDK templates when a VPC ID is changed and copy and paste the text value of the VPCID into the template. I would rather just go cdk synth them and have them pull in that new value automatically by referring to the stack output.

[michaeltarleton]

Is there a way that something can be implemented so that these cross stack references could be looked up at synthsis time?

For now my solution is to hard-code the VPC Name and do the lookup by vpc name rather than VPC ID. It's a temp fix in my eyes but necessary for me to move on w/ my project.

const vpcName = 'Some Constant VPC name across environments'
const vpc = ec2.Vpc.fromLookup(this, 'VPC', {
    vpcName
})

[markymarkus]

I bumped to this issue when trying to automate VPC CIDR allocation with CustomResource. When passing the cidr value from CustomResouce to L2 - Vpc construct I'm getting 'property must be a concrete CIDR string, got a Token '. Passing the same to L1 CfnVPC works just fine. Btw, I'm using python.

[hoegertn]

As michaelday008 pointed out the underlying issue here is that there are several phases a CDK app goes through on its journey to deployed stacks.

• Building the object model of L? constructs down to the tree with only L1 leaves
• Synthesizing a CloudFormation template out of these L1s
• Deploying this JSON file using the CloudFormation engine

These steps happen strongly one after the other. So information gathered in a later step can never be used in an earlier phase.

Now let's look at the steps described in this ticket and in which phase they happen.

Object Model phase

• Computing CIDR ranges for subnets, VPC, etc
• Looking up VPCs by id or name

CFN Deployment

• Resolving Mapping entries from CFN mappings using !FindInMap
• Importing values from CFN exports using !ImportValue
• Getting attributes of custom resources using !GetAtt

So whenever a piece of information in generated during the later phase it will be a Token in CDK and can never be used in one the action of the first phase.

This is why:

• VPC Lookups can only work with concrete ids, names etc and not with imports or mappings
• CIDR range calculation cannot work with outputs of custom resources
• Any other lookup with CFN deploy time values

So how to solve this?

For lookups you could, as already mentioned, always import by name or any other tag instead of using an id. Alternatively, you can always do the lookups, imports, etc in a script before invoking CDK and provide the values as context.

I hope this helps with some of the confusion of why certain things work and do not work and why it is not easily possible to "fix" this or why this is not a bug but working as intended.

[hoegertn]

I am not sure I can follow. Is the VPC created using the CDK app or not?

If it is created in the same app you can use the instance cross stack. IF not you should be able to look it up as it already exists.

Can you elaborate?

[aw-huit]

So how to solve this?

For lookups you could, as already mentioned, always import by name or any other tag instead of using an id. Alternatively, you can always do the lookups, imports, etc in a script before invoking CDK and provide the values as context.

I hope this helps with some of the confusion of why certain things work and do not work and why it is not easily possible to "fix" this or why this is not a bug but working as intended.

I can't get my head around why you want to pass the ${Token[TOKEN.xx]} you get from cdk.fn.importValue instead of looking up the actual value and pass it onward.

I can understand for things that are more dynamic, like Fn::If, but stack exports are string literals.

[miaekim]

I was able to synth and deploy without hard-coding.

In separate CDK package, i have vpn.ts exporting its id to SSM.

export class VpcStack extends cdk.Stack {

    public readonly vpc: Vpc;

    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
        super(parent, name, {
            ...props
        });

        this.vpc = new Vpc(this, 'sharedVPC');
        cdk.Tags.of(this.vpc).add("Name", "sharedVPC");

        new ssm.StringParameter(this, 'sharedVpcId', {
            description: 'sharedVpcId for applications',
            parameterName: 'sharedVpcId',
            stringValue: this.vpc.vpcId,
        });
    }
}

Then using valueFromLookup() worked . Which is "Reads the value of an SSM parameter during synthesis through an environmental context provider."
fromStringParameterName() also have same error 'Error: All arguments to Vpc.fromLookup() must be concrete '

export class LambdaStack extends cdk.Stack {
    constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
        super(scope, id, props);

        const vpcId = ssm.StringParameter.valueFromLookup(this, 'sharedVpcId');
        const vpc = ec2.Vpc.fromLookup(this, 'existingVPC', { vpcId: vpcId });

        const lambdaFn = new lambda.Function(this, 'Singleton', {
            code: new lambda.InlineCode(fs.readFileSync('./lib/lambda-handler.py', { encoding: 'utf-8' })),
            handler: 'index.main',
            timeout: cdk.Duration.seconds(300),
            runtime: lambda.Runtime.PYTHON_3_6,
            vpc: vpc,
        });
    }
}

Another method is to use aws-sdk to fetch stack output.

We need to implement const vpcID = cdk.Fn.importValueForSynthesis(vpcExportName); like valueFromLookup() in SSM for CfnOutput as well .

[DonDebonair]

Is there any traction on this? As it stands it appears that exporting values from one stack to another simply does not work. I'm facing the same issue in Python.

I have discovered that this is not only the case for exporting (for example) the VPC id, but for many (maybe all?) id/arn-like properties that I try to export. I get the All arguments to ... must be concrete (no Tokens) with from...() lookups for:

• Load balancers
• Load balancer listeners
• VPCs

There's probably more!

This should really be fixed.

I do have a workaround for now: many resources can be looked up by adding tags to them. Example with VPC:

In one stack

self.prod_vpc = ec2.Vpc(self, 'ECSVPCProd', max_azs=2)
Tags.of(self.prod_vpc).add('source-ag:environment-type', 'prod')

In another stack:

vpc = ec2.Vpc.from_lookup(self, 'VpcLookup', tags={'source-ag:environment-type':'prod'})

Hope this helps for people having trouble with this!

[jishi]

I have stopped using CloudFormation export/imports altogether, and rely on parameter store instead like @miaekim suggested, and it works very well.

However, for legacy cloudformation stacks where you gradually migrate to CDK it becomes cumbersome, but I guess you can create SSM parameters via CloudFormation as well.

[DonDebonair]

So for people coming to this place for help, basically there are 2 options for mitigating this issue:

• Use tags instead of CFN exports
• Use SSM parameters instead of CFN exports

[intptr-t]

However, security groups, etc. still need to be hard coded.
Currently, SSM seems to be the only generic solution.

I think that SSM does not help solve the safety issue when deleting with stack dependencies in cross-applications.
I hope for a general solution that doesn't suffer from trade-offs.

[sebastian-schlecht]

I was also running into this and solved it via SSM. How come this is not implemented? Isn't this kind of a standard use-case for going micro-services like architectures where each service ships with infra?

[jans-fp]

As some described above I was able to circumvent the problem using the aws sdk. My solution looks like this

In the VPC stack

// export the vpc id as cfn output
new cdk.CfnOutput(this, 'my-vpc-id', {
	exportName: 'my-vpc-id',
	value: this.vpc.vpcId,
  })

In your other stack install aws-sdk

npm install aws-sdk --save

Get the output using the sdk, and pass it to your stack:

import * as CloudFormation from 'aws-sdk/clients/cloudformation'

const cf = new CloudFormation({ region: process.env.AWS_REGION })

const getExportValue = (expts: { "Exports": { "Name": string, "Value": string}[] }) => (name: string) =>
    expts.Exports.filter((expt) => expt.Name.localeCompare(name) == 0)[0].Value
	
const getConfig = () => cf.listExports().promise()

const app = new cdk.App()

getConfig().then((conf: any) => {
	const vpcId = getExportValue(conf)("my-vpc-id")
	console.log(`Using vpcId ${vpcId}`)
	return new MyStack(app, ',my-stack', { vpcId })
})

[Pelentan]

SOLUTION!!! Best of all it's simple and works pretty much as you would expect it. Took much digging and points out the issue with the documentation on something that is really still bleeding edge.

First: It looks like everything that accesses a cdk. call (efs, ec2, vpn, ext) occurs after the stack is uploaded to AWS. And this is after all the checks are done prior. Which means things like "cdk.CfnParameter" are just a Token as they haven't actually run yet. And in fact, all the code you put in your working file does not run sequentially like you would expect when using the cdk. functions. So while you may be declaring a variable based of an input parameter at the top, cdk. functions that are below it may run first.

Second: The Answer- -c or --context
Example: cdk deploy -c targetEnv=prod -c orgName='Vulcan3' -c jojo="was a frog"

And you retrieve it thusly:
const target_env: string = this.node.tryGetContext('targetEnv'); (app.node for Python) (I can't be the only one who keeps typing it 'Pythong')

You can even take it a step further and be more GitOpsy about it. (Is too a word. I'm an author. I get to make up words.) On the main branch of your project (assuming you used cdk init) you'll have a file called cdk.json. Primarily used for the cdk to pass access info for your AWS account as well as other things. However, you can add to it. Everything "prod" down I added. Note that all additions are in the "context" json.

{
  "app": "npx ts-node --prefer-ts-exts bin/leaf-ancil.ts",
  "context": {
    "@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
    "@aws-cdk/core:enableStackNameDuplicates": "true",
    "aws-cdk:enableDiffNoFail": "true",
    "@aws-cdk/core:stackRelativeExports": "true",
    "@aws-cdk/aws-ecr-assets:dockerIgnoreSupport": true,
    "@aws-cdk/aws-secretsmanager:parseOwnedSecretName": true,
    "@aws-cdk/aws-kms:defaultKeyPolicies": true,
    "@aws-cdk/aws-s3:grantWriteWithoutAcl": true,
    "@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount": true,
    "@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
    "@aws-cdk/aws-efs:defaultEncryptionAtRest": true,
    "@aws-cdk/aws-lambda:recognizeVersionProps": true,
    "prod": {
      "vpc_name": "leaf-prod",
      "efs_name": "leaf-prod-legacy-efs"
    },
    "dev": {
      "vpc_name": "leaf-dev",
      "efs_name": "leaf-dev-legacy-efs"
    }
  }
}

So first you get the command line value:
const target_env: string = this.node.tryGetContext('targetEnv');

Then you get the json branch you want to use (nicely stored in git and versioned):
const build_vars = this.node.tryGetContext(target_env);

And from there you used it exactly as you would expect:
const vpc_name: string = build_vars?.vpc_name ??leaf-dev;

Hope this helps a lot of people.

[rdewaele]

I would like to point out that the inability of the CDK to import VPC IDs from another CF stack export also means that CF will not be aware of this dependency. For example, modification of the export name will be allowed. This will break the CDK stack, which will only be detected the next time the stack is synthesized. (Could not find export with name xyz.)

This is not the case with plain old cloudformation files, as the Fn::ImportValue statement makes CF aware of this dependency, which will prohibit such modifications.

[makemefeelgr8]

How come there's no way to import a VPC by id into a different stack? The issue is 3 years old!
This is the default approach! Create 1 stack with a VPC. Create multiple apps/stacks that use the said VPC.
How come this workflow is not supported?
How am I supposed to deploy my microservices?
Do you really recommend to hardcode the VPC id?
Who implemented the import function that supports only hardcoded ids?
Did anyone review that piece of code before merging?
Did you guys even try to use your own product? Is the CDK production ready?
Why is random stuff like this #19552 prioritized over the very core essential feature?!
It's not even ridiculous, it's just sad.

[github-actions]

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.

[abury]

Just chiming in to say that CDK has been amazing to use.... until I tried to create (or rather cross reference) VPCs. Given how seemless and enjoyable the rest of the process has been, slamming into the brick wall that is VPC cross referencing has been a nightmare.

I ran into a similar issue with Dynamo tables, but that's a fairly easy work around because you can reference them by ID. I followed the recommended steps listed here and just made it cross reference, but then as soon as I did something as simple as change the subnet a lambda in a different stack used, I ran into the dreaded cannot delete export error, which was worsened by the this.exportValue work around not working, I assume due to the subnets being created auomatically within the VPC.

I'm now having to delete half of my stacks just to break a single reference one of them has to a single subnet, which seems frustrating (luckily CDK makes it easy to spin them right back up again! 😂)

Definitely learned my lesson reg. cross reference, will be storing the values in SSM, which seems to be the solution and isn't that painful.

After using CDK for a few months, the cross reference issue has definitely taken up most of my time, and resulted in the most swear words per minute.

[Craig1f]

FYI: I solved this by adding aws-sdk to my cdk, and then using it to pull CFN exports:

export async function getCfnExports(region: string) {

  const cloudformation = new CloudFormation({ region });

  const exports = (await cloudformation.listExports().promise()).Exports?.reduce(
    // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
    (prev, next) => ({ ...prev, [next.Name!]: next.Value }),
    {},
  ) as { [key: string]: string };

  const CFN_OUTPUTS = {
    // Infrastructure
    vpcId: exports[config.get('cfn_exports').vpcId],
    ...
  };

  return CFN_OUTPUTS;
}

[shiva2021]

FYI: I solved this by adding aws-sdk to my cdk, and then using it to pull CFN exports:

export async function getCfnExports(region: string) {

  const cloudformation = new CloudFormation({ region });

  const exports = (await cloudformation.listExports().promise()).Exports?.reduce(
    // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
    (prev, next) => ({ ...prev, [next.Name!]: next.Value }),
    {},
  ) as { [key: string]: string };

  const CFN_OUTPUTS = {
    // Infrastructure
    vpcId: exports[config.get('cfn_exports').vpcId],
    ...
  };

  return CFN_OUTPUTS;
}

Great workaround. I hope AWS comes up with a CDK oriented solution soon.

[MrArnoldPalmer]

So if I'm following all of the discussion I believe that there are two issues here, one being weak references, which is a feature that is being discussed and designed but hasn't been implemented that allows referencing values cross stack in a way that avoids the dreaded "can't delete stack because export value is used by x" loop and would replace the need for storing vpcId in an SSM parameter.

To sum up some discussion for anyone stumbling on this issue now and looking for solutions.

1. fromLookup will never work on the result of Fn.importValue because fromLookup makes SDK calls to your account/region to describeVpcs and then filters based on the arguments provided until it finds the right one. Then it does describeSubnets, describeRouteTables, etc in order to get all of the properties of the VPC that are present on the IVpc interface. Fn.importValue is an intrinsic cloudformation function so it gets rendered in the CFN template as a string that clouformation will resolve to an actual useful value during deploy, but is not useful to fromLookup which is called during synthesis. If you need/want to use fromLookup and you don't have access to the VpcId literal, you can use the aws-sdk to get it as mentioned in some of the comments above and then pass it as an argument to your stack, or put the value into an SSM parameter and use valueFromLookup. All of these resolve the value before synthesis so that Vpc.fromLookup can do all of it's API calls.

2. You can alternatively use Vpc.fromVpcAttributes which instead of doing a bunch of API calls to get information about the VPC, just renders a bunch of references in your cloudformation template. It requires that you know a lot about the VPC that you're deploying into if any of the constructs you're using need to reference it's availabilityZones or subnets etc. However, this makes sense for some use cases where you can't do lookups at synthesis time, but essentially has all of the same limitations as if you were writing a template by hand that referenced a VPC in another stack. If you needed to reference a subnet by ID, that stack would have to add the subnet IDs to the exports and you would have to know the export name for each of these values etc.

That being said, I believe what this issue is asking for is essentially weak references, so I'm going to convert it to a discussion in case anyone has more questions.