codebuild: Arrays are not preserved between commands

[BastiQ]

Describe the bug

When using LinuxArmBuildImage with the docker image, public.ecr.aws/sam/build-python3.9:latest-arm64 and buildspec version 0.2, I run into the issue that arrays are not being preserved between commands.

According to the docs this should only happen with buildspec version 0.1 and be resolved in version 0.2.

The weird thing is, that other variables are preserved, and array variables can be looped over if the command is in the same line (see examples below).

This, btw. doesn't happen if I use LinuxBuildImage with public.ecr.aws/sam/build-python3.9:latest (no arm image).

I assume this is a bug, as arm support has just recently (v2.88.0, #26121, f522796) been added.

Expected Behavior

Current Behavior

Reproduction Steps

from aws_cdk import aws_codebuild as codebuild
# ...
example_codebuild_project = codebuild.Project(
    self,
    "EFSPackageInstallerProject",
    project_name="EFSPackageInstallerProject",
    description="Installs Python packages on EFS for use in model serving Lambda",
    build_spec=codebuild.BuildSpec.from_object(
        {
            "version": "0.2",
            "phases": {
                "build": {
                    "commands": [
                        'my_array=("first" "second" "third" "fourth")',
                        'for item in "${my_array[@]}"; do echo "$item" ; done',
                        # → empty outputs
                        'my_array=("first" "second" "third" "fourth"); for item in "${my_array[@]}"; do echo "$item" ; done',
                        # Outputs:
                        # first
                        # second
                        # third
                        # fourth

                        'myvar=hello; echo $myvar',
                        # outputs: hello
                        'myvar=hello',
                        'echo $myvar',
                        # outputs: hello
                    ]
                }
            },
        }
    ),
    environment=codebuild.BuildEnvironment(
        build_image=codebuild.LinuxArmBuildImage.from_docker_registry(
            f"public.ecr.aws/sam/build-python3.9:latest-arm64"
        ),
        compute_type=codebuild.ComputeType.SMALL,
        privileged=True,
    ),
)

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.90.0 (build 8c535e4)

Framework Version

No response

Node.js Version

v16.16.0

OS

MacOS

Language

Python

Language Version

3.9

Other information

No response

[peterwoodworth]

What makes you think this is an issue with CDK, can you identify a misconfigured option in the synthesized template?

[BastiQ]

Hey Peter, not sure if this is an issue with CDK. I thought it would make sense since ARM support was just recently added. Let me know where else I can report the issue. It takes some time to figure out that such a basic thing is not working, so I'm just trying to create awareness and potentially help others not to run into it. Feel free to close if the issue is not with CDK.

[peterwoodworth]

Yeah I don't see any explicit issues with the template. If you're able to test this same setup in the console to verify that it's not an issue with the service not fixing this particular case of the issue, that would be very helpful. Else, I can see what else I can do

[BastiQ]

Sorry, but I don't have the time for further investigation now. I also don't feel particularly excited about fixing AWS services for them. For anyone running into the same issue, just use multi-line commands. E.g.:

from aws_cdk import aws_codebuild as codebuild
# ...
example_codebuild_project = codebuild.Project(
    self,
    "EFSPackageInstallerProject",
    project_name="EFSPackageInstallerProject",
    description="Installs Python packages on EFS for use in model serving Lambda",
    build_spec=codebuild.BuildSpec.from_object(
        {
            "version": "0.2",
            "phases": {
                "build": {
                    "commands": [
                        """echo "Using an example array";
                        my_array=("first" "second" "third" "fourth");
                        for item in "${my_array[@]}"; do 
                            echo "$item";
                        done""",
                    ]
                }
            },
        }
    ),
    environment=codebuild.BuildEnvironment(
        build_image=codebuild.LinuxArmBuildImage.from_docker_registry(
            f"public.ecr.aws/sam/build-python3.9:latest-arm64"
        ),
        compute_type=codebuild.ComputeType.SMALL,
        privileged=True,
    ),
)

[peterwoodworth]

Yeah to put it simply - if it is using the correct image after you deploy, then this is almost certainly a service issue, which unfortunately I can't help much with. I submitted a ticket internally for this (P96948151), will provide an update when one becomes available

[peterwoodworth]

@BastiQ the service team let me know that arrays are not supported in buildspec command yet - From the team:

In order to workaround, try putting that in a script file and invoke that in buildspec, which should work.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[BastiQ]

Good to know, thanks for the update! Maybe something worth adding to the docs somewhere.

[peterwoodworth]

I'll forward this feedback onto them 🙂