Feature request: create wrapper to expose stdout and stderr as outputs

[patrickmoore-nc]

I have terraform-compliance running successfully in a GitHub Actions validation pipeline with a working feature. However, I would like to get the results of the execution added to Pull Requests. I have tried to do this in the style of the example at the bottom of this page:
https://github.com/hashicorp/setup-terraform. It works very nicely for the Terraform output itself, appearing as shown below with the outputs expanding on-click. However I cannot capture the output from terraform-compliance.

Terraform Initialization ⚙️ success
Terraform Validation 🤖 success
Terraform Plan 📖 success
Show Plan
Terraform Compliance 👺 failure
Show Output

After some experimentation I now understand that the setup-terraform GitHub action captures and exposes these outputs this using a Node.js wrapper, which wasn't obvious. I had naïvely expected that GitHub Actions would provide these stdout and stderr outputs as standard. I have tried various ways to achieve a similar result for terraform-compliance myself, without success.

I tried a generic capture of ${{ steps.tfcompliance.outputs.response }}, which ends up null.

I tried the ::set-output command:

name: Validation of plan by terraform-compliance
id: tfcompliance
run: |
  echo ::set-output name=stdout::$(terraform-compliance -p ../../terraform/output.plan.json -f . --no-ansi)

I then try to retrieve this output by referencing ${{ steps.tfcompliance.outputs.stdout }} but the above example results in the first line of output only, and it always finishes with successful status - because it's measuring the echo result, not the result of the terraform-compliance command. Furthermore I lose local echo of the results in the workflow log which isn't desirable.

I have tried to use this apparently confirmed method from Stack Overflow:

name: Validation of plan by terraform-compliance
id: tfcompliance
run: |
  terraform-compliance -p ../../terraform/output.plan.json -f . --no-ansi | tee test.log
  result_code=${PIPESTATUS[0]}
  echo "::set-output name=stdout::$(cat test.log)"
  exit $result_code

which results again in a null ${{ steps.tfcompliance.outputs.stdout }} - it's like the tee is completely ignored.

I then discovered that according to this GitHub Community Forum post, set-output cannot in fact deal with multiline strings, which seems pretty deficient. However, it transpires that you can substitute the line breaks (effectively turning it back into a single line string) in a way that will be automatically converted back when an output is interpolated later.
So I tried:

name: Validation of plan by terraform-compliance
id: tfcompliance
run: |
  terraform-compliance -p ../../terraform/output.plan.json -f . --no-ansi > output.log
  OUTPUT="$(cat output.log)"
  OUTPUT="${OUTPUT//'%'/'%25'}"
  OUTPUT="${OUTPUT//$'\n'/'%0A'}"
  OUTPUT="${OUTPUT//$'\r'/'%0D'}"
  echo "::set-output name=stdout::${OUTPUT}"

This didn't work at all either - still a completely blank ${{ steps.tfcompliance.outputs.stdout }}

Reading the Radish docs section describing console output, it does appear that the console output is rendered in a manner that possibly complicates this (with overprinting of previously displayed text during scenario executions etc.). I tried to use the --write-steps-once option but it's not supported by terraform-compliance despite a mention in the documentation that options should be passed through to radish. EDIT - Ahh, I see that particular radish option is enabled by default by terraform-compliance.

Considering all this, would you perhaps consider adding a JavaScript wrapper to capture and expose regular GitHub Actions outputs for stderr, stdout, and exitcode in the same way that setup-terraform does it? I think this is something that a lot of people would want, and it currently seems unnecessarily difficult to achieve independently.

In the meantime - am I missing something obvious to capture this output?

Thanks

[patrickmoore-nc]

I came back to save someone else the horrendous amount of tinkering it took to get this working - i.e. passing the entire terraform-compliance output, special characters and all, as an output and using it in a github-script step later in the job. I find it mind-blowing that GitHub made this task as difficult as it is, given how likely it is that someone might need it. They in effect made a ::set-output command that you have to develop your own wrapper for. It's half-baked. And why not just expose stdout and stderr for all job steps by default?

Beyond the line break issue which has been covered in plenty of other examples online, the trick is that you need to ingest the output data using the Node.js String.raw() method in the github-script step, to prevent it from interpreting all the special characters. However, Radish .feature files (and therefore terraform-compliance output) are likely to contain regex content which may include backticks, which will cause particular pain because Node.js uses backticks to delimit template literals (raw strings). So while we are still in bash in the original job step we need to first escape any backticks, and also strings ($) which Node.js will try to interpolate, even inside a template literal. Only then can we safely use the ::set-output command.

Finally in the github-script step, once safely ingested as a raw string, we can then remove the escaping of backticks and $ with a few simple .replace() method calls.

      # Generate a Terraform plan, continue even if failed
      - name: Terraform Plan
        id: plan
        working-directory: terraform
        continue-on-error: true
        run: |
          terraform plan -no-color -input=false -out output.plan && terraform show -json output.plan > output.plan.json

      # Run terraform-compliance on plan, continue even if failed
      - name: Validation of plan by terraform-compliance
        id: tfcomp
        working-directory: tests/terraform-compliance
        continue-on-error: true
        run: |
          terraform-compliance -p ../../terraform/output.plan.json -f . --no-ansi | tee tfcomp.log
          result_code=${PIPESTATUS[0]}
          LOG="$(cat tfcomp.log)"
          # GitHub Actions does not support multi-line strings as outputs, so line breaks must be encoded
          # they will automatically be substituted back in during GitHub Actions interpolation later, before script execution
          # first, encode % so that instances of it aren't mistaken for encodings later
          LOG="${LOG//'%'/%25}"
          LOG="${LOG//$'\n'/%0A}"
          LOG="${LOG//$'\r'/%0D}"
          # escape any backticks, which delimit template literals in Node.js (raw strings)
          # and escape '$' which will also trip up the Node.js interpolation in the github-script step
          LOG="${LOG//\`/\\\`}"
          LOG="${LOG//\$/\\\$}"
          # https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter
          echo "::set-output name=response::$LOG"
          # exit with the pertinent error code only after we have exposed the output for later steps 
          exit $result_code

      # Update the pull request with the Terraform plan and terraform-compliance output
      - name: Update Pull Request
        uses: actions/github-script@v6
        with:
          script: |
            // interpret tfcomp output as a template literal (raw string)
            // GitHub Actions will reinsert the line breaks and % during its interpolation before script execution
            var tfcomp = String.raw`${{ steps.tfcomp.outputs.response }}`;
            // put back the characters which were escaped by bash, now we have correctly ingested the raw output
            tfcomp = tfcomp.replace(/\\\$/g, '$')
            tfcomp = tfcomp.replace(/\\\`/g, '`')
            const output = `#### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
            #### Terraform Validation 🤖\`${{ steps.validate.outcome }}\`
            #### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
            <details><summary>Show Plan</summary>
            
            \`\`\`
            ${{ steps.plan.outputs.stdout }}
            \`\`\`
            
            </details>
            
            #### Terraform Compliance 👺\`${{ steps.tfcomp.outcome }}\`
            <details><summary>Show Tests</summary>
            
            \`\`\`
            ${tfcomp}
            \`\`\`
            
            </details>
            
            *Pushed by: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;
            
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })

      # Check Terraform Plan status and exit on failure
      - name: Check Terraform Plan Status
        if: steps.plan.outcome == 'failure'
        run: exit 1

      # Check terraform-compliance status and exit on failure
      - name: Check terraform-compliance Status
        if: steps.tfcomp.outcome == 'failure'
        run: exit 1

[FooBartn]

@patrickmoore-nc This comment is not a productive one in the context of your feature request. I just wanted to thank you for saving me more time than I've already spent figuring out how to do this. You deserve at least a few ☕ or 🍻 for returning back the results of your hard work. Cheers!

[andrew-hm]

@patrickmoore-nc thank you for great research on this solution. I was hoping to find a better solution that was supported by GH but after reading your comment was convinced it didnt exist. Also thanks for hat tip to String.raw. Works great and easily repeatable pattern.