Upcoming breaking change: Hidden Files will be excluded by default

[joshmgross]

From September 2nd, 2024, we will no longer include hidden files and folders as part of the default upload of the v3 and v4 upload-artifact actions. This reduces the risk that credentials are accidentally uploaded into artifacts. Customers who need to continue to upload these files can use a new option, ‘include-hidden-files’, to continue to do so.

https://github.blog/changelog/2024-08-19-notice-of-upcoming-deprecations-and-breaking-changes-in-github-actions-runners/

• Exclude hidden files by default #598
• V3 backport: Exclude hidden files by default #604

[dolfinus]

Why introducing breaking changes to v3 and v4 instead of making a new v5 version?

[uerkut]

      - name: Upload Docker Compose Files Artifact
        id: upload_docker_compose_files
        uses: actions/upload-artifact@v4.4.0
        include-hidden-files: true
        with:
          name: docker-compose-files
          path: |
            .env
            docker-compose.yaml

I've edited my workflow like above and getting the error:

The workflow is not valid. In .github/workflows/kondukto-package-build-manual.yml (Line: 318, Col: 11): Error from called workflow kondukto-io/kondukto-build/.github/workflows/docker_compose_package.yml@c343cea5db32d5d329cfe25ea5f7f84a88ecb64a (Line: 38, Col: 9): Unexpected value 'include-hidden-files'

I tried both versions v4 and v4.4.0

[dolfinus]

I've edited my workflow like above and getting the error:

You should place include-hidden-files: true under with: block

[uerkut]

Thanks a lot. My bad.

[catYalere]

Funny thing [in](include-hidden-files: true) doesn't work on v3, in one screenshoot I point directly to file and also doing a cat to see if it exists and exists

[buckett]

This change resulted in deployment failures for us because we no longer included the full set of files in our deployment artifact.

[Drowze]

This also resulted in a lot of CI failures for us... Would be nice if there was at least a warning before rolling breaking changes 🤷‍♂️

Here's how we were calling this action:

    - uses: actions/upload-artifact@v4
      with:
        name: artifacts-${{ strategy.job-index }}
        path: |-
          coverage/.resultset.json
          tests/junit.xml

[joshmgross]

Funny thing [in](include-hidden-files: true) doesn't work on v3, in one screenshoot I point directly to file and also doing a cat to see if it exists and exists

👋 Thanks for the report, we're rolling back the v3 and v3-node20 updates while we work on fixing that issue.

[gurvancampion]

Same here, I suddenly got an issue in my CI while deploying my app that never occurred before:

[tpope]

We also got burned by this. I see the logic in excluding hidden files recursively nested inside directories. But I struggle to see the value in ignoring an explicit path: .dotfile parameter.

[joshmgross]

Funny thing [in](include-hidden-files: true) doesn't work on v3, in one screenshoot I point directly to file and also doing a cat to see if it exists and exists

👋 Thanks for the report, we're rolling back the v3 and v3-node20 updates while we work on fixing that issue.

This is fixed in #608 & #609

https://github.com/actions/upload-artifact/releases/tag/v3.2.1
https://github.com/actions/upload-artifact/releases/tag/v3.2.1-node20

We're updating the major version tags v3 & v3-node20 to include those fixes.

[catYalere]

TY v3 with flag working

[antoniocoratelli]

Breaking changes should bump major version! What kind of semantic versioning is this?

And if old behavior poses security problems, issue a deprecation warning via EMAIL to organization owners that have repos using this very important action.

Breaking developers workflows on such a short notice (if writing a blog post can be considered an actual "notice") is NOT OK

[jchli]

This should've been part of a major version release. Introducing this breaking change to everyone on the loose reasoning of "because security" is insufficient and potentially irresponsible.

Consider the human behavior: do you think that folks that got blocked up on this are going to truly fix up their credentials or do you think they're more likely to just set include-hidden-files and be done with it?

[jni]

defaulting everyone to a more secure state/default

What exactly is the security analysis here? If the version had been bumped to v5, any new users would get the secure default. Over time, everyone would end up on it. By breaking everyone's CI all you achieve is that everyone adds include-hidden-files: true to their config to get their CI working again. (Have a quick browse of the incoming links... 🤦) And they will probably carry that config setting forward with any version updates.

[suryajak]

Catching up on this and I am still not sure how is this OK to have a breaking change while following semantic version. Our team has been spending time debugging pipeline failures and this thread that the impact is huge.

As a followup, can we agree on common terms (semantic version) and trust that our dependent systems honor that?
At the least, please revert this.

[dakinggg]

Even if we accept that occasionally making a breaking change for security purposes is acceptable (which I am on board with), I think this change could have been handled with significantly more care.
(1) Directly specified . files (unless I'm missing something) are not a security risk and could have been allowed to avoid some of this breakage.
(2) The direct cause of the exploit could have been addressed immediately by explicitly excluding the known secret types discovered in the exploit (e.g. .git folders), and then taking more time to roll out the include hidden files default change with proper deprecation.
(3) The "no files found" warning could have been updated to include a link to the changelog and something like "if your workflow just broke unexpectedly, look here).

As others have pointed out, the way this was rolled out more or less forced a bunch of people to add include-hidden-files: true in their workflows, which defeats the whole point of the more secure default.

[supertr0n]

The combined wasted person-hours and other indirect costs caused by this very unexpected breaking change must be quite large.

It's worth considering how many of GitHub's customers' own urgent security patches/updates for their own software have been held up from being deployed because of this random breakage of their CI pipeline.

[emmahsax]

Why introducing breaking changes to v3 and v4 instead of making a new v5 version?

To echo this sentiment, there should not have been a breaking change without a new major version. This change literally caused downtime on our production system and took us hours to debug (we had rolled back production by then of course).

Even if there were security concerns, the default should've been true in v3 and v4, false in v5, and then new v3 and v4 minor or patch updates which could write an annotation to GitHub to say that due to security concerns, the team recommends users add include-hidden-files: false in their workflows.

This solution wouldn't have broken anything for anybody, while also notifying them more to the fact that there's recommendations.

Also, totally icky and rude to be releasing this on a day when many people are not working due to a national holiday. It's almost like they wanted to just slide it in when people may not be paying attention..... 🙄

[Pixelatex]

@nebuk89 can we send you the invoice for downtime and time spent because this didn't break the pipeline but instead deployed applications without needed files to run..

Acknowledge your mistake and don't hide behind the shield that you somehow saved the day by shielding us from the big bad evil hidden files upload... Seriously, go touch grass.. this is not some zero-day you're patching, you do not ignore semver and push a breaking change for such a stupid reason.

[smokedlinq]

Why not publish a v5 version for this? This is breaking a ton of our things that build files with .folders such as Azure Functions.

[josheche]

We still use v3 of upload-artifacts. My scheduled pipelines broke last night silently because it was looking for our .env files but needed include-hidden-files to be set to true instead of the default false that was supposed to be a breaking change for v4.