Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude hidden files by default #598

Merged
merged 9 commits into from
Aug 29, 2024
Merged

Conversation

joshmgross
Copy link
Member

@joshmgross joshmgross commented Aug 15, 2024

Currently, all files within the search path are uploaded into the artifact. This includes hidden files, which could contain sensitive values that should not be accessible outside of the workflow run. To enhance the default security of this action, this PR excludes hidden files from the path by default. Users who require hidden files have validated that there are no sensitive values in their artifacts can opt-in to uploading hidden files with the include-hidden-files input.

This is part of the following breaking change:

Exclude hidden files by default in Upload Artifact GitHub Actions
From September 2nd, 2024, we will no longer include hidden files and folders as part of the default upload of the v3 and v4 upload-artifact actions. This reduces the risk that credentials are accidentally uploaded into artifacts. Customers who need to continue to upload these files can use a new option, ‘include-hidden-files’, to continue to do so.
https://github.blog/changelog/2024-08-19-notice-of-upcoming-deprecations-and-breaking-changes-in-github-actions-runners/

src/shared/search.ts Outdated Show resolved Hide resolved
@joshmgross joshmgross marked this pull request as ready for review August 15, 2024 21:39
@joshmgross joshmgross requested a review from a team as a code owner August 15, 2024 21:39
konradpabjan
konradpabjan previously approved these changes Aug 15, 2024
action.yml Outdated Show resolved Hide resolved
merge/action.yml Show resolved Hide resolved
README.md Show resolved Hide resolved
@@ -9104,6 +9121,10 @@ class DefaultGlobber {
if (!stats) {
continue;
}
// Hidden file or directory?
if (options.excludeHiddenFiles && path.basename(item.path).match(/^\./)) {
Copy link

@casperdcl casperdcl Aug 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about a few common things like gh-pages' .nojekyll and even .gitignore?

Suggested change
if (options.excludeHiddenFiles && path.basename(item.path).match(/^\./)) {
if (options.excludeHiddenFiles && path.basename(item.path).match(/^\.(?!nojekyll$)/)) {

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the feedback, for now we are going to go ahead with the 'all in' option as starting to pick and choose common things is a risk we start to go back to 'too many' common things and expose folks to the risk of mistakes causing issues.
I get the idea though, maybe in the future we could look at 'more flags' for "all-common-none", but that will be something we need to revisit at a later date so we could get a better view on what common would be :)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well at least there should probably be a black/whitelist option rather than a bool methinks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

path is effectively that allowlist/denylist option - you can allow hidden files and then specify in path which to exclude

https://github.com/actions/upload-artifact#upload-using-multiple-paths-and-exclusions

@@ -9104,6 +9121,10 @@ class DefaultGlobber {
if (!stats) {
continue;
}
// Hidden file or directory?
if (options.excludeHiddenFiles && path.basename(item.path).match(/^\./)) {
Copy link

@casperdcl casperdcl Aug 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Suggested change
if (options.excludeHiddenFiles && path.basename(item.path).match(/^\./)) {
if (options.excludeHiddenFiles && path.basename(item.path).match(/^\.(?!nojekyll$)/)) {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joshmgross joshmgross merged commit 5076954 into main Aug 29, 2024
9 checks passed
@bartekpacia
Copy link

This just broke my workflow. I think this warrants a breaking change.

@AlexSkrypnyk
Copy link

AlexSkrypnyk commented Sep 2, 2024

This has just broke 1000s of workflows that rely on artifacts produced by tools like pcov/phpunit etc. and any other reporting tools.

This should have been released as a major version.

@neoye911
Copy link

neoye911 commented Sep 3, 2024

This has just broke 1000s of workflows that rely on artifacts produced by tools like pcov/phpunit etc. and any other reporting tools.

This should have been released as a major version.

Failed our release pipeline as well. Agree.

@thomas-alias-zaiko
Copy link

Hey guys, even though this is definitely a good change, introducing a breaking change on a minor version isn't great.
Like the above comments, this also affected our company's workflows (including our production ones).
Please consider breaking changes on major versions next time 🙂

@casperdcl
Copy link

It's also likely extremely hard to debug for users. "Breaking" was included in #602's title so it's unforgivable to sneak this in a minor version.

@twm
Copy link

twm commented Sep 3, 2024

This change likely breaks every workflow that uploads the .coverage* files generated by coverage.py. This will have a significant impact in the Python ecosystem.

Did GitHub assess the number of (public) artifact uploads that would be affected by this change? I understand that avoiding upload of .git/config files containing credentials was the goal. Is forbidding the upload of any pathname beginning with . the lowest-impact way to achieve that aim?

Some other possibilities:

  1. Don't upload .git directories by default.
  2. Don't put credentials in .git/config by default.

The first option seems attractive to me, regardless of security implication, since the .git directory is redundant with the content of the Git repository. I struggle to envision a scenario where I would deliberately upload the .git directory.

The second option minimizes harm.

I must express my sympathy with all those who advocated for less impactful options, but did not prevail. Sadly, the OSS ecosystem must deal with the consequences.

@casperdcl
Copy link

casperdcl commented Sep 3, 2024

I'd still suggest reverting in a minor version and re-releasing in a major one regardless of implementation (#598, #599, #598 (comment), etc)

nkraetzschmar added a commit to gardenlinux/package-build that referenced this pull request Sep 3, 2024
@joshmgross joshmgross deleted the joshmgross/exclude-hidden-files branch September 3, 2024 15:21
@joshmgross
Copy link
Member Author

Please direct all feedback to the tracking issue #602

@actions actions locked as off-topic and limited conversation to collaborators Sep 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.