Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for .gitignore #1908

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jbedard
Copy link
Contributor

@jbedard jbedard commented Sep 4, 2024

What type of PR is this?

Feature

What package or component does this PR mostly affect?

all

What does this PR do? Why is it needed?

Allow users to opt-in to respecting the gitignore when collecting files for gazelle extensions to consume.

While bazel can access git-ignored files many repositories manually exclude git-ignored content by not including it in source file lists or globs in BUILDs or by duplicating the git-ignore entries as # gazelle:exclude.

Ideally users would put all bazel ignored content in .bazelignore but without glob support that is often not done.

This should remain opt-in for repositories wanting it but by default bazel can access git-ignored content so gazelle should as well.

Which issues(s) does this PR fix?

Fixes #1166

Other notes for review

@jbedard
Copy link
Contributor Author

jbedard commented Sep 5, 2024

@fmeum can you confirm if adding this to the gazelle core is something we'd like to do? If so are there preferences for which library to use vs manual?

@tyler-french
Copy link
Contributor

How is this different from using 'gazelle:exclude' directives?

@jbedard
Copy link
Contributor Author

jbedard commented Sep 5, 2024

People won't want to copy+paste gitignore files into BUILD files...?

@jbedard
Copy link
Contributor Author

jbedard commented Sep 5, 2024

Functionally gitignore also works a bit different with things like ! operators etc. That is one reason to use a library such as go-git instead of doing it ourselves but we could also potentially do it ourselves but don't support !...

@fmeum
Copy link
Collaborator

fmeum commented Sep 5, 2024

I kind of like this as it reduces the amount of Gazelle-specific configuration.

I don't know which library is best, but we should definitely use one rather than roll our own.

@jbedard
Copy link
Contributor Author

jbedard commented Sep 5, 2024

I don't know which library is best, but we should definitely use one rather than roll our own.

I agree we should use a real library and not try to re-implement gitignore logic, otherwise we'll just end up with a never-ending stream "it doesn't work like git" bugs.

The go-git library is what I've used in all the aspect gazelle extensions and I've never had any issues so I think that's a good place to start. If we add appropriate tests then we should be able to change the implementation later.

@jbedard
Copy link
Contributor Author

jbedard commented Sep 5, 2024

FWIW I've also tried other libraries which all had issues in the aspect gazelle extensions:

Those both also seem to be unmaintained.

Others I have not tried seem to also be unmaintained and many look too simple or slow:

walk/config.go Outdated Show resolved Hide resolved
walk/config.go Outdated Show resolved Hide resolved
walk/config.go Outdated Show resolved Hide resolved
name = "rename_%s_%s" % (TESTS[i], j),
srcs = [TEST_GEN_FILES_IN[i][j]],
outs = [TEST_GEN_FILES_OUT[i][j]],
cmd = "cp $< $@",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something here is not working on windows where this genrule is either not being generated or is not dropping the .test- from the path, I assume because of slashes.

Anyone see anything wrong here? Or have a windows box to debug it more?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave up and ignored this test on windows for now. I confirmed it is renaming the files but the .test-* files are still visible. I assume this is a windows+sandboxing issue?

walk/walk.go Outdated Show resolved Hide resolved
walk/walk.go Outdated Show resolved Hide resolved
walk/walk.go Outdated Show resolved Hide resolved
walk/config.go Outdated Show resolved Hide resolved
walk/config.go Outdated Show resolved Hide resolved
}
}

shouldUpdate := updateRels.shouldUpdate(rel, updateParent)
for _, sub := range subdirs {
if subRel := path.Join(rel, sub); updateRels.shouldVisit(subRel, shouldUpdate) {
visit(c, cexts, isBazelIgnored, knownDirectives, updateRels, wf, filepath.Join(dir, sub), subRel, shouldUpdate)
// Create a new slice to allow the nested 'visit' call to modify the slice without affecting the parent.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we expect the existing elements to get mutated? Or only append new ones? Asking because if relParts has capacity for an extra element, then subParts will be a new slice but they will share the same backing store, so mutating it's 0..n-1 index elements will also mutate relParts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only mutation is of entParts which is local to this method, however entParts = append(relParts, ".").

We might have to think about this more when the loop becomes concurrent though.

@jbedard jbedard force-pushed the gitignore branch 3 times, most recently from ed09b76 to 0f82486 Compare September 16, 2024 07:58
fmeum pushed a commit that referenced this pull request Sep 16, 2024
This reduces the noise a lot when adding more libraries such as in #1908

**What type of PR is this?**

> Other

**What package or component does this PR mostly affect?**

> all

**What does this PR do? Why is it needed?**

**Which issues(s) does this PR fix?**

Fixes #

**Other notes for review**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exclude files/folders that are .gitignore'd
5 participants