Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for pipeline root.when conditions #770

Merged
merged 86 commits into from
Sep 26, 2022
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
c49d487
Auto commit @ 2022-02-09T18:01:36Z
Feb 9, 2022
473c224
Auto commit @ 2022-02-09T18:38:21Z
Feb 9, 2022
65fb8d3
Auto commit @ 2022-02-09T18:48:45Z
Feb 9, 2022
60b5f77
Auto commit @ 2022-02-09T18:50:21Z
Feb 9, 2022
158232e
Auto commit @ 2022-02-09T18:50:35Z
Feb 9, 2022
52cf142
Auto commit @ 2022-02-09T19:36:03Z
Feb 9, 2022
b03ad04
Auto commit @ 2022-02-09T19:53:56Z
Feb 9, 2022
a295ea3
Update server/shared/procBuilder.go
LamaAni Feb 9, 2022
261e791
Update server/api/hook.go
LamaAni Feb 9, 2022
aa41a26
Update server/api/hook.go
LamaAni Feb 9, 2022
54d33e1
Update pipeline/frontend/yaml/compiler/compiler.go
LamaAni Feb 9, 2022
00f82bb
Update server/api/hook.go
LamaAni Feb 9, 2022
3d7c85c
Merge branch 'master' into when_on_root
LamaAni Feb 9, 2022
03064b9
Auto commit @ 2022-02-09T23:52:28Z
Feb 9, 2022
9cc9c44
Merge branch 'when_on_root' of github.com:LamaAni/woodpecker into whe…
Feb 9, 2022
ac63f6a
Auto commit @ 2022-02-10T00:03:25Z
Feb 10, 2022
c4d0068
Auto commit @ 2022-02-10T00:35:39Z
Feb 10, 2022
2a23706
Merge branch 'master' into when_on_root
LamaAni Feb 11, 2022
5348a1c
Merge remote-tracking branch 'public/master'
Feb 16, 2022
6d8e019
Auto commit @ 2022-02-16T18:09:27Z
Feb 16, 2022
fb05ce6
Merge branch 'master' into when_on_root
LamaAni Feb 22, 2022
b01ddf4
Merge remote-tracking branch 'public/master'
Feb 22, 2022
64ebd6e
Auto commit @ 2022-02-22T16:15:12Z
Feb 22, 2022
e5ad2f5
Merge branch 'master' of github.com:LamaAni/woodpecker
Feb 28, 2022
e38919c
Auto commit @ 2022-02-28T21:24:15Z
Feb 28, 2022
aa73b3e
Merge remote-tracking branch 'public/master'
Feb 28, 2022
4570698
Auto commit @ 2022-02-28T23:02:48Z
Feb 28, 2022
765b00a
Merge branch 'master' into when_on_root
Feb 28, 2022
35fcd34
Auto commit @ 2022-02-28T23:26:30Z
Feb 28, 2022
4fb9b27
Merge branch 'when_on_root' of github.com:LamaAni/woodpecker into whe…
Feb 28, 2022
cce4c6f
Auto commit @ 2022-03-01T01:18:05Z
Mar 1, 2022
d500489
Auto commit @ 2022-03-01T01:36:45Z
Mar 1, 2022
852ff7a
Auto commit @ 2022-03-01T16:08:31Z
Mar 1, 2022
09d6893
Merge remote-tracking branch 'public/master' into when_on_root
Mar 1, 2022
988e0ca
Auto commit @ 2022-03-01T16:13:19Z
Mar 1, 2022
1ae60c6
Auto commit @ 2022-03-01T16:32:47Z
Mar 1, 2022
fabc54d
Update .vscode/launch.json
LamaAni Mar 1, 2022
9417a18
Auto commit @ 2022-03-01T16:57:26Z
Mar 1, 2022
a80d7fd
Merge branch 'when_on_root' of github.com:LamaAni/woodpecker into whe…
Mar 1, 2022
5c38af7
Auto commit @ 2022-03-01T16:58:01Z
Mar 1, 2022
e7cbd04
Auto commit @ 2022-03-01T16:58:31Z
Mar 1, 2022
68f0c18
Auto commit @ 2022-03-01T16:58:56Z
Mar 1, 2022
094cd02
Auto commit @ 2022-03-01T17:01:02Z
Mar 1, 2022
7bd7535
Auto commit @ 2022-03-01T17:03:13Z
Mar 1, 2022
8c648b8
Merge branch 'master' into when_on_root
LamaAni Mar 2, 2022
546a753
Auto commit @ 2022-03-02T19:28:39Z
Mar 2, 2022
e8965dd
Auto commit @ 2022-03-02T19:33:40Z
Mar 2, 2022
2c92efd
Auto commit @ 2022-03-02T19:40:42Z
Mar 2, 2022
21d86f5
Auto commit @ 2022-03-02T19:42:37Z
Mar 2, 2022
ed41f19
Auto commit @ 2022-03-02T20:09:28Z
Mar 2, 2022
bf487d2
Auto commit @ 2022-03-02T20:10:41Z
Mar 2, 2022
ab43487
Auto commit @ 2022-03-02T20:13:00Z
Mar 2, 2022
4eaa9af
Auto commit @ 2022-03-02T20:29:33Z
Mar 2, 2022
6bae242
Auto commit @ 2022-03-02T20:40:37Z
Mar 2, 2022
d62bfbf
Auto commit @ 2022-03-02T20:41:30Z
Mar 2, 2022
70fed08
Auto commit @ 2022-03-02T20:43:10Z
Mar 2, 2022
4d47566
Merge branch 'when_on_root' of github.com:LamaAni/woodpecker into whe…
Mar 2, 2022
45f51cf
Auto commit @ 2022-03-02T20:44:09Z
Mar 2, 2022
57eeae8
Auto commit @ 2022-03-02T21:39:26Z
Mar 2, 2022
88929b5
Auto commit @ 2022-03-02T23:22:55Z
Mar 2, 2022
b7fa8c3
Auto commit @ 2022-03-02T23:49:36Z
Mar 2, 2022
be93faf
Auto commit @ 2022-03-03T00:00:59Z
Mar 3, 2022
1800b52
Merge branch 'master' into when_on_root
LamaAni Mar 3, 2022
6bbf345
Merge branch 'master' into when_on_root
6543 Jun 15, 2022
6179f46
Merge branch 'master' into when_on_root
6543 Jun 15, 2022
92c5efd
move out to #970
6543 Jun 15, 2022
f403d80
Merge branch 'master' into when_on_root
6543 Jun 15, 2022
82859e2
Merge remote-tracking branch 'upstream/master' into when_on_root
anbraten Aug 9, 2022
7bb6669
add when to docs
anbraten Aug 9, 2022
f6e79fa
improve docs
anbraten Aug 9, 2022
a1c4921
undo change
anbraten Aug 9, 2022
a89a6e8
remove unused property
anbraten Aug 9, 2022
75fd98a
Merge remote-tracking branch 'upstream/master' into when_on_root
anbraten Aug 14, 2022
846c9a7
fix code
anbraten Aug 14, 2022
7d3647a
Merge branch 'master' into when_on_root
6543 Sep 1, 2022
8d4cb85
rm unrelated
6543 Sep 1, 2022
4c0cca3
update jsonschema
6543 Sep 1, 2022
43a1426
diff match constrain between step and pipeline
6543 Sep 1, 2022
88f694a
ajust test
6543 Sep 1, 2022
92cd7f2
Merge branch 'master' into when_on_root
6543 Sep 1, 2022
9ee22f1
deprecate old branches filter
6543 Sep 1, 2022
95e37d3
Apply suggestions from code review
anbraten Sep 3, 2022
2695014
Merge branch 'master' into when_on_root
anbraten Sep 3, 2022
d6d0f9e
some comment adjustments
anbraten Sep 19, 2022
64a52ab
Merge remote-tracking branch 'upstream/master' into when_on_root
anbraten Sep 20, 2022
8ebb77f
Merge branch 'master' into when_on_root
anbraten Sep 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions pipeline/frontend/yaml/compiler/compiler.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,14 @@ func New(opts ...Option) *Compiler {
func (c *Compiler) Compile(conf *yaml.Config) *backend.Config {
config := new(backend.Config)

if !conf.Constraints.Match(c.metadata) {
// basically dont do nothing. No need to run in this case.
// This would also remove this item from the UI
// which seems to be as the case for each of the container
// contraints.
LamaAni marked this conversation as resolved.
Show resolved Hide resolved
return config
}

// create a default volume
config.Volumes = append(config.Volumes, &backend.Volume{
Name: fmt.Sprintf("%s_default", c.prefix),
Expand Down
28 changes: 15 additions & 13 deletions pipeline/frontend/yaml/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,21 @@ import (
type (
// Config defines a pipeline configuration.
Config struct {
Cache types.Stringorslice
Platform string
Branches constraint.List
Workspace Workspace
Clone Containers
Pipeline Containers
Services Containers
Networks Networks
Volumes Volumes
Labels types.SliceorMap
DependsOn []string `yaml:"depends_on,omitempty"`
RunsOn []string `yaml:"runs_on,omitempty"`
SkipClone bool `yaml:"skip_clone"`
// TODO: change to When (also in Pipeline)
6543 marked this conversation as resolved.
Show resolved Hide resolved
Constraints constraint.Constraints `yaml:"when,omitempty"`
Cache types.Stringorslice
Platform string
Branches constraint.List
Workspace Workspace
Clone Containers
Pipeline Containers
Services Containers
Networks Networks
Volumes Volumes
Labels types.SliceorMap
DependsOn []string `yaml:"depends_on,omitempty"`
RunsOn []string `yaml:"runs_on,omitempty"`
SkipClone bool `yaml:"skip_clone"`
}

// Workspace defines a pipeline workspace.
Expand Down
5 changes: 5 additions & 0 deletions pipeline/frontend/yaml/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ func TestParse(t *testing.T) {
g.Fail(err)
}

g.Assert(out.Constraints.Event.Match("push")).Equal(true)

g.Assert(out.Workspace.Base).Equal("/go")
g.Assert(out.Workspace.Path).Equal("src/github.com/octocat/hello-world")
g.Assert(out.Volumes.Volumes[0].Name).Equal("custom")
Expand Down Expand Up @@ -72,6 +74,9 @@ func TestParse(t *testing.T) {

var sampleYaml = `
image: hello-world
when:
event:
- push
build:
context: .
dockerfile: Dockerfile
Expand Down
31 changes: 23 additions & 8 deletions server/api/hook.go
Original file line number Diff line number Diff line change
Expand Up @@ -186,23 +186,23 @@ func PostHook(c *gin.Context) {
return
}

filtered, err := branchFiltered(build, remoteYamlConfigs)
filtered, err := checkIfFiltered(build, remoteYamlConfigs)
if err != nil {
msg := "failure to parse yaml from hook"
log.Debug().Err(err).Str("repo", repo.FullName).Msg(msg)
log.Info().Err(err).Str("repo", repo.FullName).Msg(msg)
c.String(http.StatusBadRequest, msg)
return
}
if filtered {
msg := "ignoring hook: branch does not match restrictions defined in yaml"
log.Debug().Str("repo", repo.FullName).Msg(msg)
msg := "ignoring hook: branch/event does not match restrictions defined in (any) yaml"
LamaAni marked this conversation as resolved.
Show resolved Hide resolved
log.Info().Str("repo", repo.FullName).Msg(msg)
c.String(http.StatusOK, msg)
return
}

if zeroSteps(build, remoteYamlConfigs) {
msg := "ignoring hook: step conditions yield zero runnable steps"
log.Debug().Str("repo", repo.FullName).Msg(msg)
log.Info().Str("repo", repo.FullName).Msg(msg)
c.String(http.StatusOK, msg)
return
}
Expand Down Expand Up @@ -269,7 +269,7 @@ func PostHook(c *gin.Context) {
}

// TODO: parse yaml once and not for each filter function
func branchFiltered(build *model.Build, remoteYamlConfigs []*remote.FileMeta) (bool, error) {
func checkIfFiltered(build *model.Build, remoteYamlConfigs []*remote.FileMeta) (bool, error) {
LamaAni marked this conversation as resolved.
Show resolved Hide resolved
log.Trace().Msgf("hook.branchFiltered(): build branch: '%s' build event: '%s' config count: %d", build.Branch, build.Event, len(remoteYamlConfigs))

if build.Event == model.EventTag || build.Event == model.EventDeploy {
Expand All @@ -284,11 +284,26 @@ func branchFiltered(build *model.Build, remoteYamlConfigs []*remote.FileMeta) (b
}
log.Trace().Msgf("config '%s': %#v", remoteYamlConfig.Name, parsedPipelineConfig)

if parsedPipelineConfig.Branches.Match(build.Branch) {
return false, nil
// if was filtered by the constraints (event) continue
if !parsedPipelineConfig.Constraints.Event.Match(string(build.Event)) {
continue
}

// if was filtered by the constraints (branch) continue
if !parsedPipelineConfig.Constraints.Branch.Match(build.Branch) {
continue
}

// if was filtered by the branch (legacy) continue
if !parsedPipelineConfig.Branches.Match(build.Branch) {
continue
}
6543 marked this conversation as resolved.
Show resolved Hide resolved

// at least one config yielded in a valid run.
return false, nil
}

// no configs yielded a valid run.
return true, nil
}

Expand Down
6 changes: 6 additions & 0 deletions server/shared/procBuilder.go
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,12 @@ func (b *ProcBuilder) Build() ([]*BuildItem, error) {
return nil, err
}

// checking if filtered.
if !parsed.Constraints.Match((metadata)) {
proc.State = model.StatusSkipped
}

// legacy check.
LamaAni marked this conversation as resolved.
Show resolved Hide resolved
if !parsed.Branches.Match(b.Curr.Branch) && (b.Curr.Event != model.EventDeploy && b.Curr.Event != model.EventTag) {
proc.State = model.StatusSkipped
}
Expand Down
46 changes: 46 additions & 0 deletions server/shared/procBuilder_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,52 @@ pipeline:
}
}

func TestRootWhenFilter(t *testing.T) {
t.Parallel()

b := ProcBuilder{
Repo: &model.Repo{},
Curr: &model.Build{Event: "tester"},
Last: &model.Build{},
Netrc: &model.Netrc{},
Secs: []*model.Secret{},
Regs: []*model.Registry{},
Link: "",
Yamls: []*remote.FileMeta{
{Data: []byte(`
when:
event:
- tester
pipeline:
xxx:
image: scratch
`)},
{Data: []byte(`
when:
event:
- push
pipeline:
xxx:
image: scratch
`)},
{Data: []byte(`
pipeline:
build:
image: scratch
`)},
},
}

buildItems, err := b.Build()
if err != nil {
t.Fatal(err)
}

if len(buildItems) != 2 {
t.Fatal("Should have generated 2 buildItems")
}
}

func TestZeroSteps(t *testing.T) {
t.Parallel()

Expand Down