From 0c2f9291c6921831d3b23e09032564f922c6b0be Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Mon, 29 Aug 2022 23:31:50 +0200 Subject: [PATCH 01/10] fix make vendor target --- Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/Makefile b/Makefile index 0f02cfaa8c..1425301046 100644 --- a/Makefile +++ b/Makefile @@ -52,6 +52,7 @@ else all: build +.PHONY: vendor vendor: go mod tidy go mod vendor From 9fbccd8dff1cd7d77d089e023527a266e19c2efb Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Mon, 29 Aug 2022 23:33:15 +0200 Subject: [PATCH 02/10] docs: ajust highliting --- docs/docs/20-usage/20-pipeline-syntax.md | 36 ++++++++++++------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/docs/20-usage/20-pipeline-syntax.md b/docs/docs/20-usage/20-pipeline-syntax.md index f224c93144..6a86d96625 100644 --- a/docs/docs/20-usage/20-pipeline-syntax.md +++ b/docs/docs/20-usage/20-pipeline-syntax.md @@ -100,7 +100,7 @@ pipeline: Woodpecker gives the ability to skip individual commits by adding `[CI SKIP]` to the commit message. Note this is case-insensitive. -```diff +```sh git commit -m "updated README [CI SKIP]" ``` @@ -234,7 +234,7 @@ Commands of every pipeline step are executed serially as if you would enter them There is no magic here. The above commands are converted to a simple shell script. The commands in the above example are roughly converted to the below script: -```diff +```sh #!/bin/sh set -e @@ -244,7 +244,7 @@ go test The above shell script is then executed as the container entrypoint. The below docker command is an (incomplete) example of how the script is executed: -```bash +```sh docker run --entrypoint=build.sh golang ``` @@ -315,21 +315,21 @@ pipeline: Execute a step if the branch is `master` or `develop`: -```diff +```yaml when: - branch: [master, develop] ``` Execute a step if the branch starts with `prefix/*`: -```diff +```yaml when: - branch: prefix/* ``` Execute a step using custom include and exclude logic: -```diff +```yaml when: - branch: include: [ master, release/* ] @@ -340,7 +340,7 @@ when: Execute a step if the build event is a `tag`: -```diff +```yaml when: - event: tag ``` @@ -355,14 +355,14 @@ when: Execute a step for all non-pull request events: -```diff +```yaml when: - event: [push, tag, deployment] ``` Execute a step for all build events: -```diff +```yaml when: - event: [push, pull_request, tag, deployment] ``` @@ -372,7 +372,7 @@ when: This filter only applies to tag events. Use glob expression to execute a step if the tag name starts with `v`: -```diff +```yaml when: - event: tag tag: v* @@ -400,14 +400,14 @@ This condition should be used in conjunction with a [matrix](/docs/usage/matrix- Execute a step for a specific platform: -```diff +```yaml when: - platform: linux/amd64 ``` Execute a step for a specific platform using wildcards: -```diff +```yaml when: - platform: [ linux/*, windows/amd64 ] ``` @@ -416,7 +416,7 @@ when: Execute a step for deployment events matching the target deployment environment: -```diff +```yaml when: - environment: production - event: deployment @@ -426,7 +426,7 @@ when: Execute a step for a single matrix permutation: -```diff +```yaml when: - matrix: GO_VERSION: 1.5 @@ -437,7 +437,7 @@ when: Execute a step only on a certain Woodpecker instance matching the specified hostname: -```diff +```yaml when: - instance: stage.woodpecker.company.com ``` @@ -452,14 +452,14 @@ Gitea only support **push** at the moment ([go-gitea/gitea#18228](https://github Execute a step only on a pipeline with certain files being changed: -```diff +```yaml when: - path: "src/*" ``` You can use [glob patterns](https://github.com/bmatcuk/doublestar#patterns) to match the changed files and specify if the step should run if a file matching that pattern has been changed `include` or if some files have **not** been changed `exclude`. -```diff +```yaml when: - path: include: [ '.woodpecker/*.yml', '*.ini' ] @@ -559,7 +559,7 @@ The base attribute defines a shared base volume available to all pipeline steps. This would be equivalent to the following docker commands: -```bash +```sh docker volume create my-named-volume docker run --volume=my-named-volume:/go golang:latest From 583bb97209786368edb747d28e7d17e744f486b8 Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Mon, 29 Aug 2022 23:33:34 +0200 Subject: [PATCH 03/10] docs: mention default event filter --- docs/docs/20-usage/20-pipeline-syntax.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/docs/20-usage/20-pipeline-syntax.md b/docs/docs/20-usage/20-pipeline-syntax.md index 6a86d96625..040bb50769 100644 --- a/docs/docs/20-usage/20-pipeline-syntax.md +++ b/docs/docs/20-usage/20-pipeline-syntax.md @@ -338,6 +338,14 @@ when: #### `event` +Default filter steps by event types. + +:::info +**default is: `push, pull_request, tag, deployment`.** +::: + +existing types: `push`, `pull_request`, `tag`, `deployment`, `cron` + Execute a step if the build event is a `tag`: ```yaml From 03a23162ab1cdbf95e6d935de7d655685523bd61 Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Mon, 29 Aug 2022 23:39:00 +0200 Subject: [PATCH 04/10] implement default event filter in code --- .../frontend/yaml/constraint/constraint.go | 23 ++++++- .../yaml/constraint/constraint_test.go | 67 ++++++++++--------- pipeline/schema/schema.json | 12 ++-- server/shared/procBuilder_test.go | 4 +- 4 files changed, 66 insertions(+), 40 deletions(-) diff --git a/pipeline/frontend/yaml/constraint/constraint.go b/pipeline/frontend/yaml/constraint/constraint.go index b6e763ed59..45948c6fff 100644 --- a/pipeline/frontend/yaml/constraint/constraint.go +++ b/pipeline/frontend/yaml/constraint/constraint.go @@ -63,7 +63,13 @@ func (when *When) Match(metadata frontend.Metadata) bool { return true } } - return when.IsEmpty() + + if when.IsEmpty() { + // test against default Constraints + empty := &Constraint{} + return empty.Match(metadata) + } + return false } func (when *When) IncludesStatus(status string) bool { @@ -120,6 +126,16 @@ func (when *When) UnmarshalYAML(value *yaml.Node) error { // Match returns true if all constraints match the given input. If a single // constraint fails a false value is returned. func (c *Constraint) Match(metadata frontend.Metadata) bool { + // if event filter is not set, set default + if c.Event.IsEmpty() { + c.Event.Include = []string{ + frontend.EventPush, + frontend.EventPull, + frontend.EventTag, + frontend.EventDeploy, + } + } + match := c.Platform.Match(metadata.Sys.Platform) && c.Environment.Match(metadata.Curr.Target) && c.Event.Match(metadata.Curr.Event) && @@ -140,6 +156,11 @@ func (c *Constraint) Match(metadata frontend.Metadata) bool { return match } +// IsEmpty return true if a constraint has no conditions +func (c List) IsEmpty() bool { + return len(c.Include) == 0 && len(c.Exclude) == 0 +} + // Match returns true if the string matches the include patterns and does not // match any of the exclude patterns. func (c *List) Match(v string) bool { diff --git a/pipeline/frontend/yaml/constraint/constraint_test.go b/pipeline/frontend/yaml/constraint/constraint_test.go index 94c364b18b..1f60e6a5c9 100644 --- a/pipeline/frontend/yaml/constraint/constraint_test.go +++ b/pipeline/frontend/yaml/constraint/constraint_test.go @@ -383,89 +383,90 @@ func TestConstraintMap(t *testing.T) { func TestConstraints(t *testing.T) { testdata := []struct { + desc string conf string with frontend.Metadata want bool }{ - // no constraints, must match { + desc: "no constraints, must match on default events", conf: "", - with: frontend.Metadata{}, + with: frontend.Metadata{ + Curr: frontend.Build{ + Event: frontend.EventPush, + }, + }, want: true, }, - // branch constraint { + desc: "global branch filter", conf: "{ branch: develop }", - with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Branch: "master"}}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush, Commit: frontend.Commit{Branch: "master"}}}, want: false, }, { + desc: "global branch filter", conf: "{ branch: master }", - with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Branch: "master"}}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush, Commit: frontend.Commit{Branch: "master"}}}, want: true, }, - // environment constraint - { - conf: "{ branch: develop }", - with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Branch: "master"}}}, - want: false, - }, - { - conf: "{ branch: master }", - with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Branch: "master"}}}, - want: true, - }, - // repo constraint { + desc: "repo constraint", conf: "{ repo: owner/* }", - with: frontend.Metadata{Repo: frontend.Repo{Name: "owner/repo"}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush}, Repo: frontend.Repo{Name: "owner/repo"}}, want: true, }, { + desc: "repo constraint", conf: "{ repo: octocat/* }", - with: frontend.Metadata{Repo: frontend.Repo{Name: "owner/repo"}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush}, Repo: frontend.Repo{Name: "owner/repo"}}, want: false, }, - // ref constraint { + desc: "ref constraint", conf: "{ ref: refs/tags/* }", - with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Ref: "refs/tags/v1.0.0"}}}, + with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Ref: "refs/tags/v1.0.0"}, Event: frontend.EventPush}}, want: true, }, { + desc: "ref constraint", conf: "{ ref: refs/tags/* }", - with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Ref: "refs/heads/master"}}}, + with: frontend.Metadata{Curr: frontend.Build{Commit: frontend.Commit{Ref: "refs/heads/master"}, Event: frontend.EventPush}}, want: false, }, - // platform constraint { + desc: "platform constraint", conf: "{ platform: linux/amd64 }", - with: frontend.Metadata{Sys: frontend.System{Platform: "linux/amd64"}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush}, Sys: frontend.System{Platform: "linux/amd64"}}, want: true, }, { + desc: "platform constraint", conf: "{ repo: linux/amd64 }", - with: frontend.Metadata{Sys: frontend.System{Platform: "windows/amd64"}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush}, Sys: frontend.System{Platform: "windows/amd64"}}, want: false, }, - // instance constraint { + desc: "instance constraint", conf: "{ instance: agent.tld }", - with: frontend.Metadata{Sys: frontend.System{Host: "agent.tld"}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush}, Sys: frontend.System{Host: "agent.tld"}}, want: true, }, { + desc: "instance constraint", conf: "{ instance: agent.tld }", - with: frontend.Metadata{Sys: frontend.System{Host: "beta.agent.tld"}}, + with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush}, Sys: frontend.System{Host: "beta.agent.tld"}}, want: false, }, } for _, test := range testdata { - c := parseConstraints(t, test.conf) - got, want := c.Match(test.with), test.want - if got != want { - t.Errorf("Expect %+v matches %q is %v", test.with, test.conf, want) - } + t.Run(test.desc, func(t *testing.T) { + c := parseConstraints(t, test.conf) + got, want := c.Match(test.with), test.want + if got != want { + t.Errorf("Expect %+v matches %q is %v", test.with, test.conf, want) + } + }) } } diff --git a/pipeline/schema/schema.json b/pipeline/schema/schema.json index 5068647695..1d97aec627 100644 --- a/pipeline/schema/schema.json +++ b/pipeline/schema/schema.json @@ -186,13 +186,11 @@ "oneOf": [ { "type": "array", - "items": { - "enum": ["push", "pull_request", "tag", "deployment"] - }, - "minLength": 1 + "minLength": 1, + "items": { "$ref": "#/definitions/event_enum" } }, { - "enum": ["push", "pull_request", "tag", "deployment"] + "$ref": "#/definitions/event_enum" } ] }, @@ -270,6 +268,10 @@ } } }, + "event_enum": { + "default": ["push", "pull_request", "tag", "deployment"], + "enum": ["push", "pull_request", "tag", "deployment", "cron"] + }, "step_image": { "description": "Read more: https://woodpecker-ci.org/docs/usage/pipeline-syntax#image", "type": "string" diff --git a/server/shared/procBuilder_test.go b/server/shared/procBuilder_test.go index 77bb891c55..2b92281caf 100644 --- a/server/shared/procBuilder_test.go +++ b/server/shared/procBuilder_test.go @@ -465,7 +465,9 @@ depends_on: [ shouldbefiltered ] func TestTree(t *testing.T) { t.Parallel() - build := &model.Build{} + build := &model.Build{ + Event: model.EventPush, + } b := ProcBuilder{ Repo: &model.Repo{}, From dfaa4e8eb8c2f98beae59e8a2c30b49e0a2df651 Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Mon, 29 Aug 2022 23:44:46 +0200 Subject: [PATCH 05/10] add dedicated test --- pipeline/frontend/yaml/constraint/constraint_test.go | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/pipeline/frontend/yaml/constraint/constraint_test.go b/pipeline/frontend/yaml/constraint/constraint_test.go index 1f60e6a5c9..1a48f37a40 100644 --- a/pipeline/frontend/yaml/constraint/constraint_test.go +++ b/pipeline/frontend/yaml/constraint/constraint_test.go @@ -458,6 +458,14 @@ func TestConstraints(t *testing.T) { with: frontend.Metadata{Curr: frontend.Build{Event: frontend.EventPush}, Sys: frontend.System{Host: "beta.agent.tld"}}, want: false, }, + { + desc: "no constraints, and event get filtered by default default event filter", + conf: "", + with: frontend.Metadata{ + Curr: frontend.Build{Event: "non-default"}, + }, + want: false, + }, } for _, test := range testdata { t.Run(test.desc, func(t *testing.T) { From f7613e53b6d366b02359d060e07a51c54d743919 Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Mon, 29 Aug 2022 23:53:08 +0200 Subject: [PATCH 06/10] update --- docs/docs/20-usage/20-pipeline-syntax.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/docs/20-usage/20-pipeline-syntax.md b/docs/docs/20-usage/20-pipeline-syntax.md index 040bb50769..0a7eeb60d2 100644 --- a/docs/docs/20-usage/20-pipeline-syntax.md +++ b/docs/docs/20-usage/20-pipeline-syntax.md @@ -338,13 +338,13 @@ when: #### `event` -Default filter steps by event types. - :::info -**default is: `push, pull_request, tag, deployment`.** +**By default steps are filtered by following event types:** + +`push, pull_request, tag, deployment`. ::: -existing types: `push`, `pull_request`, `tag`, `deployment`, `cron` +Available types: `push`, `pull_request`, `tag`, `deployment`, `cron` Execute a step if the build event is a `tag`: @@ -361,14 +361,14 @@ when: + branch: main ``` -Execute a step for all non-pull request events: +Execute a step for multiple events: ```yaml when: - event: [push, tag, deployment] ``` -Execute a step for all build events: +Execute a step for all pipeline events: ```yaml when: From 56124d35813faaabca1dc4ceb1571261e5dc70b2 Mon Sep 17 00:00:00 2001 From: 6543 <6543@obermui.de> Date: Mon, 29 Aug 2022 23:54:38 +0200 Subject: [PATCH 07/10] Update pipeline/schema/schema.json Co-authored-by: Anbraten --- pipeline/schema/schema.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/schema/schema.json b/pipeline/schema/schema.json index 1d97aec627..259e68189b 100644 --- a/pipeline/schema/schema.json +++ b/pipeline/schema/schema.json @@ -270,7 +270,7 @@ }, "event_enum": { "default": ["push", "pull_request", "tag", "deployment"], - "enum": ["push", "pull_request", "tag", "deployment", "cron"] + "enum": ["push", "pull_request", "tag", "deployment"] }, "step_image": { "description": "Read more: https://woodpecker-ci.org/docs/usage/pipeline-syntax#image", From 25897eb9c43ee47fb7e67c9ba0aa271d79d4eb00 Mon Sep 17 00:00:00 2001 From: Anbraten Date: Mon, 29 Aug 2022 23:59:06 +0200 Subject: [PATCH 08/10] Apply suggestions from code review --- docs/docs/20-usage/20-pipeline-syntax.md | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/docs/docs/20-usage/20-pipeline-syntax.md b/docs/docs/20-usage/20-pipeline-syntax.md index 0a7eeb60d2..797d8badb7 100644 --- a/docs/docs/20-usage/20-pipeline-syntax.md +++ b/docs/docs/20-usage/20-pipeline-syntax.md @@ -341,10 +341,10 @@ when: :::info **By default steps are filtered by following event types:** -`push, pull_request, tag, deployment`. +`push`, `pull_request, `tag`, `deployment`. ::: -Available types: `push`, `pull_request`, `tag`, `deployment`, `cron` +Available types: `push`, `pull_request`, `tag`, `deployment` Execute a step if the build event is a `tag`: @@ -368,12 +368,6 @@ when: - event: [push, tag, deployment] ``` -Execute a step for all pipeline events: - -```yaml -when: - - event: [push, pull_request, tag, deployment] -``` #### `tag` From 2985a133dc0b2089f37c1e6cf5fe5955054af2bb Mon Sep 17 00:00:00 2001 From: Anbraten Date: Mon, 29 Aug 2022 23:59:55 +0200 Subject: [PATCH 09/10] Update docs/docs/20-usage/20-pipeline-syntax.md --- docs/docs/20-usage/20-pipeline-syntax.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/docs/20-usage/20-pipeline-syntax.md b/docs/docs/20-usage/20-pipeline-syntax.md index 797d8badb7..69f7df0071 100644 --- a/docs/docs/20-usage/20-pipeline-syntax.md +++ b/docs/docs/20-usage/20-pipeline-syntax.md @@ -368,7 +368,6 @@ when: - event: [push, tag, deployment] ``` - #### `tag` This filter only applies to tag events. From af8338b38b5d87a28c5a01d0bc7ca0385c68a6ac Mon Sep 17 00:00:00 2001 From: Anbraten Date: Tue, 30 Aug 2022 00:00:30 +0200 Subject: [PATCH 10/10] Update docs/docs/20-usage/20-pipeline-syntax.md --- docs/docs/20-usage/20-pipeline-syntax.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/20-usage/20-pipeline-syntax.md b/docs/docs/20-usage/20-pipeline-syntax.md index 69f7df0071..23773c5c8c 100644 --- a/docs/docs/20-usage/20-pipeline-syntax.md +++ b/docs/docs/20-usage/20-pipeline-syntax.md @@ -344,7 +344,7 @@ when: `push`, `pull_request, `tag`, `deployment`. ::: -Available types: `push`, `pull_request`, `tag`, `deployment` +Available events: `push`, `pull_request`, `tag`, `deployment` Execute a step if the build event is a `tag`: