refactor/ci tasks improvement new #5303

antoooks · 2023-08-31T09:45:01Z

fix generate-kustomize-builtin-plugins
add diff checker on generate-kustomize-builtin-plugins
add go mod tidy and go work sync on presubmit check
add generate-kustomize-builtin-plugins on presubmit check
ensure pluginator logic does not create diff
add comments

addresses #4977

k8s-ci-robot · 2023-08-31T09:45:03Z

This PR has multiple commits, and the default merge method is: merge.
You can request commits to be squashed using the label: tide/merge-method-squash

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2023-08-31T09:45:10Z

Hi @antoooks. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

… generate-kustomize-builtin-plugins, add go work sync on presubmit check, add generate-kustomize-builtin-plugins on presubmit check

antoooks · 2023-09-16T17:30:54Z

/assign @koba1t @Debanitrkl

koba1t · 2023-09-17T19:24:52Z

Hi @antoooks
Thanks for your contribution!

make generate-kustomize-builtin-plugins command failed on my computer when I was testing your PR.
Are you successfully running for your computer?

Makefile

antoooks · 2023-09-18T01:19:22Z

Hi @antoooks Thanks for your contribution!

make generate-kustomize-builtin-plugins command failed on my computer when I was testing your PR. Are you successfully running for your computer?

Yes it worked in mine, could you paste me some of the error messages? 😃

koba1t · 2023-09-18T15:17:27Z

@antoooks

could you paste me some of the error messages?

The following error message appears. Do you get this error?

$ make generate-kustomize-builtin-plugins
for plugin in /Users/kob/git/kustomize/plugin/builtin/annotationstransformer /Users/kob/git/kustomize/plugin/builtin/configmapgenerator /Users/kob/git/kustomize/plugin/builtin/hashtransformer /Users/kob/git/kustomize/plugin/builtin/helmchartinflationgenerator /Users/kob/git/kustomize/plugin/builtin/iampolicygenerator /Users/kob/git/kustomize/plugin/builtin/imagetagtransformer /Users/kob/git/kustomize/plugin/builtin/labeltransformer /Users/kob/git/kustomize/plugin/builtin/namespacetransformer /Users/kob/git/kustomize/plugin/builtin/patchjson6902transformer /Users/kob/git/kustomize/plugin/builtin/patchstrategicmergetransformer /Users/kob/git/kustomize/plugin/builtin/patchtransformer /Users/kob/git/kustomize/plugin/builtin/prefixtransformer /Users/kob/git/kustomize/plugin/builtin/replacementtransformer /Users/kob/git/kustomize/plugin/builtin/replicacounttransformer /Users/kob/git/kustomize/plugin/builtin/resourcegenerator /Users/kob/git/kustomize/plugin/builtin/secretgenerator /Users/kob/git/kustomize/plugin/builtin/sortordertransformer /Users/kob/git/kustomize/plugin/builtin/suffixtransformer /Users/kob/git/kustomize/plugin/builtin/valueaddtransformer; do \
                echo "generating ${plugin} ..."; \
                set -e; \
                cd ${plugin}; \
                go generate pluginator .; \
        done; \
        cd ../../../; \
        make no-diff \

generating /Users/kob/git/kustomize/plugin/builtin/annotationstransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/configmapgenerator ...
generating /Users/kob/git/kustomize/plugin/builtin/hashtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/helmchartinflationgenerator ...
generating /Users/kob/git/kustomize/plugin/builtin/iampolicygenerator ...
generating /Users/kob/git/kustomize/plugin/builtin/imagetagtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/labeltransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/namespacetransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/patchjson6902transformer ...
generating /Users/kob/git/kustomize/plugin/builtin/patchstrategicmergetransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/patchtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/prefixtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/replacementtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/replicacounttransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/resourcegenerator ...
go: cannot find main module, but found .git/config in /Users/kob/git/kustomize
        to create a module there, run:
        cd ../../.. && go mod init
make: *** [generate-kustomize-builtin-plugins] Error 1

antoooks · 2023-09-18T16:21:46Z

@antoooks

could you paste me some of the error messages?

The following error message appears. Do you get this error?

$ make generate-kustomize-builtin-plugins
for plugin in /Users/kob/git/kustomize/plugin/builtin/annotationstransformer /Users/kob/git/kustomize/plugin/builtin/configmapgenerator /Users/kob/git/kustomize/plugin/builtin/hashtransformer /Users/kob/git/kustomize/plugin/builtin/helmchartinflationgenerator /Users/kob/git/kustomize/plugin/builtin/iampolicygenerator /Users/kob/git/kustomize/plugin/builtin/imagetagtransformer /Users/kob/git/kustomize/plugin/builtin/labeltransformer /Users/kob/git/kustomize/plugin/builtin/namespacetransformer /Users/kob/git/kustomize/plugin/builtin/patchjson6902transformer /Users/kob/git/kustomize/plugin/builtin/patchstrategicmergetransformer /Users/kob/git/kustomize/plugin/builtin/patchtransformer /Users/kob/git/kustomize/plugin/builtin/prefixtransformer /Users/kob/git/kustomize/plugin/builtin/replacementtransformer /Users/kob/git/kustomize/plugin/builtin/replicacounttransformer /Users/kob/git/kustomize/plugin/builtin/resourcegenerator /Users/kob/git/kustomize/plugin/builtin/secretgenerator /Users/kob/git/kustomize/plugin/builtin/sortordertransformer /Users/kob/git/kustomize/plugin/builtin/suffixtransformer /Users/kob/git/kustomize/plugin/builtin/valueaddtransformer; do \
                echo "generating ${plugin} ..."; \
                set -e; \
                cd ${plugin}; \
                go generate pluginator .; \
        done; \
        cd ../../../; \
        make no-diff \

generating /Users/kob/git/kustomize/plugin/builtin/annotationstransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/configmapgenerator ...
generating /Users/kob/git/kustomize/plugin/builtin/hashtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/helmchartinflationgenerator ...
generating /Users/kob/git/kustomize/plugin/builtin/iampolicygenerator ...
generating /Users/kob/git/kustomize/plugin/builtin/imagetagtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/labeltransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/namespacetransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/patchjson6902transformer ...
generating /Users/kob/git/kustomize/plugin/builtin/patchstrategicmergetransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/patchtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/prefixtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/replacementtransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/replicacounttransformer ...
generating /Users/kob/git/kustomize/plugin/builtin/resourcegenerator ...
go: cannot find main module, but found .git/config in /Users/kob/git/kustomize
        to create a module there, run:
        cd ../../.. && go mod init
make: *** [generate-kustomize-builtin-plugins] Error 1

@koba1t I have tested by cloning it inside a docker container and I cannot find this plugin/builtin/resourcegenerator module on master or on my branch, I think this is something added when you switched your work tree. I suggest you start cloning clean on a different directory. Then do make uninstall-tools and make install-tools.

koba1t · 2023-09-18T17:31:23Z

@antoooks

Yes, You are right.
My computer has an empty dir named plugin/builtin/resourcegenerator and this empty dir is watched from Makefile script at this line
This command is successful after deleting this empty dir.

Apologies for overlooking the mistake.

antoooks · 2023-09-19T01:37:52Z

@antoooks

Yes, You are right. My computer has an empty dir named plugin/builtin/resourcegenerator and this empty dir is watched from Makefile script at this line This command is successful after deleting this empty dir.

Apologies for overlooking the mistake.

No worries, glad it worked! I also have added all the feedback for workspace-sync step

rayandas · 2023-09-21T05:17:19Z

/ok-to-test
Edit: Ah! Seems like we need a ok-to-test from maintainers.

koba1t

Hi, @antoooks!
I added a few review comments. Please check my comments!

koba1t · 2023-09-28T07:37:35Z

Makefile

-
+# Pushes dependencies in the go.work file back to go.mod files of each workspace module.
+.PHONY: workspace-sync
+workspace sync:


Do you have any specific reason to remove the hyphen here?
I think it is better to use a unified name here.

No issue here, yes we can go with unified name

koba1t · 2023-09-28T07:51:22Z

Makefile-plugins.mk

 .PHONY: generate-kustomize-builtin-plugins
-generate-kustomize-builtin-plugins: $(builtinplugins)
+generate-kustomize-builtin-plugins: $(builtplugins)


Maybe I think it is better to separate a step of generate and a step of diff check.
Because generate-kustomize-builtin-plugins will be used in developing built-in plugins. This command may error when developing plugins before exec git commit.
I think it better to separate them there, It will be useful for developing plugins.

I see your point. Do you know if diff check will be called anywhere else? Since the context is strictly limited to builtin plugins maybe I should use builtin-plugins no diff as target name instead

I imagine making a new builtin-plugins diff command, and that makes called from here.
I'll leave it to you to figure out how to do it.

koba1t · 2023-10-18T17:45:59Z

/assign

koba1t · 2023-10-19T07:27:52Z

@antoooks
Thanks for the fix! I appreciate your contribution!
This PR looks good to me. I'll merge this PR!

/lgtm
/approve

k8s-ci-robot · 2023-10-19T07:27:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: antoooks, koba1t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [koba1t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 31, 2023

k8s-ci-robot requested review from koba1t and yuwenma August 31, 2023 09:45

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 31, 2023

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 31, 2023

antoooks mentioned this pull request Aug 31, 2023

draft: refactor: ci-task-improvement #5296

Closed

antoooks mentioned this pull request Sep 14, 2023

feat: add CheckDiffOnFile in git module on gorepomod #5302

Closed

antoooks added 3 commits September 16, 2023 21:47

refactor: fix generate-kustimize-builtin-plugins, add diff checker on…

532abf7

… generate-kustomize-builtin-plugins, add go work sync on presubmit check, add generate-kustomize-builtin-plugins on presubmit check

add commentaries, change command order on presubmit-check

d78fe66

add go mod tidy before workspace sync

8f28349

antoooks force-pushed the refactor/ci-tasks-improvement-new branch from d9b088d to 8f28349 Compare September 16, 2023 14:47

Edit commentaries and ensure pluginator logic does not create diff

a90c19d

antoooks marked this pull request as ready for review September 16, 2023 17:25

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 16, 2023

k8s-ci-robot requested a review from natasha41575 September 16, 2023 17:25

antoooks closed this Sep 16, 2023

antoooks reopened this Sep 16, 2023

k8s-ci-robot assigned Debanitrkl and koba1t Sep 16, 2023

koba1t requested changes Sep 17, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

use doGoMod.sh forr go mod tidy

9e101ee

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 21, 2023

koba1t requested changes Sep 28, 2023

View reviewed changes

separate generate plugins and check diff step

d91ea65

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 19, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 19, 2023

k8s-ci-robot merged commit 752bb2e into kubernetes-sigs:master Oct 19, 2023
9 checks passed

antoooks mentioned this pull request Oct 22, 2023

Some CI tasks should include verification that no diff was produced #4977

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor/ci tasks improvement new #5303

refactor/ci tasks improvement new #5303

antoooks commented Aug 31, 2023 •

edited

Loading

k8s-ci-robot commented Aug 31, 2023

k8s-ci-robot commented Aug 31, 2023

antoooks commented Sep 16, 2023

koba1t commented Sep 17, 2023

antoooks commented Sep 18, 2023 •

edited

Loading

koba1t commented Sep 18, 2023

antoooks commented Sep 18, 2023

koba1t commented Sep 18, 2023

antoooks commented Sep 19, 2023

rayandas commented Sep 21, 2023 •

edited

Loading

koba1t left a comment

koba1t Sep 28, 2023

antoooks Sep 28, 2023

koba1t Sep 28, 2023

antoooks Sep 28, 2023

koba1t Sep 28, 2023

koba1t commented Oct 18, 2023

koba1t commented Oct 19, 2023

k8s-ci-robot commented Oct 19, 2023

refactor/ci tasks improvement new #5303

refactor/ci tasks improvement new #5303

Conversation

antoooks commented Aug 31, 2023 • edited Loading

k8s-ci-robot commented Aug 31, 2023

k8s-ci-robot commented Aug 31, 2023

antoooks commented Sep 16, 2023

koba1t commented Sep 17, 2023

antoooks commented Sep 18, 2023 • edited Loading

koba1t commented Sep 18, 2023

antoooks commented Sep 18, 2023

koba1t commented Sep 18, 2023

antoooks commented Sep 19, 2023

rayandas commented Sep 21, 2023 • edited Loading

koba1t left a comment

Choose a reason for hiding this comment

koba1t Sep 28, 2023

Choose a reason for hiding this comment

antoooks Sep 28, 2023

Choose a reason for hiding this comment

koba1t Sep 28, 2023

Choose a reason for hiding this comment

antoooks Sep 28, 2023

Choose a reason for hiding this comment

koba1t Sep 28, 2023

Choose a reason for hiding this comment

koba1t commented Oct 18, 2023

koba1t commented Oct 19, 2023

k8s-ci-robot commented Oct 19, 2023

antoooks commented Aug 31, 2023 •

edited

Loading

antoooks commented Sep 18, 2023 •

edited

Loading

rayandas commented Sep 21, 2023 •

edited

Loading