refactor: refact the webhook API and life process #18255

chlins · 2023-02-21T03:45:54Z

Introduce new APIs for webhook jobs management.
Refact legacy APIs for backforward compatible.
Migrate the webhook jobs process to unified execution/task framework.

Closes: #18210

Thank you for contributing to Harbor!

Comprehensive Summary of your change

Issue being fixed

Fixes #18210

Please indicate you've done the following:

Well Written Title and Summary of the PR
Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
Accepted the DCO. Commits without the DCO will delay acceptance.
Made sure tests are passing and test coverage is added if needed.
Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

codecov · 2023-02-21T03:49:17Z

Codecov Report

Merging #18255 (301d02a) into main (1f3f732) will increase coverage by 0.48%.
The diff coverage is 63.00%.

@@            Coverage Diff             @@
##             main   #18255      +/-   ##
==========================================
+ Coverage   67.02%   67.51%   +0.48%     
==========================================
  Files         971      969       -2     
  Lines      104893   105685     +792     
  Branches     2580     2580              
==========================================
+ Hits        70305    71352    +1047     
+ Misses      30847    30525     -322     
- Partials     3741     3808      +67

Flag	Coverage Δ
unittests	`67.51% <63.00%> (+0.48%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/core/middlewares/middlewares.go	`0.00% <ø> (ø)`
src/pkg/notification/policy/manager.go	`56.92% <ø> (+14.06%)`	⬆️
src/server/v2.0/handler/handler.go	`0.00% <0.00%> (ø)`
src/server/v2.0/handler/webhook.go	`47.05% <48.71%> (ø)`
src/server/v2.0/handler/webhook_job.go	`46.93% <54.54%> (ø)`
src/server/v2.0/handler/model/webhook_job.go	`69.69% <69.69%> (ø)`
src/controller/webhook/controller.go	`82.92% <82.92%> (ø)`
src/pkg/notification/notification.go	`90.47% <100.00%> (+60.65%)`	⬆️
src/server/v2.0/handler/model/webhook_policy.go	`71.42% <100.00%> (ø)`
...dashboard/schedule-card/schedule-card.component.ts	`43.28% <0.00%> (-2.99%)`	⬇️
... and 13 more

src/pkg/notification/hook/hook.go

stonezdj · 2023-02-28T08:07:53Z

src/server/route.go

@@ -48,7 +47,6 @@ func registerRoutes() {
 	web.Router("/api/internal/renameadmin", &api.InternalAPI{}, "post:RenameAdmin")
 	web.Router("/api/internal/syncquota", &api.InternalAPI{}, "post:SyncQuota")

-	web.Router("/service/notifications/jobs/webhook/:id([0-9]+)", &jobs.Handler{}, "post:HandleNotificationJob")


Maybe some of this route are not used anymore?

/service/notifications/jobs/adminjob/:id([0-9]+) /service/notifications/jobs/scan/:uuid /service/notifications/jobs/replication/:id([0-9]+) /service/notifications/jobs/replication/task/:id([0-9]+) /service/notifications/jobs/retention/task/:id([0-9]+)

agree that we can raise another PR to remove the unused route.

src/server/v2.0/handler/model/notification_job.go

src/pkg/notification/hook/hook.go

src/server/v2.0/handler/webhook.go

wy65701436

lgtm

1. Introduce new APIs for webhook jobs management. 2. Refact legacy APIs for backforward compatible. 3. Migrate the webhook jobs process to unified execution/task framework. Closes: goharbor#18210 Signed-off-by: chlins <chenyuzh@vmware.com>

refactor: refact the notification job API and life process 1. Introduce new APIs for webhook jobs management. 2. Refact legacy APIs for backforward compatible. 3. Migrate the webhook jobs process to unified execution/task framework. Closes: goharbor#18210 Signed-off-by: chlins <chenyuzh@vmware.com>

Signed-off-by: Kay Yan <kay.yan@daocloud.io> Improve swagger descriptions (without changing security settings) (goharbor#18205) * Improve swagger definitions. References goharbor#15624 Remove chartmuseum test case (goharbor#18267) Harbor deprecates chartmuseum as of v2.8.0 Epic: goharbor#17958 Discussion: goharbor#15057 Signed-off-by: Yang Jiao <jiaoya@vmware.com> Remove chart api (goharbor#18265) * remove chart api from swagger Delete the api path for chart in the swagger Signed-off-by: Wang Yan <wangyan@vmware.com> * Remove chart APIs 1. Remove some chart-related steps for Dockerfile for portal container 2. Remove chart APIs for API center Signed-off-by: AllForNothing <sshijun@vmware.com> --------- Signed-off-by: Wang Yan <wangyan@vmware.com> Signed-off-by: AllForNothing <sshijun@vmware.com> Co-authored-by: Wang Yan <wangyan@vmware.com> Sort user and usergroup by most match order (goharbor#18273) fixes goharbor#17859 Signed-off-by: stonezdj <daojunz@vmware.com> Update the replication rule related to the Chartmuseum (goharbor#18274) Update the registry and replication rule related to the Chartmuseum Update replication_policy and registry as Harbor v2.8.0 deprecates chartmuseum. Harbor deprecates chartmuseum as of v2.8.0 Epic: goharbor#17958 Discussion: goharbor#15057 Signed-off-by: Yang Jiao <jiaoya@vmware.com> refactor: refact the webhook API and life process (goharbor#18255) refactor: refact the notification job API and life process 1. Introduce new APIs for webhook jobs management. 2. Refact legacy APIs for backforward compatible. 3. Migrate the webhook jobs process to unified execution/task framework. Closes: goharbor#18210 Signed-off-by: chlins <chenyuzh@vmware.com> Add retry times for calling CVE data export execution list API (goharbor#18297) 1.Fixes goharbor#17879 2.Add retry times to avoid endless API calls for CVE data export execution list API Signed-off-by: AllForNothing <sshijun@vmware.com> Get job log by job_id in worker (goharbor#18261) Get job by job_id in redis Get the last 10MB of data if it exceeds 10MB Signed-off-by: stonezdj <daojunz@vmware.com> Enhance webhook UI (goharbor#18291) 1. Add execution list for a certain webhook policy 2. Add task list for a certain execution Signed-off-by: AllForNothing <sshijun@vmware.com> Update UI for the deprecation of Artifact Hub (goharbor#18303) 1. Update edit-registry page to enable the edition for Artifact Hub 2. Update edit-replication-rule page to enable the edition for rules contain Artifact Hub Signed-off-by: AllForNothing <sshijun@vmware.com> Add logs column for worker datagrid (goharbor#18307) 1. Related back-end PR goharbor#18261 Signed-off-by: AllForNothing <sshijun@vmware.com>

chlins added the target/2.8.0 label Feb 21, 2023

chlins requested a review from a team as a code owner February 21, 2023 03:45

chlins assigned stonezdj and wy65701436 Feb 21, 2023

github-actions bot assigned MinerYang and Vad1mo Feb 21, 2023

chlins force-pushed the refactor/webhook-job branch from 1a684b9 to 7015be9 Compare February 21, 2023 03:49

chlins added the release-note/new-feature New Harbor Feature label Feb 21, 2023

chlins force-pushed the refactor/webhook-job branch 6 times, most recently from 01dd2c0 to 90df049 Compare February 24, 2023 08:25

stonezdj reviewed Feb 28, 2023

View reviewed changes

src/pkg/notification/hook/hook.go Show resolved Hide resolved

chlins force-pushed the refactor/webhook-job branch 2 times, most recently from ecfd0d3 to 4220552 Compare March 1, 2023 01:36

stonezdj reviewed Mar 1, 2023

View reviewed changes

AllForNothing mentioned this pull request Mar 1, 2023

Enhance webhook UI #18291

Merged

5 tasks

wy65701436 reviewed Mar 1, 2023

View reviewed changes

src/server/v2.0/handler/model/notification_job.go Outdated Show resolved Hide resolved

wy65701436 reviewed Mar 2, 2023

View reviewed changes

src/pkg/notification/hook/hook.go Show resolved Hide resolved

wy65701436 reviewed Mar 2, 2023

View reviewed changes

src/server/v2.0/handler/webhook.go Outdated Show resolved Hide resolved

wy65701436 approved these changes Mar 2, 2023

View reviewed changes

stonezdj approved these changes Mar 2, 2023

View reviewed changes

chlins force-pushed the refactor/webhook-job branch from 4220552 to b9be2eb Compare March 2, 2023 07:58

chlins changed the title ~~refactor: refact the notification job API and life process~~ refactor: refact the webhook API and life process Mar 2, 2023

chlins force-pushed the refactor/webhook-job branch from b9be2eb to b4b6a70 Compare March 2, 2023 08:46

chlins force-pushed the refactor/webhook-job branch from b4b6a70 to 301d02a Compare March 3, 2023 01:23

chlins merged commit 90db04e into goharbor:main Mar 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: refact the webhook API and life process #18255

refactor: refact the webhook API and life process #18255

chlins commented Feb 21, 2023

codecov bot commented Feb 21, 2023 •

edited

Loading

stonezdj Feb 28, 2023

wy65701436 Mar 1, 2023

wy65701436 left a comment

refactor: refact the webhook API and life process #18255

refactor: refact the webhook API and life process #18255

Conversation

chlins commented Feb 21, 2023

Comprehensive Summary of your change

Issue being fixed

codecov bot commented Feb 21, 2023 • edited Loading

Codecov Report

stonezdj Feb 28, 2023

Choose a reason for hiding this comment

wy65701436 Mar 1, 2023

Choose a reason for hiding this comment

wy65701436 left a comment

Choose a reason for hiding this comment

codecov bot commented Feb 21, 2023 •

edited

Loading