Let Kaniko E2E test work with a KO_DOCKER_REPO env #4401

afrittoli · 2021-12-01T15:45:58Z

Changes

If test runner set a KO_DOCKER_REPO variable, use it, so we run the tests
against an external container registry. If not set, the kaniko tests
will spin up its own local registry. Bring back the GCP secret support
so this works in the existing CI, but also supports the case where
KO_DOCKER_REPO points to a container registry where no secret required,
like in the kind based tests.

Add a "test_setup" function to the e2e script, which is picked up after
the setup of the cluster, which provisions a GCP service account to be
used by tests via the GCP_SERVICE_ACCOUNT_KEY_PATH env variable.

Since the local registry runs on HTTP (not HTTPS), the local registry
approach does not work for the helm test as the kubelet tries to use
HTTPS. If KO_DOCKER_REPO is not specified we either fail or skip the
test (no change in behaviour) depending on the value of missingKoFatal

This setup makes it easier to run E2E tests in different environments,
where a registry may or may not be available.

Signed-off-by: Andrea Frittoli andrea.frittoli@uk.ibm.com
/kind misc

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

[-] Docs included if any changes are user facing
Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Release notes block below has been filled in or deleted (only if no user facing changes)

Release Notes

NONE

tekton-robot · 2021-12-01T15:46:07Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign afrittoli
You can assign the PR to them by writing /assign @afrittoli in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

afrittoli · 2021-12-01T15:46:16Z

/cc @vdemeester

vdemeester

Sounds ok, I just need to validate that it works fine on our OpenShift CI 😅

test/ko_test.go

afrittoli · 2021-12-02T23:27:51Z

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-integration-tests

afrittoli · 2021-12-03T12:21:03Z

/test pull-tekton-pipeline-unit-tests

If test runner set a KO_DOCKER_REPO variable, use it, so we run the tests against an external container registry. If not set, the kaniko tests will spin up its own local registry. Bring back the GCP secret support so this works in the existing CI, but also supports the case where KO_DOCKER_REPO points to a container registry where no secret required, like in the kind based tests. Add a "test_setup" function to the e2e script, which is picked up after the setup of the cluster, which provisions a GCP service account to be used by tests via the GCP_SERVICE_ACCOUNT_KEY_PATH env variable. Since the local registry runs on HTTP (not HTTPS), the local registry approach does not work for the helm test as the kubelet tries to use HTTPS. If KO_DOCKER_REPO is not specified we either fail or skip the test (no change in behaviour) depending on the value of missingKoFatal This setup makes it easier to run E2E tests in different environments, where a registry may or may not be available. Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>

tekton-robot · 2021-12-03T13:05:26Z

@afrittoli: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-tekton-pipeline-integration-tests	`cdec71d`	link	`/test pull-tekton-pipeline-integration-tests`
pull-tekton-pipeline-alpha-integration-tests	`cdec71d`	link	`/test pull-tekton-pipeline-alpha-integration-tests`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

tekton-robot · 2021-12-03T13:05:26Z

@afrittoli: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-tekton-pipeline-integration-tests	`cdec71d`	link	`/test pull-tekton-pipeline-integration-tests`
pull-tekton-pipeline-alpha-integration-tests	`cdec71d`	link	`/test pull-tekton-pipeline-alpha-integration-tests`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

afrittoli · 2021-12-03T13:09:22Z

Creating a service account with the right permissions to run the tests does not seem to work:

=================================
==== SETTING UP TEST CLUSTER ====
=================================
- Project is tekton-prow-8
- Cluster is gke_tekton-prow-8_us-central1_tpipeline-e2e-cls1466744752903819268
- User is prow-account@tekton-releases.iam.gserviceaccount.com
- Docker is gcr.io/tekton-prow-8/tpipeline-e2e-img
Updated property [core/project].
WARNING: You do not appear to have access to project [tekton-prow-8] or it does not exist.
Created service account [tekton-prow-8].
ERROR: (gcloud.projects.add-iam-policy-binding) User [prow-account@tekton-releases.iam.gserviceaccount.com] does not have permission to access projects instance [tekton-prow-8:getIamPolicy] (or it may not exist): Cloud Resource Manager API has not been used in project 574248271492 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=574248271492 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
- '@type': type.googleapis.com/google.rpc.Help
  links:
  - description: Google developers console API activation
    url: https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=574248271492
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: googleapis.com
  metadata:
    consumer: projects/574248271492
    service: cloudresourcemanager.googleapis.com
  reason: SERVICE_DISABLED
ERROR: (gcloud.iam.service-accounts.keys.create) FAILED_PRECONDITION: Key creation is not allowed on this service account.
- '@type': type.googleapis.com/google.rpc.PreconditionFailure
  violations:
  - description: Key creation is not allowed on this service account.
    subject: projects/tekton-prow-8/serviceAccounts/tekton-prow-8@tekton-prow-8.iam.gserviceaccount.com?configvalue=tekton-prow-8%40tekton-prow-8.iam.gserviceaccount.com
    type: constraints/iam.disableServiceAccountKeyCreation
================================

afrittoli · 2021-12-03T18:16:42Z

/hold

nikhil-thomas · 2021-12-06T00:04:02Z

Sounds ok, I just need to validate that it works fine on our OpenShift CI 😅

@vdemeester I think this would be ok as:

at present, we use a separate e2e test script (but i agree, this is not ideal and we should start using this script as soon as we can).
we will not touch the ìnitialize part any way for tests on OpenShift. The switches in PR: Add some switches to the e2e script ⚙️ #4400 will make this easy to disable while testing on OpenShift
we run tests with github.com/tektoncd/pipeline/test/v1alpha1.missingKoFatal=false here

cc @barthy1

barthy1 · 2021-12-06T15:28:22Z

just fyi I've tested the proposed code with s390x nightly pipeline test suite (pure k8s, KO_DOCKER_REPO is specified and remote, registry is HTTP). Everything works perfectly.

tekton-robot · 2022-03-06T15:52:29Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

tekton-robot · 2022-03-31T12:13:31Z

@afrittoli: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2022-04-30T12:50:50Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

tekton-robot · 2022-05-30T13:45:16Z

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

tekton-robot · 2022-05-30T13:45:18Z

@tekton-robot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot added release-note-none Denotes a PR that doesnt merit a release note. kind/misc Categorizes issue or PR as a miscellaneuous one. labels Dec 1, 2021

tekton-robot requested review from dibyom and jerop December 1, 2021 15:46

tekton-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 1, 2021

tekton-robot requested a review from vdemeester December 1, 2021 15:46

afrittoli force-pushed the env_var_registry branch 2 times, most recently from a766e09 to 1a64c95 Compare December 1, 2021 16:19

vdemeester reviewed Dec 1, 2021

View reviewed changes

test/ko_test.go Show resolved Hide resolved

afrittoli force-pushed the env_var_registry branch 4 times, most recently from 8ee2448 to 88681e9 Compare December 2, 2021 11:25

tekton-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 2, 2021

afrittoli force-pushed the env_var_registry branch 5 times, most recently from d5247e3 to 6c0e090 Compare December 2, 2021 23:26

afrittoli force-pushed the env_var_registry branch from 6c0e090 to 1b6305c Compare December 3, 2021 12:09

afrittoli force-pushed the env_var_registry branch from 1b6305c to cdec71d Compare December 3, 2021 12:22

tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 3, 2021

tekton-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 6, 2022

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 31, 2022

tekton-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 30, 2022

tekton-robot closed this May 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Let Kaniko E2E test work with a KO_DOCKER_REPO env #4401

Let Kaniko E2E test work with a KO_DOCKER_REPO env #4401

afrittoli commented Dec 1, 2021 •

edited

Loading

tekton-robot commented Dec 1, 2021

afrittoli commented Dec 1, 2021

vdemeester left a comment

afrittoli commented Dec 2, 2021

afrittoli commented Dec 3, 2021

tekton-robot commented Dec 3, 2021

tekton-robot commented Dec 3, 2021

afrittoli commented Dec 3, 2021

afrittoli commented Dec 3, 2021

nikhil-thomas commented Dec 6, 2021 •

edited

Loading

barthy1 commented Dec 6, 2021

tekton-robot commented Mar 6, 2022

tekton-robot commented Mar 31, 2022

tekton-robot commented Apr 30, 2022

tekton-robot commented May 30, 2022

tekton-robot commented May 30, 2022

Let Kaniko E2E test work with a KO_DOCKER_REPO env #4401

Let Kaniko E2E test work with a KO_DOCKER_REPO env #4401

Conversation

afrittoli commented Dec 1, 2021 • edited Loading

Changes

Submitter Checklist

Release Notes

tekton-robot commented Dec 1, 2021

afrittoli commented Dec 1, 2021

vdemeester left a comment

Choose a reason for hiding this comment

afrittoli commented Dec 2, 2021

afrittoli commented Dec 3, 2021

tekton-robot commented Dec 3, 2021

tekton-robot commented Dec 3, 2021

afrittoli commented Dec 3, 2021

afrittoli commented Dec 3, 2021

nikhil-thomas commented Dec 6, 2021 • edited Loading

barthy1 commented Dec 6, 2021

tekton-robot commented Mar 6, 2022

tekton-robot commented Mar 31, 2022

tekton-robot commented Apr 30, 2022

tekton-robot commented May 30, 2022

tekton-robot commented May 30, 2022

afrittoli commented Dec 1, 2021 •

edited

Loading

nikhil-thomas commented Dec 6, 2021 •

edited

Loading