Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow PipelineResource implementations to modify the entire Pod spec. #1345

Merged
merged 1 commit into from
Sep 24, 2019

Conversation

dlorenc
Copy link
Contributor

@dlorenc dlorenc commented Sep 23, 2019

Changes

This change simplifies the interface by removing the GetUpload/Download container
and volume methods and replaces it with a more generic "modifier" system.

This can be cleaned up a bit more still, and is intended for an early review at this point.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

See the contribution guide for more details.

Reviewer Notes

If API
changes

are included, additive
changes

must be approved by at least two
OWNERS and backwards
incompatible
changes

must be approved by more than 50% of the
OWNERS
, and they must
first be added in a backwards compatible
way
.

@googlebot googlebot added the cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit label Sep 23, 2019
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dlorenc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot requested review from afrittoli and a user September 23, 2019 15:47
@tekton-robot tekton-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 23, 2019
@dlorenc
Copy link
Contributor Author

dlorenc commented Sep 23, 2019

This is intended to replace #1215

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/build_gcs_resource.go 79.4% 86.8% 7.4
pkg/apis/pipeline/v1alpha1/cloud_event_resource.go 78.9% 88.2% 9.3
pkg/apis/pipeline/v1alpha1/cluster_resource.go 71.8% 76.9% 5.1
pkg/apis/pipeline/v1alpha1/gcs_resource.go 81.8% 88.4% 6.6
pkg/apis/pipeline/v1alpha1/git_resource.go 70.0% 80.0% 10.0
pkg/apis/pipeline/v1alpha1/image_resource.go 50.0% 56.2% 6.2
pkg/apis/pipeline/v1alpha1/pull_request_resource.go 77.3% 87.5% 10.2
pkg/apis/pipeline/v1alpha1/resource_types.go 0.0% 11.1% 11.1
pkg/reconciler/taskrun/resources/input_resources.go 91.7% 92.6% 0.9
pkg/reconciler/taskrun/resources/output_resource.go 90.5% 92.1% 1.6

pkg/apis/pipeline/v1alpha1/build_gcs_resource.go Outdated Show resolved Hide resolved
pkg/apis/pipeline/v1alpha1/resource_types.go Outdated Show resolved Hide resolved
pkg/apis/pipeline/v1alpha1/resource_types.go Outdated Show resolved Hide resolved
pkg/apis/pipeline/v1alpha1/gcs_resource.go Outdated Show resolved Hide resolved
@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/build_gcs_resource.go 79.4% 86.8% 7.4
pkg/apis/pipeline/v1alpha1/cloud_event_resource.go 78.9% 88.2% 9.3
pkg/apis/pipeline/v1alpha1/cluster_resource.go 71.8% 76.9% 5.1
pkg/apis/pipeline/v1alpha1/gcs_resource.go 81.8% 88.4% 6.6
pkg/apis/pipeline/v1alpha1/git_resource.go 70.0% 80.0% 10.0
pkg/apis/pipeline/v1alpha1/image_resource.go 50.0% 56.2% 6.2
pkg/apis/pipeline/v1alpha1/pull_request_resource.go 77.3% 87.5% 10.2
pkg/apis/pipeline/v1alpha1/resource_types.go 0.0% 11.1% 11.1
pkg/reconciler/taskrun/resources/input_resources.go 91.7% 92.6% 0.9
pkg/reconciler/taskrun/resources/output_resource.go 90.5% 92.1% 1.6

@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/build_gcs_resource.go 79.4% 86.1% 6.7
pkg/apis/pipeline/v1alpha1/cloud_event_resource.go 78.9% 88.2% 9.3
pkg/apis/pipeline/v1alpha1/cluster_resource.go 71.8% 76.3% 4.5
pkg/apis/pipeline/v1alpha1/gcs_resource.go 81.8% 87.2% 5.4
pkg/apis/pipeline/v1alpha1/git_resource.go 70.0% 78.9% 8.9
pkg/apis/pipeline/v1alpha1/image_resource.go 50.0% 56.2% 6.2
pkg/apis/pipeline/v1alpha1/pull_request_resource.go 77.3% 85.0% 7.7
pkg/apis/pipeline/v1alpha1/resource_types.go 0.0% 11.8% 11.8
pkg/reconciler/taskrun/resources/input_resources.go 91.7% 92.6% 0.9
pkg/reconciler/taskrun/resources/output_resource.go 90.5% 92.1% 1.6

This change simplifies the interface by removing the GetUpload/Download container
and volume methods and replaces it with a more generic "modifier" system.

This can be cleaned up a bit more still, and is intended for an early review at this point.
@tekton-robot
Copy link
Collaborator

The following is the coverage report on pkg/.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/pipeline/v1alpha1/build_gcs_resource.go 79.4% 86.1% 6.7
pkg/apis/pipeline/v1alpha1/cloud_event_resource.go 78.9% 88.2% 9.3
pkg/apis/pipeline/v1alpha1/cluster_resource.go 71.8% 76.3% 4.5
pkg/apis/pipeline/v1alpha1/gcs_resource.go 81.8% 87.2% 5.4
pkg/apis/pipeline/v1alpha1/git_resource.go 70.0% 78.9% 8.9
pkg/apis/pipeline/v1alpha1/image_resource.go 50.0% 56.2% 6.2
pkg/apis/pipeline/v1alpha1/pull_request_resource.go 77.3% 85.0% 7.7
pkg/apis/pipeline/v1alpha1/resource_types.go 0.0% 11.8% 11.8
pkg/reconciler/taskrun/resources/input_resources.go 91.7% 92.6% 0.9
pkg/reconciler/taskrun/resources/output_resource.go 90.5% 92.1% 1.6

@dlorenc
Copy link
Contributor Author

dlorenc commented Sep 24, 2019

cc @bobcatfish this needs style cleanups - comment fixes and other minor stuff - but can you take another look at the general shape?

@bobcatfish
Copy link
Collaborator

Interesting, yeah @dlorenc i think this is pretty close now the the interface we'd have in the extensible version.

Interesting that you went the way of having only one InternalTaskModifier that looks like:

func (tm *InternalTaskModifier) GetStepsToPrepend() []Step {
	return tm.StepsToPrepend
}

func (tm *InternalTaskModifier) GetStepsToAppend() []Step {
	return tm.StepsToAppend
}

I think I would have had many TaskModifiers that returned different sets of steps but I think now we'r e just in "i would have done it differently" territory.

Looks good to me!

I think this one could probably use a detailed commit message about how this relates to the extensibility model and some of the different iterations we want through fo sho.

@bobcatfish
Copy link
Collaborator

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 24, 2019
@bobcatfish
Copy link
Collaborator

lookit all that coverage goin up

@bobcatfish
Copy link
Collaborator

image

@tekton-robot tekton-robot merged commit e83fb4c into tektoncd:master Sep 24, 2019
@bobcatfish
Copy link
Collaborator

I think this one could probably use a detailed commit message about how this relates to the extensibility model and some of the different iterations we want through fo sho.

... and then i merged it so we couldn't.

Hey @dlorenc I know it's too late, but would you be open to doing a follow up PR that adds docstrings to the new functions & interface?

// resource's data.
func (s *BuildGCSResource) GetDownloadSteps(sourcePath string) ([]Step, error) {
// GetInputTaskModifier returns a TaskModifier that prepends a step to a Task to fetch the archive or manifest.
func (s *BuildGCSResource) GetInputTaskModifier(ts *TaskSpec, sourcePath string) (TaskModifier, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im a bit iffy about ts being in this interface, what do you think @dlorenc ? From what I can tell it seems to be for the sake of the getStorageVolumeSpec function, and it looks like that's being used for:

func getStorageVolumeSpec(s PipelineStorageResourceInterface, spec *TaskSpec) ([]corev1.Volume, error) {
	var storageVol []corev1.Volume
	mountedSecrets := map[string]string{}

	for _, volume := range spec.Volumes {
		mountedSecrets[volume.Name] = ""
	}
...
}

I'm having a hard time understanding what that's actually doing and why it wants access to the TaskSpec to do it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it's needed to prevent adding a volume to a task with a name that already exists. I'm not sure if that is actually a realistic scenario, or being overly defensive.

At a minimum, I'll change this to just pass ts as an object rather than a pointer so it can't be mutated.

@dlorenc
Copy link
Contributor Author

dlorenc commented Sep 25, 2019

Hey @dlorenc I know it's too late, but would you be open to doing a follow up PR that adds docstrings to the new functions & interface?

Nope never!!! Yeah I should have put a /hold on this, sorry :(

@dlorenc dlorenc deleted the modifiers branch September 25, 2019 14:37
dlorenc added a commit to dlorenc/build-pipeline that referenced this pull request Sep 25, 2019
…erged.

This adds missing docstrings to the new interface methods and changes the way
TaskSpec is passed so it can't be mutated.
tekton-robot pushed a commit that referenced this pull request Sep 25, 2019
This adds missing docstrings to the new interface methods and changes the way
TaskSpec is passed so it can't be mutated.
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Trying to make the CLA bot happy with ppl from different companies work on one commit lgtm Indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants