Support alternative implementations of a specific Task

[afrittoli]

Description and motivation

A Tekton Task has an interface that is defined by the collection of its inputs (resources + params) and outputs (resources). It should be possible to replace a Task in a Pipeline with a different one as long as their inputs and outputs are compatible.

Tekton however does not really provide any way to switch the implementation of a Task in a Pipeline dynamically or without significantly altering the Pipeline definition.

The feature proposed here introduces a way to dynamically select a compatible Task implementation at run time based on inputs or conditions. For instance a release pipeline could use a different Task to tag images depending on the container registry it's running against.

State of the Art

There are ways this could be partially achieved today using Conditions.
A number of Tasks, each with a Condition could be defined in the Pipeline. However there is no way to enforce the conditions being mutually exclusive, so it may be that multiple Tasks are executed at the end. Another issue is with the output resources. Any Task that depend on the outputs of the Tasks with conditions, would not have a way to specify which Task its inputs are coming from.

Use Cases

• Build a docker image with a different tool
• Deploy with alternate strategies
• Different implementations of a Task to work with different cloud APIs
• Interface with alternate implementations of a service

Possible Implementations

One way to handle this could be to allow for the Task name in the pipeline spec to be dynamic, i.e. only defined at runtime. It could be built out of parameters or resources.
Alternatively a new kind of MultiTask could be implemented, that in its definition references all the possible Task implementations and the condition for their selection, in a way that enforces mutual exclusion. Consumers of the MultiTask outputs would reference to the MultiTask name as source of their inputs.
It might be possible to introduce AbstractTasks, i.e. Tasks with interface spec but no actual steps in them, but I fear that would be to rigid, an suffer similar type of issue as abstract classes in object orient programming.

Notes

This may sound similar to #215 - at least from the issue name - but it's a different feature - even if there may be points of overlap.

[skaegi]

Maybe a little low-tech ... but a Task could have a step that internally wraps a dynamically named (or even generated) Task or Pipeline, run it and then copy it's inputs and outputs (and maybe even logs) back out to the containing TaskRun. It would definitely be a bit of work to support traceability but I think still possible with clever labelling.

[afrittoli]

Maybe a little low-tech ... but a Task could have a step that internally wraps a dynamically named (or even generated) Task or Pipeline, run it and then copy it's inputs and outputs (and maybe even logs) back out to the containing TaskRun. It would definitely be a bit of work to support traceability but I think still possible with clever labeling.

Nice idea, that would move the granularity back; but without the benefit of running on the same node. I think there are multiple possible ways to implement this - but whatever the solution I'd like it to be very much declarative - so that its clear from the pipeline/task/step specs what we expect to happen.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[afrittoli]

I think this is still a valid use case.
This design doc goes in the direction of fixing this: https://docs.google.com/document/d/1G2QbpiMUHSs4LOqcNaIRswcdvoy8n7XuhTV8tXdcE7A/edit#
@sbwsg @bobcatfish

[bobcatfish]

@afrittoli I still don't totally understand this use case - this means that someone has written a Pipeline and they want to use the entire Pipeline as-is, but they want to swap out individual Tasks. I guess I don't totally understand why it's not worth writing a different Pipeline in that case.

The Task specialization proposal COULD include this, but for the question of "when do you want this specialization", it seemed like "when running the Pipeline" was the one point at which no one really needed to do the specialization.

Do you have some use cases today (maybe for dogfooding?) where you need this?

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[vdemeester]

/remove-lifecycle stale

[bobcatfish]

I wonder if this is related to some of the discussions we're having in #2298, specifically about being able to make changes to a Task without having to change the Pipeline (e.g. if you use an OCI reference in a Pipeline and want to try out some changes to a Task before it's published)

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.