Allow Task sidecars to be specified in Pipeline/PipelineRun

[michaelsauter]

Feature request

At the moment, sidecars are defined as part of a Task. I don't see a way to add a sidecar to a task from the Pipeline/PipelineRun definition. However, I think this would be really helpful.

Use case

Our use case is this: we (an enterprise platform) want to provide a set of Tekton tasks (https://github.com/opendevstack/ods-pipeline) in the cluster for our users to consume. The goal is to provide a streamlined way to build & deploy applications. One challenge is to hit a good balance between a common standard across the platform and enough flexibility to not be a pain for consumers. If sidecars could be specified from a pipeline definition, it would help with this balance.

As an example, we have a task that builds Go applications. This task also runs linters, checks format and executes the test suite. Sometimes, those tests will want to use some dependencies (e.g. a database or external service) to test proper integration. If the task definition is the only place to specify sidecars, this doesn't really work as the task authors (the platform) cannot know which sidecars are needed. The ideal place to specify which sidecar to use (image, environment variables, resource constraints, ...) is the pipeline(run), which is authored by the consumers, not the task.

Within the current design, the only option seems to be to create "sidecar variants" for our tasks, and allow to exchange the image via parameter (as implemented in opendevstack/ods-pipeline#210). However that has serious limitations as e.g. env variables cannot be changed, resource constraints cannot be set and one is limited to one sidecar only.

Additional Information

• The situation described in the use case also motivates my comments on being able to adjust task resource constraints via parameters (see Handling parameter interpolation in fields not designed for it #1530).
• Note this is different from Pipeline level sidecar #2973, as that discusses sidecars that run alongside a pipeline. My request here does not change how sidecars work (their lifecycle), "just" where they are specified.

[michaelsauter]

ping

Any comment on this? Or is this the wrong place to suggest such a feature? 😄

[michaelsauter]

@vdemeester @bobcatfish maybe you can comment on this as you discussed on the linked issue #2973? I would love to get some idea on whether this is something you would consider contributions for, or whether this is impossible, or .... anything really :)

[michaelsauter]

Alright, one more attempt :) Any comment on this? Let me know if you need more info, or if I should bring this up in another place.

[vdemeester]

@michaelsauter sorry 😓
So, as of today, your use case could be achieve by an early "in parallel" Task that provision the dependency you need, and a finally task that assure to bring it down, no matter the results, am I right ? Although, you might also want the option to be able to "parametrize" the task you run (in order to bring up and down a different dependency, depending on the paramater, right ?)

[afrittoli]

@michaelsauter sorry 😓 So, as of today, your use case could be achieve by an early "in parallel" Task that provision the dependency you need, and a finally task that assure to bring it down, no matter the results, am I right ? Although, you might also want the option to be able to "parametrize" the task you run (in order to bring up and down a different dependency, depending on the paramater, right ?)

As per @michaelsauter comment:

Note this is different from Pipeline level sidecar #2973, as that discusses sidecars that run alongside a pipeline. My request here does not change how sidecars work (their lifecycle), "just" where they are specified.

I think this is a different use case. Sidecars would only live for the duration of a Task, but they would be specified at Pipeline level, to move the decision about which sidecars are needed from Task authoring time to Pipeline authoring time.

In my mind this would look like:

spec:
  tasks:
    - name: first-task
      taskRef:
        name: a-task-without-sidecars
      sidecars:
        - image: busybox
          name: hello-sidecar
          script: |
            echo 'Hello from sidecar!'

[vdemeester]

oh I see, thanks @afrittoli .. I didn't read it as such, but now I think I understand 😅

[michaelsauter]

@vdemeester @afrittoli Thanks for picking this up!

And yes, the example you are giving @afrittoli is exactly what I have in mind with this feature idea.

[michaelsauter]

We just hit this use case yesterday again. Any update on whether this is feasible and desired?

I'd be happy to take a stab at this but would need some pointers where to start :)

[nikhil-thomas]

@michaelsauter May be a custom mutating webhook is what this case needs. For example, in tektoncd/operator we have a webhook which injects some environment variables into TaskRun Pods.

• https://github.com/tektoncd/operator/blob/main/docs/Proxy.md#proxy-support
• https://github.com/tektoncd/operator/blob/main/cmd/openshift/proxy-webhook/main.go

In short, a quick way to improve this situation might be to deploy the custom webhook beside the TektonPipelines installation on your cluster 🧑‍💻. To consider supporting such a webhook in the operator or in pipelines we will have to define the problem in a wider scope (eg: enduser specific TaskRun extensions 😇 )

[michaelsauter]

@nikhil-thomas Thanks for this hint! This is indeed something we could explore.

However I believe our use case should not require a mutating webhook, nor are we asking for full-blown enduser specific TaskRun extensions. We simply want to allow specifying sidecars when authoring pipelines instead of only when authoring tasks. As stated before I think this is a crucial feature to make tasks reusable.

Since there seems to be no quick solution to this, we are planning to sidestep the issue altogether for now. We plan to get rid of cluster tasks which are centrally provided, and instead provide the tasks everyone should be using as a Helm chart via Git, to be installed in their own namespace. The Helm chart allows templating, which allows users to specify the sidecars they want to use while still largely authoring the tasks centrally for them. This solution also "fixes" the other issue linked above that pipeline authors cannot modify resource constraints (now they can via Helm templating).

Maybe this "workaround" helps someone else as well ...

I would still appreciate a "proper" solution within Tekton though!

[nikhil-thomas]

cc @siamaksade

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[michaelsauter]

Any chance this can move forward as a feature request? The workaround isn't great.

[vdemeester]

/remove-lifecycle stale

[michaelsauter]

With the availability of remote task resolution (via Git or ArtifactHub), this issue will become more pressing for us in the future.

So far our workaround is that each user defines the task in the Helm chart and can therefore add the sidecar at "Helm chart authoring time". We would like to make things easier for our users so that they do not need to maintain the task definitions in a Helm chart anymore, and can simply reference tasks from a central Git repo / Artefact Hub. However, then they loose the ability to define the sidecars as that feature is not available at "pipeline run authoring time".

Is this feature something on your radar?

[lbernick]

I don't know of any plans to implement this feature at the moment, but it sounds like it would add a lot of value for your team. Would you consider proposing a TEP for this?

[michaelsauter]

@lbernick Sure: tektoncd/community#877. Let me know how I can help to move this forward.

[lbernick]

Thanks Michael! We will see if anyone is able to review the proposal at our regular monday working group. Other great ways to get eyes on the proposal are to post on the #tep channel on the tekton slack or highlight your proposal on the tekton-dev or tekton-users mailing list. This doc has a bit more detail on the TEP process.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.