Completed pipelines turn into PipelineRunTimeout

[brovoca]

Expected Behavior

A completed pipeline should keep the "completed" status and not do spooky things.

Actual Behavior

Already completed pipelines suddenly change status into "PipelineRunTimeout"

~ >>> date --iso-8601=s && kubectl -n tekton-neo get pipelineruns
2023-08-10T11:07:08+02:00
NAME                         SUCCEEDED   REASON               STARTTIME   COMPLETIONTIME
pc-markdown-r8f4z            True        Succeeded            111m        111m
pc-markdown-z7hcq            True        Succeeded            2m32s       2m
pc-review-controller-5lzpk   True        Completed            111m        110m
pc-review-controller-q4pt8   True        Completed            2m32s       103s

~ >>> date --iso-8601=s && kubectl -n tekton-neo get pipelineruns
2023-08-10T14:57:41+02:00
NAME                         SUCCEEDED   REASON               STARTTIME   COMPLETIONTIME
pc-markdown-r8f4z            False       PipelineRunTimeout   5h42m       3h34m
pc-markdown-z7hcq            True        Succeeded            3h53m       3h52m
pc-review-controller-5lzpk   False       PipelineRunTimeout   5h42m       3h34m
pc-review-controller-q4pt8   True        Completed            3h53m       3h52m

Steps to Reproduce the Problem

1. Install Tekton Operator v0.67.0
2. Trigger a pipeline
3. Wait a few hours

Additional Info

• Kubernetes version:

  Output of kubectl version:

  WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
  Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.4", GitCommit:"fa3d7990104d7c1f16943a67f11b154b71f6a132", GitTreeState:"clean", BuildDate:"2023-07-19T12:14:48Z", GoVersion:"go1.20.6", Compiler:"gc", Platform:"linux/amd64"}
  Kustomize Version: v5.0.1
  Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"f18584a06fc476806da2e340e2eed960659871e8", GitTreeState:"clean", BuildDate:"2023-06-12T18:45:20Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"}
  

• Tekton Pipeline version:

  Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

  v0.47.0
  

Yesterday we upgraded from Tekton Operator v0.66.0 to v0.67.0, and pretty much all the existing old pipelines changed status. I deleted all of them and triggered new ones today morning. Now, a few hours later, 2 of the pipelines that were triggered at the same time (by the same event) have changed status.

[brovoca]

I have opened the same issue at tektoncd/pipeline#7041 as I don't know where would be best to track it.

[vdemeester]

Definitely in pipeline, so I am going to go ahead and close this one 🙃

[brovoca]

@vdemeester I think the ball is back in the operator court, because this bug was fixed in Tekton Pipelines v0.47.1 (tektoncd/pipeline#6812). Please release a patch for the operator that bumps the pipelines version to v0.47.1.

[brovoca]

I found #1543, so I'll bump that issue instead.