kube: load KubeConfig (token) from FS on every reconcile

[hiddeco]

4371610 is a partial cherry-pick of commit ae4f499, including
changes around kube. This to include some of the changes around the
construction of the ConfigFlags RESTClientGetter, as an attempt to
solve token refresh issues.

As icing on the cake, 93bd5ed always retrieves the current token from the FS which fixes #479 and fluxcd/flux2#2074.

[pjbgf]

LGTM

🔝 🔝!

[stefanprodan]

I think we need to change the title and the description of this PR to be about token refresh. The code looks Ok to me.

[hiddeco]

@stefanprodan I changed the PR title to make mention of it.

[cgchinmay]

Hi, could you elaborate on how "always retrieves the current token from the FS" fixes the token refresh issue ?

[hiddeco]

My understanding is that by initializing it from zero every time, it reads the latest runtime values and thereby is able to obtain a correct token.

Note that this part of the code is very Helm specific, and the further controller (and controller-runtime manager) properly update their token automagically.

[michael12312]

hi @hiddeco

Per the doc from K8s mentioned:
The following Kubernetes client SDKs refresh tokens automatically within the required time frame:
Go v0.15.7 and later

I see helm-controller already use the version of k8s.io/client-go v0.23.6.

Per my understanding the expired token issue should not exists, can you explain why you put this fixing? Why load KubeConfig (token) from FS on every reconcile?

[hiddeco]

As said before in #480 (comment).

This patch is purely required because of how we integrate with Helm, and the controller-runtime client itself has been updating the token as expected. This process does however not update the token in the loaded REST config, from which we used to derive our client for Helm. It should not be taken as a further example on how to solve this for your project, as the context around why and how we require this does likely not apply to you.

[cgchinmay]

As said before in #480 (comment).

This patch is purely required because of how we integrate with Helm, and the controller-runtime client itself has been updating the token as expected. This process does however not update the token in the loaded REST config, from which we used to derive our client for Helm. It should not be taken as a further example on how to solve this for your project, as the context around why and how we require this does likely not apply to you.

Thanks for the explanation. I had the same thought as @michael12312 to ask the question. Also, I noticed the same issue with Multus even though it uses Go client > v0.15.7. So was trying to understand if there's a common pattern here.

[hiddeco]

Think you need to look in the area of https://github.com/k8snetworkplumbingwg/multus-cni/blob/706de7c2c6a7d86401e1b0650bc050e4bdee984b/pkg/k8sclient/k8sclient.go#L384-L412 in that case. Although on a first 10 second glance, https://github.com/k8snetworkplumbingwg/multus-cni/blob/706de7c2c6a7d86401e1b0650bc050e4bdee984b/pkg/k8sclient/k8sclient.go#L405 should be doing the right thing and pretty much equals to what we are doing here.

[michael12312]

hi @hiddeco

I saw you commit this change 4 days ago, want to ask: when do you plan to create 1 release for this change? Any tag we can refer to?

[hiddeco]

@michael12312 KubeCon and finalizing of source-controller changes got in the way for a bit. #491 is now pending.