Issue 1548

[omerap12]

No description provided.

[k8s-ci-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: omerap12
Once this PR has been reviewed and has the lgtm label, please assign arangogutierrez for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[netlify]

✅ Deploy Preview for kubernetes-sigs-nfd ready!

Name	Link
🔨 Latest commit	bb435df
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-nfd/deploys/66ade3e41e6650000882e49f
😎 Deploy Preview	https://deploy-preview-1814--kubernetes-sigs-nfd.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[marquiz]

Thanks @omerap12 for starting to work on this! There was one misconception regarding my description in #1548 (this PR seems to address that, i.e. nfd-master, instead of #1806).

In addition to monitoring the (internal) metrics server we could watch the goroutines of node updater pool (and set health to false if they exit, for whatever reaseon). Then we could think about using SetWatchErrorHandler.() to monitor the informers in the nfd api controller.

[marquiz]

Oh, sorry for the ambiguity. I wasn't thinking about kubernetes metrics-server but the metrics http server inside nfd-master itself that is started with utils.CreateMetricsServer(). We could e.g. just set a health flag to false if the goroutine started there exits.

[omerap12]

Hi @marquiz, thanks for the review! I’ll make the necessary code adjustments and have updated the title as well, lol. I’ll keep you posted.

[omerap12]

Hey @marquiz,

Here's a summary of what I've done so far:

• I created a custom health server within the NFD master. This server has two boolean fields that indicate the health status of the internal metric server and whether any of the major nodeUpdaterPool functions have panicked.
• The custom health server periodically checks these fields. If either of them is false, the pod's liveness status is set to false.
  Looking forward to more instructions! :)

[omerap12]

@marquiz , Friendly ping :)

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.