Allign k8s configuration settings #29908

ChrsMark · 2022-01-19T09:28:14Z

What does this PR do?

This PR adds namespace and moves host to node settings under the kubernetes module.

Why is it important?

In order to have alignment between the metadata configurations.

Related issues

Closes #29132

Signed-off-by: chrismark <chrismarkou92@gmail.com>

elasticmachine · 2022-01-19T09:28:34Z

Pinging @elastic/integrations (Team:Integrations)

mergify · 2022-01-19T09:39:41Z

This pull request does not have a backport label. Could you fix it @ChrsMark? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2022-01-19T11:41:45Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-01-20T09:34:29.325+0000
Duration: 118 min 14 sec
Commit: b0ff4c5

Test stats 🧪

Test	Results
Failed	0
Passed	47381
Skipped	4155
Total	51536

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

ChrsMark · 2022-01-19T11:44:23Z

/package

ChrsMark · 2022-01-19T11:45:24Z

Note to myself: add changelog entry before merging

Signed-off-by: chrismark <chrismarkou92@gmail.com>

tetianakravchenko · 2022-01-20T08:44:17Z

libbeat/processors/add_kubernetes_metadata/config.go

@@ -29,7 +29,7 @@ import (
 type kubeAnnotatorConfig struct {
 	KubeConfig        string                       `config:"kube_config"`
 	KubeClientOptions kubernetes.KubeClientOptions `config:"kube_client_options"`
-	Host              string                       `config:"host"`
+	Node              string                       `config:"node"`


should it first be deprecated instead of just dropping old setting name (the same as for autodiscover)? should it be considered a breaking change?
not sure what is the process here

Actually it should be done like this the same time we did for autodiscover and processor. However we can claim that the change goes all the way with the k8s features and ship it under 8.1 as a breaking change which already had a notice. In any case this is was not set in any default manifests or configs so I don't forsee any huge impact.

tetianakravchenko · 2022-01-20T08:44:53Z

metricbeat/module/kubernetes/util/kubernetes.go

@@ -51,12 +51,13 @@ type kubernetesConfig struct {
 	KubeConfig        string                       `config:"kube_config"`
 	KubeClientOptions kubernetes.KubeClientOptions `config:"kube_client_options"`

-	Host       string        `config:"host"`
+	Node       string        `config:"node"`


the same question here

tetianakravchenko · 2022-01-20T08:59:23Z

metricbeat/module/kubernetes/util/kubernetes.go

@@ -262,14 +263,15 @@ func getResourceMetadataWatchers(config *kubernetesConfig, resource kubernetes.R

 	options := kubernetes.WatchOptions{
 		SyncTimeout: config.SyncPeriod,
+		Namespace:   config.Namespace,


should be added a check if config.Namespace is defined?

smth like here -

beats/libbeat/autodiscover/providers/kubernetes/pod.go

Lines 98 to 100 in fb2f53c

if config.Namespace != "" {

options.Namespace = config.Namespace

}

Not sure if it really matters but I'm adding the check anyways. I would expect an empty value to have no impact.

interesting why this check is added only for options that are used for nodeWatcher, but not for watcher (in autodiscover -

beats/libbeat/autodiscover/providers/kubernetes/pod.go

Lines 98 to 100 in fb2f53c

if config.Namespace != "" {

options.Namespace = config.Namespace

}

), maybe it is just a redundancy there. Anyway please let me know in case you figure it out. I am fine to keep it here as it is now

let me try to remove it and see what happens.

tetianakravchenko · 2022-01-20T09:00:50Z

metricbeat/modules.d/kubernetes.yml.disabled

  # If kube_config is not set, KUBECONFIG environment variable will be checked
  # and if not present it will fall back to InCluster
+  #kube_config: ~/.kube/config


this setting is already defined in file - line 41, should be removed then

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark · 2022-01-20T12:28:52Z

/package

ChrsMark · 2022-01-24T11:59:05Z

@tetianakravchenko I think we are fine to remove the check for the namespace even e2e tests for beats passed. Could you have a final review on this?

tetianakravchenko

LGTM!

Allign k8s configuration settings

b518ce0

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark self-assigned this Jan 19, 2022

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 19, 2022

ChrsMark added the Team:Integrations Label for the Integrations team label Jan 19, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 19, 2022

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Jan 19, 2022

ChrsMark requested a review from tetianakravchenko January 19, 2022 11:44

ChrsMark added the backport-v8.1.0 Automated backport with mergify label Jan 19, 2022

mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Jan 19, 2022

Add changelog and docs

fb2f53c

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark mentioned this pull request Jan 20, 2022

Align kubernetes module configuration #29132

Closed

tetianakravchenko reviewed Jan 20, 2022

View reviewed changes

Fixes

b0ff4c5

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark mentioned this pull request Jan 24, 2022

Allign k8s configuration settings elastic/integrations#2556

Closed

tetianakravchenko approved these changes Jan 24, 2022

View reviewed changes

ChrsMark merged commit c1d4f28 into elastic:master Jan 24, 2022

yashtewari pushed a commit to build-security/beats that referenced this pull request Jan 30, 2022

Allign k8s configuration settings (elastic#29908)

329ceab

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allign k8s configuration settings #29908

Allign k8s configuration settings #29908

ChrsMark commented Jan 19, 2022

elasticmachine commented Jan 19, 2022

mergify bot commented Jan 19, 2022

elasticmachine commented Jan 19, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

ChrsMark commented Jan 19, 2022

ChrsMark commented Jan 19, 2022

tetianakravchenko Jan 20, 2022 •

edited

Loading

ChrsMark Jan 20, 2022 •

edited

Loading

tetianakravchenko Jan 20, 2022

tetianakravchenko Jan 20, 2022

ChrsMark Jan 20, 2022

tetianakravchenko Jan 20, 2022 •

edited

Loading

ChrsMark Jan 20, 2022

tetianakravchenko Jan 20, 2022

ChrsMark commented Jan 20, 2022

ChrsMark commented Jan 24, 2022

tetianakravchenko left a comment

	if config.Namespace != "" {
	options.Namespace = config.Namespace
	}

Allign k8s configuration settings #29908

Allign k8s configuration settings #29908

Conversation

ChrsMark commented Jan 19, 2022

What does this PR do?

Why is it important?

Related issues

elasticmachine commented Jan 19, 2022

mergify bot commented Jan 19, 2022

elasticmachine commented Jan 19, 2022 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

ChrsMark commented Jan 19, 2022

ChrsMark commented Jan 19, 2022

tetianakravchenko Jan 20, 2022 • edited Loading

Choose a reason for hiding this comment

ChrsMark Jan 20, 2022 • edited Loading

Choose a reason for hiding this comment

tetianakravchenko Jan 20, 2022

Choose a reason for hiding this comment

tetianakravchenko Jan 20, 2022

Choose a reason for hiding this comment

ChrsMark Jan 20, 2022

Choose a reason for hiding this comment

tetianakravchenko Jan 20, 2022 • edited Loading

Choose a reason for hiding this comment

ChrsMark Jan 20, 2022

Choose a reason for hiding this comment

tetianakravchenko Jan 20, 2022

Choose a reason for hiding this comment

ChrsMark commented Jan 20, 2022

ChrsMark commented Jan 24, 2022

tetianakravchenko left a comment

Choose a reason for hiding this comment

elasticmachine commented Jan 19, 2022 •

edited by jenkins-beats-ci bot

Loading

tetianakravchenko Jan 20, 2022 •

edited

Loading

ChrsMark Jan 20, 2022 •

edited

Loading

tetianakravchenko Jan 20, 2022 •

edited

Loading