Fixes search usage telemetry

[Flyingliuhub]

Signed-off-by: Tao liu liutaoaz@amazon.com

Description

Search Telemetry is used to analyze performance for sucess/failed search request in Dashboards. It will save telemetry data into .kibana_1 index, there are thousands of concurrent search request from Dashboards. It causes significant load in OpenSearch cluster. We have been seen some case 1/3 Search/Update request of total are come from this. This PR will suppress the search usage telemetry based on the switch defined on opensearch_dashboard.yml or data plugin config.

Issues Resolved

resolves #928

Check List

• New functionality includes testing.
  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
    • yarn test:ftr
• New functionality has been documented.
• Commits are signed per the DCO using --signoff

yarn test:jest

Test Suites: 1 skipped, 1432 passed, 1432 of 1433 total
Tests:       24 skipped, 9 todo, 10721 passed, 10754 total
Snapshots:   2452 passed, 2452 total
Time:        241.591 s
Ran all test suites.
Done in 243.82s.

yarn test:jest_integration

Test Suites: 50 passed, 50 total
Tests:       5 skipped, 438 passed, 443 total
Snapshots:   77 passed, 77 total
Time:        294.497 s
Ran all test suites.
         │ info [opensearch] cleanup complete
Done in 296.78s.

opensearch_dashboard.yml enabled value	data plugin config enabled value	Results
TRUE	FALSE	Passed
FALSE	FALSE	Passed
TRUE	TRUE	Passed
FALSE	TRUE	Passed
NONE	TRUE	Passed
NONE	FALSE	Passed

[seraphjiang]

I like the this table, explain how to use this very well. what's our recommendation to make it a user manual or doc
@kavilla @seanneumann

opensearch_dashboard.yml enabled value	data plugin config enabled value	Results
TRUE	FALSE	Passed
FALSE	FALSE	Passed
TRUE	TRUE	Passed
FALSE	TRUE	Passed
NONE	TRUE	Passed
NONE	FALSE	Passed

[mihirsoni]

@Flyingliuhub Thanks this is super nice, it would be great to add some UTs to ensure the config works in expected behavior.

[kavilla]

@Flyingliuhub Thanks this is super nice, it would be great to add some UTs to ensure the config works in expected behavior.

+1

[kavilla]

Do we need add this config here for Docker containers:

https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/bin/opensearch-dashboards-docker#L17

and then update

https://github.com/opensearch-project/opensearch-build/blob/main/docker/release/config/opensearch-dashboards/opensearch-dashboards-docker-entrypoint.sh#L21 ?

[Flyingliuhub]

Do we need add this config here for Docker containers:

https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/bin/opensearch-dashboards-docker#L17

and then update

https://github.com/opensearch-project/opensearch-build/blob/main/docker/release/config/opensearch-dashboards/opensearch-dashboards-docker-entrypoint.sh#L21 ?

We used the default value from config, do we need to change docker file ?

[Flyingliuhub]

Thanks, I added some unit tests.

[seraphjiang]

LGTM

[mihirsoni]

LGTM !! Thanks for UT.

[Flyingliuhub]

Thanks Mihir.

[kavilla]

Is this still valid: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/server/server.api.md?plain=1#L1104

I know we don't currently use these docs but I think @ashwin-pc just made a proposal here: opensearch-project/documentation-website#492 to start including this docs which I agree with. So might be worth to update this doc here.

[kavilla]

I like the this table, explain how to use this very well. what's our recommendation to make it a user manual or doc @kavilla @seanneumann
opensearch_dashboard.yml enabled value data plugin config enabled value Results
TRUE FALSE Passed
FALSE FALSE Passed
TRUE TRUE Passed
FALSE TRUE Passed
NONE TRUE Passed
NONE FALSE Passed

You mean like how to let the public know? Yeah right now the current strategy has been create an issue with the technical details in this repo: https://github.com/opensearch-project/documentation-website/issues/new?assignees=&labels=enhancement%2C+untriaged&template=FEATURE_REQUEST_TEMPLATE.md&title=%5BFEATURE%5D

[kavilla]

I have 1 nit about the default value and 1 comment I think you should address about the potentially undefined case but I'm still ecstatic that this is in here!

So don't want to block this. I think if you just make a commit on top of this should not wipe out my approval. Just don't rebase, that will definitely clear everyone's approval.