Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Pipelines] Migrate to new ES client #96406

Merged

Conversation

jloleysens
Copy link
Contributor

Summary

See #73973

How to test

Steps I followed:

  1. Start Kibana and ES
  2. Navigate to the ingest pipelines subsection in the management section
  3. See that all existing pipelines load correctly
  4. Create a new pipeline, also testing the pipeline as you add new processors. Use the functionality provided to retrieve documents from the cluster (can pick a document from the .kibana index, you should just retrieve a corresponding doc id too)
  5. Save the pipeline
  6. Modify the pipeline you just deleted
  7. Create another pipeline
  8. Bulk delete both pipelines that were just created

- removed use of isEsError to detect legacy errors
- refactored types to use types from @elastic/elasticsearch
  instead (where appropriate)

tested get, put, post, delete, simulate and documents endpoints
locally
@jloleysens jloleysens added v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more release_note:skip Skip the PR/issue when compiling release notes Feature:Ingest Node Pipelines Ingest node pipelines management v7.13.0 labels Apr 7, 2021
@jloleysens jloleysens requested a review from a team as a code owner April 7, 2021 13:06
@elasticmachine
Copy link
Contributor

Pinging @elastic/es-ui (Team:Elasticsearch UI)

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

Copy link
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for migrating another plugin @jloleysens! I left two comments that I'd like to see addressed before approving.

Would you also mind updating x-pack/test/api_integration/apis/management/ingest_pipelines/lib/elasticsearch.ts to use the new es-js client?

if (isEsError(error)) {
// ES returns 404 when there are no pipelines
// Instead, we return an empty array and 200 status back to the client
if (error.status === 404) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this check no longer needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! This behaviour needs to be preserved!


try {
const pipelines = await callAsCurrentUser('ingest.getPipeline');
const { body: pipelines } = await clusterClient.asCurrentUser.ingest.getPipeline({
Copy link
Contributor

@alisonelizabeth alisonelizabeth Apr 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think an id is required or needed to fetch all pipelines.

I'm actually getting an error currently when trying to load the list view as-is. (I'm surprised a test didn't catch this 🤔 nm, looks like it did 😄 )

Screen Shot 2021-04-07 at 1 35 25 PM

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha this is what happens when you commit test code! This is how I came across the issue with messages returned in handlEsError 😅 . Will fix!

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

Copy link
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest LGTM. Thanks @jloleysens!

I still see that x-pack/test/api_integration/apis/management/ingest_pipelines/lib/elasticsearch.ts is using the legacy client. I'd prefer that is addressed as part of this PR so we don't have to go back and fix it. I'm going to go ahead and approve to unblock you, with the assumption that that will be updated before merging 😄 .

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@jloleysens
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@jloleysens jloleysens merged commit f31e13c into elastic:master Apr 13, 2021
@jloleysens jloleysens deleted the ingest-piplines/migrate-legacy-es-client branch April 13, 2021 08:51
jloleysens added a commit that referenced this pull request Apr 14, 2021
* - migrated use of legacy.client to client
- removed use of isEsError to detect legacy errors
- refactored types to use types from @elastic/elasticsearch
  instead (where appropriate)

tested get, put, post, delete, simulate and documents endpoints
locally

* remove use of legacyEs service in functional test

* fixing type issues and API response object

* remove id from get all request!

* reinstated logic for handling 404 from get all pipelines request

* clarify error handling with comments and small variable name refactor

* updated delete error responses

* update functional test

* refactor use of legacyEs

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Ingest Node Pipelines Ingest node pipelines management release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.13.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants