[Ingest Pipelines] Processor forms for processors K-S #75638

jloleysens · 2020-08-21T10:20:35Z

Summary

KV
Lowercase
Pipeline
Remove
Rename
Script
Set
Set security user
Split
Sort

How to test

Start Kibana with basic license
Go to ingest pipelines plugin in management section
Add a processor for each of the forms listed above (attempt to submit to see which fields are required)
Create the pipeline

Notes

Have not spent time on optimising the layout of the forms. We can definitely discuss this here
Copy review is still needed
Trying to devise a good form of test coverage for these forms
I intend to add processor forms in batches of ~10 so that each form gets proper review and testing
Continuation of [Ingest Pipelines] Processor forms for processors A-D #72849 and [Ingest Pipelines] Processor forms for processors E-J #75054

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

Also: - Also refactored name of defaultBoolToUndef to undefinedIfValue to make it more broadly applicable for deserializing when default values have been provided - Refactor to pass in original processor options value so that a form can initialize based on those values

- also fixed a bug that would render a horizontal line even if the form has no fields of its own (like the drop processor form)

- also added comments to set_security_user processor

elasticmachine · 2020-08-21T10:20:37Z

Pinging @elastic/es-ui (Team:Elasticsearch UI)

yuliacech

Hi @jloleysens , amazing work on this batch of processors!
I tested all new processors locally and all worked for me. Left you a couple comments in the code. I think the only fix needed before merging is trim_value in KV processor, the rest is non-blocking, just some ideas for UI.

...ion/components/pipeline_processors_editor/components/manage_processor_form/processors/kv.tsx

...components/pipeline_processors_editor/components/manage_processor_form/processors/script.tsx

...n/components/pipeline_processors_editor/components/manage_processor_form/processors/sort.tsx

jloleysens · 2020-08-24T11:22:27Z

@elasticmachine merge upstream

jloleysens · 2020-08-25T11:13:55Z

@elasticmachine merge upstream

…f common heights

…nes/processors-forms-k-s * 'master' of github.com:elastic/kibana: (216 commits) [Ingest Manager] Split Registry errors into Connection & Response (elastic#76558) [Security Solution] add an excess validation instead of the exact match (elastic#76472) Introduce TS incremental builds & move src/test_utils to TS project (elastic#76082) fix bad merge (elastic#76629) [Newsfeed] Ensure the version format when calling the API (elastic#76381) remove server_extensions mixin (elastic#76606) Remove legacy applications and legacy mode (elastic#75987) [Discover] Fix sidebar element focus behavior when adding / removing columns (elastic#75749) Replace FetchOptions with ISearchOptions (elastic#76538) Move streams utils to the core (elastic#76397) [Ingest Manager] Wording & linking improvements (elastic#76284) remove legacy kibana plugin (elastic#76064) [Form lib] Fix regression on field not being validated after reset to its default value. (elastic#76379) Legacy SO import: Fix bug causing multiple overrides to only show the last confirm modal (elastic#76482) [APM] Service maps layout enhancements (elastic#76481) [Security Solution][Detection Engine] Remove RuleTypeSchema in favor of Type for TypeScript (elastic#76586) [Security Solution][Exceptions] - Updates enum schema and tests (elastic#76544) Index Pattern class - remove toJSON and toString (elastic#76246) skip failing suite (elastic#76581) Split up and clarify Enterprise Search codeowners (elastic#76580) ... # Conflicts: # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processor_settings_fields.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/foreach.tsx

…oleysens/kibana into ingest-pipelines/processors-forms-k-s * 'ingest-pipelines/processors-forms-k-s' of github.com:jloleysens/kibana: (216 commits) [Ingest Manager] Split Registry errors into Connection & Response (elastic#76558) [Security Solution] add an excess validation instead of the exact match (elastic#76472) Introduce TS incremental builds & move src/test_utils to TS project (elastic#76082) fix bad merge (elastic#76629) [Newsfeed] Ensure the version format when calling the API (elastic#76381) remove server_extensions mixin (elastic#76606) Remove legacy applications and legacy mode (elastic#75987) [Discover] Fix sidebar element focus behavior when adding / removing columns (elastic#75749) Replace FetchOptions with ISearchOptions (elastic#76538) Move streams utils to the core (elastic#76397) [Ingest Manager] Wording & linking improvements (elastic#76284) remove legacy kibana plugin (elastic#76064) [Form lib] Fix regression on field not being validated after reset to its default value. (elastic#76379) Legacy SO import: Fix bug causing multiple overrides to only show the last confirm modal (elastic#76482) [APM] Service maps layout enhancements (elastic#76481) [Security Solution][Detection Engine] Remove RuleTypeSchema in favor of Type for TypeScript (elastic#76586) [Security Solution][Exceptions] - Updates enum schema and tests (elastic#76544) Index Pattern class - remove toJSON and toString (elastic#76246) skip failing suite (elastic#76581) Split up and clarify Enterprise Search codeowners (elastic#76580) ...

kibanamachine · 2020-09-03T18:21:06Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 048c42a

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
ingestPipelines	426	+10	416

async chunks size

id	value	diff	baseline
ingestPipelines	731.5KB	+25.8KB	705.7KB

History

💔 Build #72289 failed dd24adf
💔 Build #72280 failed 480523f
💔 Build #70232 failed e4c1341
💔 Build #69668 failed a3be97b
💔 Build #69642 failed 77fe50b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

yuliacech

Hi @jloleysens ,
I tested KV and some other processors locally and all worked for me.
Great job on all the improvements, especially editor heights constants!

jloleysens · 2020-09-04T12:11:10Z

@yuliacech thanks for the re-review!

* added kv processor form * added lowercase processor form * Added pipeline processor form * added remove processor form * added rename processor form * Add script processor form Also: - Also refactored name of defaultBoolToUndef to undefinedIfValue to make it more broadly applicable for deserializing when default values have been provided - Refactor to pass in original processor options value so that a form can initialize based on those values * Updated the existing set processor form - also fixed a bug that would render a horizontal line even if the form has no fields of its own (like the drop processor form) * Added set_security_user form * Added split processor form - also added comments to set_security_user processor * added sort processor form * fix: duplicate i18n identifier * added field to kv processor form for trim value * replaced all editor heights with a clearer indication of the source of common heights * remove unused translations * remove unused translation Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* master: (47 commits) Do not require id & description when creating a logstash pipeline (elastic#76616) Remove commented src/core/tsconfig file (elastic#76792) Replaced whitelistedHosts with allowedHosts in actions ascii docs (elastic#76731) [Dashboard First] Genericize Attribute Service (elastic#76057) [ci-metrics] unify distributable file count metrics (elastic#76448) [Security Solution][Detections] Handle conflicts on alert status update (elastic#75492) [eslint] convert to @typescript-eslint/no-unused-expressions (elastic#76471) [DOCS] Add default time range filter to advanced settings (elastic#76414) [Security Solution] Refactor NetworkTopNFlow to use Search Strategy (elastic#76249) [Dashboard] Update Index Patterns when Child Index Patterns Change (elastic#76356) [ML] Add option to Advanced Settings to set default time range filter for AD jobs (elastic#76347) Add CSM app to CODEOWNERS (elastic#76793) [Security Solution][Exceptions] - Updates exception item find sort field (elastic#76685) [Security Solution][Detections][Tech Debt] - Move to using common io-ts types (elastic#75009) [Lens] Drag dimension to replace (elastic#75895) URI encode the index names we fetch in the fetchIndices lib function. (elastic#76584) [Security Solution] Resolver retrieve entity id of documents without field mapped (elastic#76562) [Ingest Manager] validate agent route using AJV instead kbn-config-schema (elastic#76546) Updated non-dev usages of node-forge (elastic#76699) [Ingest Pipelines] Processor forms for processors K-S (elastic#75638) ...

* added kv processor form * added lowercase processor form * Added pipeline processor form * added remove processor form * added rename processor form * Add script processor form Also: - Also refactored name of defaultBoolToUndef to undefinedIfValue to make it more broadly applicable for deserializing when default values have been provided - Refactor to pass in original processor options value so that a form can initialize based on those values * Updated the existing set processor form - also fixed a bug that would render a horizontal line even if the form has no fields of its own (like the drop processor form) * Added set_security_user form * Added split processor form - also added comments to set_security_user processor * added sort processor form * fix: duplicate i18n identifier * added field to kv processor form for trim value * replaced all editor heights with a clearer indication of the source of common heights * remove unused translations * remove unused translation Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

jloleysens added 10 commits August 21, 2020 10:28

added kv processor form

1179a76

added lowercase processor form

772426e

Added pipeline processor form

c881b26

added remove processor form

e11c0e2

added rename processor form

38806b4

Updated the existing set processor form

7f7a4f5

- also fixed a bug that would render a horizontal line even if the form has no fields of its own (like the drop processor form)

Added set_security_user form

4d3379b

Added split processor form

932b574

- also added comments to set_security_user processor

added sort processor form

77fe50b

jloleysens added v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more release_note:skip Skip the PR/issue when compiling release notes v7.10.0 Feature:Ingest Node Pipelines Ingest node pipelines management labels Aug 21, 2020

jloleysens requested a review from yuliacech August 21, 2020 10:20

jloleysens requested a review from a team as a code owner August 21, 2020 10:20

fix: duplicate i18n identifier

a3be97b

yuliacech reviewed Aug 21, 2020

View reviewed changes

elasticmachine and others added 7 commits August 25, 2020 05:14

Merge branch 'master' into ingest-pipelines/processors-forms-k-s

e4c1341

added field to kv processor form for trim value

91d5f8b

replaced all editor heights with a clearer indication of the source o…

a30dfc8

…f common heights

remove unused translations

dd24adf

remove unused translation

048c42a

yuliacech approved these changes Sep 4, 2020

View reviewed changes

jloleysens merged commit 2326f3f into elastic:master Sep 4, 2020

jloleysens deleted the ingest-pipelines/processors-forms-k-s branch September 4, 2020 12:11

jloleysens mentioned this pull request Sep 4, 2020

[7.x] [Ingest Pipelines] Processor forms for processors K-S (#75638) #76762

Merged

alisonelizabeth mentioned this pull request Sep 4, 2020

[Ingest pipelines] Forms for processors T-U #76710

Merged

jrodewig mentioned this pull request Sep 8, 2020

[Ingest Pipelines] Add descriptions for ingest processors K-S #76981

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Ingest Pipelines] Processor forms for processors K-S #75638

[Ingest Pipelines] Processor forms for processors K-S #75638

jloleysens commented Aug 21, 2020 •

edited

Loading

elasticmachine commented Aug 21, 2020

yuliacech left a comment

jloleysens commented Aug 24, 2020

jloleysens commented Aug 25, 2020

kibanamachine commented Sep 3, 2020

yuliacech left a comment

jloleysens commented Sep 4, 2020

[Ingest Pipelines] Processor forms for processors K-S #75638

[Ingest Pipelines] Processor forms for processors K-S #75638

Conversation

jloleysens commented Aug 21, 2020 • edited Loading

Summary

How to test

Notes

Checklist

For maintainers

elasticmachine commented Aug 21, 2020

yuliacech left a comment

Choose a reason for hiding this comment

jloleysens commented Aug 24, 2020

jloleysens commented Aug 25, 2020

kibanamachine commented Sep 3, 2020

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

async chunks size

History

yuliacech left a comment

Choose a reason for hiding this comment

jloleysens commented Sep 4, 2020

jloleysens commented Aug 21, 2020 •

edited

Loading