-
Notifications
You must be signed in to change notification settings - Fork 110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
added support for mustache scripting of rollup.target_index field #435
added support for mustache scripting of rollup.target_index field #435
Conversation
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
Looks like the test and build workflows are failing due to code style errors.
Which shouldn't be hard to fix |
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
We allow source index names to have wildcards, so we can have scenarios like this: To prevent these failures at runtime and clarify the behavior let's validate that the user isn't using both wildcard source indices and a template in their target index. I think an easy way would just be to temporarily resolve the target index name in the init block of Rollup.kt and validate that it doesn't contain any wildcards. I don't mean updating the target index name with the resolved template, just resolving here to perform input validation. Feel free to implement it however you find best though! |
...n/org/opensearch/indexmanagement/rollup/action/mapping/TransportUpdateRollupMappingAction.kt
Show resolved
Hide resolved
...n/org/opensearch/indexmanagement/rollup/action/mapping/TransportUpdateRollupMappingAction.kt
Outdated
Show resolved
Hide resolved
...main/kotlin/org/opensearch/indexmanagement/rollup/util/RollupFieldValueExpressionResolver.kt
Outdated
Show resolved
Hide resolved
...main/kotlin/org/opensearch/indexmanagement/rollup/util/RollupFieldValueExpressionResolver.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/org/opensearch/indexmanagement/indexstatemanagement/action/RollupActionIT.kt
Outdated
Show resolved
Hide resolved
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
…etIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
...rg/opensearch/indexmanagement/indexstatemanagement/step/rollup/AttemptCreateRollupJobStep.kt
Outdated
Show resolved
Hide resolved
...main/kotlin/org/opensearch/indexmanagement/rollup/action/index/TransportIndexRollupAction.kt
Outdated
Show resolved
Hide resolved
Thanks Petar, I am happy to approve after these. |
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
dbb884d
to
6dc3385
Compare
Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
* added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * defekt fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * tests Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * small refactor/improvements Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * lint fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added using toMap() Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * removed catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * exception fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * linter fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * fixed IT fail Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit e858ab2)
…) (#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * defekt fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * tests Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * small refactor/improvements Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * lint fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added using toMap() Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * removed catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * exception fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * linter fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * fixed IT fail Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com>
…ensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * exception fix * linter fix * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
…ensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * exception fix * linter fix * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
…ensearch-project#435) (opensearch-project#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * defekt fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * tests Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * small refactor/improvements Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * lint fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added using toMap() Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * removed catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * exception fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * linter fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * fixed IT fail Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com> Signed-off-by: Angie Zhang <langelzh@amazon.com>
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com>
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit 70cf4ea)
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit 70cf4ea)
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com>
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com>
…ensearch-project#435) (opensearch-project#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * defekt fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * tests Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * small refactor/improvements Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * lint fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added using toMap() Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * removed catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * exception fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * linter fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * fixed IT fail Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com>
…ch-project#586) * added support for mustache scripting of rollup.target_index field (opensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com>
…ensearch-project#435) (opensearch-project#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * defekt fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * tests Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * small refactor/improvements Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * lint fixes Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added using toMap() Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * removed catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * exception fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * linter fix Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * fixed IT fail Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com> Signed-off-by: Ronnak Saxena <ronsax@amazon.com>
…ch-project#586) * added support for mustache scripting of rollup.target_index field (opensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <petar.dzepina@gmail.com> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <petar.dzepina@gmail.com> Signed-off-by: Ronnak Saxena <ronsax@amazon.com>
Issue #, if available: 61
Description of changes:
Added capability to use mustache scripting as target_index field value. Only supported param right now is source_index: target_index: rollup_{{ctx.source_index}}
CheckList:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.