Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.8] Bugfix/202 transform date add date conversion #804

Merged
merged 1 commit into from
Jun 1, 2023

Conversation

opensearch-trigger-bot[bot]
Copy link
Contributor

Backport 42833b1 from #622

* 202: Added format property when specifying the date histogram

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* 202: Added component responsible for building the target index mapping once the transform is being triggered.

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* 202: date_histogram considered in the case of the creating the target index for the date fields when transform is executed

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* 202: Enabled target index date field mappings if those fields are used in aggregations or as a term aggregation for defining the buckets

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* Updated code according to comments. Added targetIndexMapping when transform preview action is triggered

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* Updated schema versions

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* Addressed the comments

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* Refactored transform tests related with aggregation based on a date field.
Updated transform preview action to consider target index mapping when using a date field. Kept formatting of the date field in target index.

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* detekt fix

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* Added zone in IT

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

* Added function for creating target index mapping that considers transform mapping json. Target date field mappings are generated after transform validation when running transform. Removed target index date field values formatting. emoved default format for date_histogram because of the rollup. Updated schema version in test.

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>

---------

Signed-off-by: Stevan Buzejic <buzejic.stevan@gmail.com>
(cherry picked from commit 42833b1)
@codecov
Copy link

codecov bot commented Jun 1, 2023

Codecov Report

Merging #804 (8523aeb) into 2.8 (5c42012) will increase coverage by 0.08%.
The diff coverage is 72.04%.

@@             Coverage Diff              @@
##                2.8     #804      +/-   ##
============================================
+ Coverage     76.00%   76.08%   +0.08%     
- Complexity     2864     2904      +40     
============================================
  Files           364      366       +2     
  Lines         16420    16548     +128     
  Branches       2364     2386      +22     
============================================
+ Hits          12480    12591     +111     
- Misses         2573     2583      +10     
- Partials       1367     1374       +7     
Impacted Files Coverage Δ
...h/indexmanagement/transform/util/TransformUtils.kt 0.00% <0.00%> (ø)
...earch/indexmanagement/transform/TransformRunner.kt 82.24% <66.66%> (+0.32%) ⬆️
...indexmanagement/transform/util/TransformContext.kt 70.00% <66.66%> (-5.00%) ⬇️
...xmanagement/transform/TargetIndexMappingService.kt 69.14% <69.14%> (ø)
.../action/preview/TransportPreviewTransformAction.kt 85.71% <72.00%> (-1.25%) ⬇️
...management/common/model/dimension/DateHistogram.kt 77.55% <77.77%> (-0.48%) ⬇️
...arch/indexmanagement/transform/TransformIndexer.kt 69.09% <85.71%> (ø)
...pensearch/indexmanagement/IndexManagementPlugin.kt 90.20% <100.00%> (+0.03%) ⬆️
...ndexmanagement/transform/TransformSearchService.kt 71.42% <100.00%> (+1.17%) ⬆️

... and 12 files with indirect coverage changes

@bowenlan-amzn bowenlan-amzn merged commit c611ac0 into 2.8 Jun 1, 2023
@github-actions github-actions bot deleted the backport/backport-622-to-2.8 branch June 1, 2023 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants