Enhance profile API to add model centric result controlled by view parameter #714

zane-neo · 2023-01-28T08:11:41Z

Description

This change is to add model centric result for profile API, so that user can fetch either node centric result or model centric result or both.

Issues Resolved

#646

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…amter Signed-off-by: Zan Niu <zaniu@amazon.com>

…ameter Signed-off-by: Zan Niu <zaniu@amazon.com>

codecov-commenter · 2023-01-28T08:34:04Z

Codecov Report

Merging #714 (f3a2888) into 2.x (1bf57c5) will increase coverage by 0.70%.
The diff coverage is 95.40%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff              @@
##                2.x     #714      +/-   ##
============================================
+ Coverage     84.34%   85.04%   +0.70%     
- Complexity     1052     1077      +25     
============================================
  Files            99      100       +1     
  Lines          3833     3919      +86     
  Branches        352      371      +19     
============================================
+ Hits           3233     3333     +100     
+ Misses          450      430      -20     
- Partials        150      156       +6

Flag	Coverage Δ
ml-commons	`85.04% <95.40%> (+0.70%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...va/org/opensearch/ml/rest/RestMLProfileAction.java	`94.62% <90.47%> (-3.46%)`	⬇️
...arch/ml/action/profile/MLProfileModelResponse.java	`100.00% <100.00%> (ø)`
.../java/org/opensearch/ml/utils/RestActionUtils.java	`81.63% <100.00%> (+0.38%)`	⬆️
...java/org/opensearch/ml/profile/MLModelProfile.java	`62.96% <0.00%> (+16.66%)`	⬆️
...g/opensearch/ml/profile/MLPredictRequestStats.java	`77.27% <0.00%> (+20.45%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

…rameter (opensearch-project#714) * Enhance profile API to add model centric result controled by view paramter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> --------- Signed-off-by: Zan Niu <zaniu@amazon.com>

…in (#711) * [wjunshen] #N/A feat: fix after the latest rebase Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: fix after rebase Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: fix after rebase Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: fix after rebase Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: fix after the latest rebase Signed-off-by: wujunshen <frank_wjs@hotmail.com> * Increment version to 2.6.0-SNAPSHOT (#671) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * fix profile API in example doc (#712) Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * change model url to public repo in text embedding model example doc (#713) Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * Enhance profile API to add model centric result controlled by view parameter (#714) * Enhance profile API to add model centric result controled by view paramter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> --------- Signed-off-by: Zan Niu <zaniu@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * add planning work nodes to model (#715) * add planning work nodes to model Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add test Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * skip running syncup job if no model index (#717) Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * refactor: add DL model class (#722) * refactor: add DL model class Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix model url in example doc Signed-off-by: Yaliang Wu <ylwu@amazon.com> * address comments Signed-off-by: Yaliang Wu <ylwu@amazon.com> * fix failed ut Signed-off-by: Yaliang Wu <ylwu@amazon.com> --------- Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * tune model config: change pooling mode to optional (#724) Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: make the log readable Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: add error log Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: Refer to PR #717,just checking if index exists in cluster metadata Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: change RunTimeException to MLException Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: also consider COMPLETED_WITH_ERROR Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: remove ML_MODEL_RELOAD_MAX_RETRY_TIMES in CommonValue.java Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: remove Result class Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: change "reload" and "retry" to a full word Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: change log info sentence Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [wjunshen] #N/A feat: code format Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [Signed-off-by: wjunshen<wjunshen@amazon.com>] #N/A feat: 1. Let the collection just have all ids of ml node 2. print out full exception stack trace Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [Signed-off-by: wjunshen<wjunshen@amazon.com>] #N/A feat: use LOAD_THREAD_POOL to replace generic Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [Signed-off-by: wjunshen<wjunshen@amazon.com>] #N/A feat: print the whole exception stack Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [Signed-off-by: wjunshen<wjunshen@amazon.com>] #N/A feat: change the IndexNotFoundException to MLException Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [Signed-off-by: wjunshen<wjunshen@amazon.com>] #N/A feat: add logs when receiving indexRequestBuilder.execute exception Signed-off-by: wujunshen <frank_wjs@hotmail.com> * [Signed-off-by: wjunshen<wjunshen@amazon.com>] #N/A feat: change the test code after code review Signed-off-by: wujunshen <frank_wjs@hotmail.com> --------- Signed-off-by: wujunshen <frank_wjs@hotmail.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com> Signed-off-by: Zan Niu <zaniu@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Yaliang Wu <ylwu@amazon.com> Co-authored-by: zane-neo <zaniu@amazon.com>

…rameter (opensearch-project#714) * Enhance profile API to add model centric result controled by view paramter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> --------- Signed-off-by: Zan Niu <zaniu@amazon.com>

…rameter (opensearch-project#714) * Enhance profile API to add model centric result controled by view paramter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> --------- Signed-off-by: Zan Niu <zaniu@amazon.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com>

…rameter (#714) * Enhance profile API to add model centric result controled by view paramter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> * Enhance profile API to add model centric result controled by view parameter Signed-off-by: Zan Niu <zaniu@amazon.com> --------- Signed-off-by: Zan Niu <zaniu@amazon.com> Signed-off-by: Yaliang Wu <ylwu@amazon.com>

zane-neo added 2 commits January 28, 2023 13:56

Enhance profile API to add model centric result controled by view par…

5116a70

…amter Signed-off-by: Zan Niu <zaniu@amazon.com>

Enhance profile API to add model centric result controled by view par…

856498e

…ameter Signed-off-by: Zan Niu <zaniu@amazon.com>

zane-neo requested a review from a team January 28, 2023 08:11

zane-neo mentioned this pull request Jan 28, 2023

Enhance profile API to add model centric result controlled by view parameter #701

Closed

5 tasks

Enhance profile API to add model centric result controled by view par…

f3a2888

…ameter Signed-off-by: Zan Niu <zaniu@amazon.com>

zane-neo requested a review from ylwu-amzn January 28, 2023 08:38

ylwu-amzn approved these changes Jan 31, 2023

View reviewed changes

zane-neo requested review from wujunshen and jngz-es and removed request for wujunshen January 31, 2023 00:52

jngz-es approved these changes Jan 31, 2023

View reviewed changes

rbhavna approved these changes Jan 31, 2023

View reviewed changes

zane-neo merged commit 2f7e028 into opensearch-project:2.x Jan 31, 2023

ylwu-amzn added the enhancement New feature or request label Jan 31, 2023

ylwu-amzn mentioned this pull request Mar 4, 2023

[Backport to main] Backport changes from 2.x to main #782

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance profile API to add model centric result controlled by view parameter #714

Enhance profile API to add model centric result controlled by view parameter #714

zane-neo commented Jan 28, 2023 •

edited

Loading

codecov-commenter commented Jan 28, 2023 •

edited

Loading

Enhance profile API to add model centric result controlled by view parameter #714

Enhance profile API to add model centric result controlled by view parameter #714

Conversation

zane-neo commented Jan 28, 2023 • edited Loading

Description

Issues Resolved

Check List

codecov-commenter commented Jan 28, 2023 • edited Loading

Codecov Report

zane-neo commented Jan 28, 2023 •

edited

Loading

codecov-commenter commented Jan 28, 2023 •

edited

Loading