[OE] Support compression for requests #6366

kavilla · 2024-04-08T11:16:50Z

Description

Support compressing traffic with opensearch.compression: true or opensearch.compress: 'gzip'.

Also, set compression in the Node server and OpenSearch traffic for CI.

Issues Resolved

#5296

Changelog

feat: Support compressing traffic with opensearch.compression: true or opensearch.compress: 'gzip'

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

ashwin-pc

LGTM

src/core/server/opensearch/opensearch_config.ts

github-actions · 2024-04-16T22:54:53Z

❌ Invalid Prefix For Manual Changeset Creation

Invalid description prefix. Found "feat". Only "skip" entry option is permitted for manual commit of changeset files.

If you were trying to skip the changelog entry, please use the "skip" entry option in the ##Changelog section of your PR description.

Support compressing traffic with `opensearch.compression: true`. Also, set compression in the Node server and OpenSearch traffic for CI. Issue resolved: opensearch-project#5296 Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

codecov · 2024-05-02T03:42:59Z

Codecov Report

Attention: Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.

Project coverage is 67.69%. Comparing base (9ac5203) to head (840f186).
Report is 412 commits behind head on main.

Files with missing lines	Patch %	Lines
src/core/server/opensearch/client/client_config.ts	0.00%	1 Missing and 1 partial ⚠️
src/core/server/opensearch/opensearch_config.ts	50.00%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6366      +/-   ##
==========================================
- Coverage   67.70%   67.69%   -0.01%     
==========================================
  Files        3417     3417              
  Lines       66922    66928       +6     
  Branches    10888    10891       +3     
==========================================
  Hits        45310    45310              
- Misses      18966    18968       +2     
- Partials     2646     2650       +4

Flag	Coverage Δ
Linux_1	`33.17% <16.66%> (-0.01%)`	⬇️
Linux_2	`55.58% <33.33%> (-0.01%)`	⬇️
Linux_3	`45.25% <16.66%> (-0.02%)`	⬇️
Linux_4	`34.84% <16.66%> (-0.01%)`	⬇️
Windows_1	`33.19% <16.66%> (-0.01%)`	⬇️
Windows_2	`55.55% <33.33%> (-0.01%)`	⬇️
Windows_3	`45.25% <16.66%> (-0.01%)`	⬇️
Windows_4	`34.84% <16.66%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

AMoo-Miki · 2024-05-14T23:56:42Z

src/core/server/opensearch/opensearch_config.ts

+        ? DEFAULT_COMPRESSION
+        : typeof rawConfig.compression === 'string'
+        ? rawConfig.compression
+        : undefined;


An expression of rawConfig.compression = false gets out of this as undefined which is the wrong message out of this function (because it was defined). For the sanity of future devs, we should:

Suggested change

: undefined;

: false;

Also, for the sanity of future devs, can we not use nested ternaries, especially without parens? calling a function with readable if statements isn't the worst thing imo :)

BionIT · 2024-06-05T03:30:58Z

@AMoo-Miki @ashwin-pc could you help take another look at this PR which is targeting 2.15?

AMoo-Miki · 2024-06-07T18:56:25Z

@ashwin-pc, I think Rocky is tied up. If you agree with my suggestion, we can move this along.

virajsanghvi · 2024-06-08T00:05:00Z

src/core/server/opensearch/client/client_config.ts

@@ -120,6 +121,10 @@ export function parseClientOptions(config: OpenSearchClientConfig, scoped: boole
    clientOptions.disablePrototypePoisoningProtection = config.disablePrototypePoisoningProtection;
  }

+  if (config.compression) {
+    clientOptions.compression = config.compression;


do we want to add a test for this given we updated the test to not do this by default?

virajsanghvi · 2024-06-08T00:07:46Z

src/core/server/opensearch/opensearch_config.ts

+        ? DEFAULT_COMPRESSION
+        : typeof rawConfig.compression === 'string'
+        ? rawConfig.compression
+        : undefined;


Also, for the sanity of future devs, can we not use nested ternaries, especially without parens? calling a function with readable if statements isn't the worst thing imo :)

ananzh · 2024-06-10T20:21:42Z

move to 2.16

kavilla requested review from ananzh, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, zhongnansu, manasvinibs, ZilongX, Flyingliuhub, curq, bandinib-amzn, SuZhou-Joe, ruanyl, BionIT and xinruiba as code owners April 8, 2024 11:16

github-actions bot added the distinguished-contributor label Apr 8, 2024

kavilla linked an issue Apr 8, 2024 that may be closed by this pull request

[Resource Management] Compress traffic #5296

Open

kavilla force-pushed the kavilla/support_compression branch from fdc842b to 218f752 Compare April 8, 2024 22:49

ashwin-pc previously approved these changes Apr 8, 2024

View reviewed changes

src/core/server/opensearch/opensearch_config.ts Outdated Show resolved Hide resolved

kavilla dismissed ashwin-pc’s stale review via 1e6846d April 9, 2024 00:10

SuZhou-Joe reviewed Apr 9, 2024

View reviewed changes

src/core/server/opensearch/opensearch_config.ts Outdated Show resolved Hide resolved

AMoo-Miki reviewed Apr 11, 2024

View reviewed changes

src/core/server/opensearch/opensearch_config.ts Outdated Show resolved Hide resolved

AMoo-Miki previously approved these changes Apr 11, 2024

View reviewed changes

ashwin-pc previously approved these changes Apr 12, 2024

View reviewed changes

ashwin-pc added the operational excellence label Apr 12, 2024

github-actions bot added failed changeset Skip-Changelog PRs that are too trivial to warrant a changelog or release notes entry and removed failed changeset labels Apr 16, 2024

kavilla added 2 commits May 1, 2024 19:57

[OE] Support compression for requests

e3b9e8f

Support compressing traffic with `opensearch.compression: true`. Also, set compression in the Node server and OpenSearch traffic for CI. Issue resolved: opensearch-project#5296 Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

address comments

0b166e8

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

kavilla dismissed stale reviews from ashwin-pc and AMoo-Miki via 0b166e8 May 2, 2024 03:29

kavilla force-pushed the kavilla/support_compression branch from 007f616 to 0b166e8 Compare May 2, 2024 03:29

kavilla requested review from seraphjiang, SuZhou-Joe, ashwin-pc and AMoo-Miki May 2, 2024 03:29

Changeset file for PR opensearch-project#6366 created/updated

840f186

github-actions bot removed the Skip-Changelog PRs that are too trivial to warrant a changelog or release notes entry label May 2, 2024

kavilla added the backport 2.x label May 2, 2024

kavilla added the v2.15.0 label May 2, 2024

AMoo-Miki reviewed May 14, 2024

View reviewed changes

yujin-emma assigned AMoo-Miki May 28, 2024

BionIT assigned ananzh Jun 5, 2024

ananzh mentioned this pull request Jun 7, 2024

[Meta][2.15 Release] OpenSearch Dashboards Visualization+Discover+CVEs+Maintenance PRs sign-off #6962

Closed

BionIT assigned ashwin-pc Jun 7, 2024

virajsanghvi reviewed Jun 8, 2024

View reviewed changes

peterzhuamazon mentioned this pull request Jun 10, 2024

[RELEASE] Release version 2.15.0 opensearch-project/opensearch-build#4681

Closed

74 tasks

ananzh added v2.16.0 and removed v2.15.0 labels Jun 10, 2024

ananzh added v2.17.0 and removed v2.16.0 labels Jul 25, 2024

ashwin-pc added v2.18.0 and removed v2.17.0 labels Sep 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[OE] Support compression for requests #6366

[OE] Support compression for requests #6366

kavilla commented Apr 8, 2024 •

edited

Loading

ashwin-pc left a comment

github-actions bot commented Apr 16, 2024

codecov bot commented May 2, 2024 •

edited

Loading

AMoo-Miki May 14, 2024

virajsanghvi Jun 8, 2024

BionIT commented Jun 5, 2024

AMoo-Miki commented Jun 7, 2024 •

edited

Loading

virajsanghvi Jun 8, 2024

virajsanghvi Jun 8, 2024

ananzh commented Jun 10, 2024

[OE] Support compression for requests #6366

Are you sure you want to change the base?

[OE] Support compression for requests #6366

Conversation

kavilla commented Apr 8, 2024 • edited Loading

Description

Issues Resolved

Changelog

Check List

ashwin-pc left a comment

Choose a reason for hiding this comment

github-actions bot commented Apr 16, 2024

❌ Invalid Prefix For Manual Changeset Creation

codecov bot commented May 2, 2024 • edited Loading

Codecov Report

AMoo-Miki May 14, 2024

Choose a reason for hiding this comment

virajsanghvi Jun 8, 2024

Choose a reason for hiding this comment

BionIT commented Jun 5, 2024

AMoo-Miki commented Jun 7, 2024 • edited Loading

virajsanghvi Jun 8, 2024

Choose a reason for hiding this comment

virajsanghvi Jun 8, 2024

Choose a reason for hiding this comment

ananzh commented Jun 10, 2024

kavilla commented Apr 8, 2024 •

edited

Loading

codecov bot commented May 2, 2024 •

edited

Loading

AMoo-Miki commented Jun 7, 2024 •

edited

Loading