Skip to content

Latest commit

 

History

History
460 lines (272 loc) · 16.7 KB

opensearch.release-notes-1.2.0.md

File metadata and controls

460 lines (272 loc) · 16.7 KB
Raw

Version 1.2.0 Release Notes

  • Upgrading gson to 2.8.9 (#1541) (#1546)

    Vacha - Tue, 16 Nov 2021 15:22:38 -0800

    EAD -> refs/heads/1.2, refs/remotes/upstream/1.2, refs/remotes/origin/1.2 Signed-off-by: Vacha <vachshah@amazon.com>

  • Add staged version 1.1.1 (#1505)

    Nick Knize - Thu, 4 Nov 2021 14:19:07 -0500

    Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

  • Remove old ES libraries used in reindex due to CVEs (#1359) (#1497)

    Xue Zhou - Tue, 2 Nov 2021 15:54:24 -0700

    This commit removes old ES libraries version 090 and 176 due to CVE Signed-off-by: Xue Zhou <xuezhou@amazon.com>

  • Upgrading dependencies (#1491) (#1495)

    Vacha - Tue, 2 Nov 2021 15:46:18 -0700

    Signed-off-by: Vacha <vachshah@amazon.com>

  • remove codeQL warning about implicit narrowing conversion in compound assignment (#1403) (#1496)

    Xue Zhou - Tue, 2 Nov 2021 15:27:53 -0700

    Signed-off-by: Xue Zhou <xuezhou@amazon.com>

  • Cleanup for Checkstyle #1370 (#1492)

    Owais Kazi - Tue, 2 Nov 2021 13:33:15 -0700

    Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

  • Backporting #1488 spotless check on plugins (#1489)

    Himanshu Setia - Mon, 1 Nov 2021 19:26:45 -0700

    Signed-off-by: Himanshu Setia <setiah@amazon.com>

  • Upgrading dependencies in hdfs plugin (#1466) (#1485)

    Vacha - Mon, 1 Nov 2021 17:12:48 -0700

    Signed-off-by: Vacha <vachshah@amazon.com>

  • Adding spotless support for subprojects under :test (#1479)

    Sarat Vemulapalli - Mon, 1 Nov 2021 15:16:45 -0700

    Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

  • Make TranslogDeletionPolicy abstract for extension (#1456) (#1478)

    Rabi Panda - Mon, 1 Nov 2021 12:41:05 -0700

    As part of the commit 2ebd0e04, we added a new method to the EnginePlugin to provide a custom TranslogDeletionPolicy. This commit makes minTranslogGenRequired method abstract in this class for implementation by child classes. The default implementation is provided by DefaultTranslogDeletionPolicy. Signed-off-by: Rabi Panda <adnapibar@gmail.com>

  • Remove deprecated settings and logic for translog pruning by retention lease. (#1416) (#1471)

    Rabi Panda - Mon, 1 Nov 2021 10:26:20 -0500

    The settings and the corresponding logic for translog pruning by retention

    lease which were added as part of #1100 have been deprecated. This commit removes those deprecated code in favor of an extension point for providing a custom TranslogDeletionPolicy. Signed-off-by: Rabi Panda <adnapibar@gmail.com>

  • Run spotless and exclude checkstyle on rest-api-spec module (#1462) (#1472)

    Owais Kazi - Fri, 29 Oct 2021 17:27:17 -0700

  • [Backport 1.x] Add extension point for custom TranslogDeletionPolicy in EnginePlugin (#1404) (#1424)

    Rabi Panda - Fri, 29 Oct 2021 12:14:25 -0700

    • Add extension point for custom TranslogDeletionPolicy in EnginePlugin. (#1404) This commit adds a method that can be used to provide a custom TranslogDeletionPolicy from within plugins that implement the EnginePlugin interface. This enables plugins to provide a custom deletion policy with the current limitation that only one plugin can override the policy. An exception will be thrown if more than one plugin overrides the policy. Signed-off-by: Rabi Panda <adnapibar@gmail.com>

    • Close first engine instance before creating second (#1457) When creating the second instance of an InternalEngine using the same translog config of the default InternalEngine instance, the second instance will attempt to delete all the existing translog files. I found a deterministic test failure when running with the seed E3E6AAD95ABD299B. As opposed to creating a second engine instance with a different translog location, just close the first one before creating the second. Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Andrew Ross <andrross@amazon.com>

  • Upgrade to Lucene 8.10.1 (#1440) (#1459)

    Nick Knize - Thu, 28 Oct 2021 13:23:25 -0700

    This commit upgrades to the latest release of lucene 8.10 Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

  • Run spotless and exclude checkstyle on modules module (#1442) (#1453)

    Owais Kazi - Thu, 28 Oct 2021 11:02:02 -0700

    Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

  • Fixing bwc test for repository-multi-version (#1441) (#1451)

    Vacha - Wed, 27 Oct 2021 20:22:12 -0400

    Signed-off-by: Vacha <vachshah@amazon.com>

  • [BUG] SymbolicLinkPreservingUntarTransform fails on Windows (#1433) (#1439)

    Andriy Redko - Tue, 26 Oct 2021 15:03:44 -0400

    efs/heads/sr-sandbox Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Upgrading mockito version to make it consistent across the repo (#1410) (#1435)

    Vacha - Tue, 26 Oct 2021 08:48:49 -0400

    Signed-off-by: Vacha <vachshah@amazon.com>

  • Adjust CodeCache size to eliminate JVM warnings (and crashes) (#1426) (#1432)

    Andriy Redko - Tue, 26 Oct 2021 08:48:16 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Run spotless and checkstyle on libs module (#1428)(#1434)

    Owais Kazi - Mon, 25 Oct 2021 15:13:54 -0700

    Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

  • Run spotless and exclude checkstyle on plugins module (#1417) (#1423)

    Owais Kazi - Fri, 22 Oct 2021 13:23:14 -0700

    Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

  • Fix windows build (mostly) (#1412) (#1420)

    Daniel Doubrovkine (dB.) - Fri, 22 Oct 2021 12:41:44 -0400

  • Run spotless and exclude checkstyle on client module #1392 (#1414)

    Owais Kazi - Thu, 21 Oct 2021 19:44:46 -0400

    Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

  • Removing Jenkinsfile (not used), replaced by opensearch-build/jenkins/opensearch/Jenkinsfile (#1408) (#1411)

    Andriy Redko - Thu, 21 Oct 2021 19:42:09 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • [repository-azure] plugin should use Azure Storage SDK v12 for Java (#1302) (#1409)

    Andriy Redko - Thu, 21 Oct 2021 19:41:44 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Add EngineConfig extensions to EnginePlugin (#1387) (#1401)

    Nick Knize - Thu, 21 Oct 2021 08:58:47 -0500

    This commit adds an extension point to EngineConfig through EnginePlugin using

    a new EngineConfigFactory mechanism. EnginePlugin provides interface methods to override configurations in EngineConfig. The EngineConfigFactory produces a new instance of the EngineConfig using these overrides. Defaults are used absent overridden configurations. This serves as a mechanism to override Engine configurations (e.g., CodecService, TranslogConfig) enabling Plugins to have higher fidelity for changing Engine behavior without having to override the entire Engine (which is only permitted for a single plugin). Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

  • Update node attribute check to version update (1.2) check for shard indexing pressure serialization. (#1398)

    Saurabh Singh - Thu, 21 Oct 2021 09:20:52 -0400

    This is required to not mandate test have the cluster service initialized while asserting node attributes. Signed-off-by: Saurabh Singh <sisurab@amazon.com> Co-authored-by: Saurabh Singh <sisurab@amazon.com>

  • [BUG] Fix org.opensearch.action.admin.cluster.node.stats.NodeStatsTests.testSerialization (#1399)

    Andriy Redko - Thu, 21 Oct 2021 09:16:15 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Run spotless and exclude checkstyle on server module #1380 (#1391)

    Owais Kazi - Wed, 20 Oct 2021 17:51:26 -0500

    Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

  • Minor fix for the flaky test to reduce concurrency (#1361) (#1397)

    Saurabh Singh - Wed, 20 Oct 2021 15:29:04 -0400

    Signed-off-by: Saurabh Singh <sisurab@amazon.com> Co-authored-by: Saurabh Singh <sisurab@amazon.com>

  • [Backport] [1.x] Clarified JDK usage in DEVELOPER_GUIDE.md, fixed minor issue with configuring runtime JDK (#1372)

    Andriy Redko - Wed, 20 Oct 2021 15:27:38 -0400

    • Clarified JDK usage in DEVELOPER_GUIDE.md, fixed minor issue with configuring runtime JDK Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

    • Fixed minor typos in DEVELOPER_GUIDE.md Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Upgrading netty version to 4.1.69.Final (#1363) (#1382)

    Vacha - Mon, 18 Oct 2021 22:05:45 -0400

    Signed-off-by: Vacha <vachshah@amazon.com>

  • Allow building on FreeBSD (#1091) (#1374)

    Romain Tartière - Sun, 17 Oct 2021 15:53:10 -0400

    • Allow building on FreeBSD With this set of change, we are able to successfuly run:
    ./gradlew publishToMavenLocal -Dbuild.snapshot=false

    This step is used in the OpenSearch repository context when building plugins in the current state of the CI. While here, reorder OS conditions alphabetically. Before building, the openjdk14 package was installed and the environment was adjusted to use it:

    sudo pkg install openjdk14
export JAVA_HOME=/usr/local/openjdk14/
export
PATH=$JAVA_HOME/bin:$PATH

    Signed-off-by: Romain Tartière <romain@blogreen.org>

  • Upgrade hadoop dependencies for hdfs plugin (#1335) (#1369)

    Vacha - Fri, 15 Oct 2021 08:56:48 -0400

  • Add Shard Level Indexing Pressure (#1336) (#1343)

    Saurabh Singh - Mon, 11 Oct 2021 12:14:18 -0700

    Shard level indexing pressure improves the current Indexing Pressure framework which performs memory accounting at node level and rejects the requests. This takes a step further to have rejections based on the memory accounting at shard level along with other key performance factors like throughput and last successful requests.

    Key features

    • Granular tracking of indexing tasks performance, at every shard level, for each node role i.e. coordinator, primary and replica.

    • Smarter rejections by discarding the requests intended only for problematic index or shard, while still allowing others to continue (fairness in rejection).

    • Rejections thresholds governed by combination of configurable parameters (such as memory limits on node) and dynamic parameters (such as latency increase, throughput degradation).

    • Node level and shard level indexing pressure statistics exposed through stats api.

    • Integration of Indexing pressure stats with Plugins for for metric visibility and auto-tuning in future.

    • Control knobs to tune to the key performance thresholds which control rejections, to address any specific requirement or issues.

    • Control knobs to run the feature in shadow-mode or enforced-mode. In shadow-mode only internal rejection breakdown metrics will be published while no actual rejections will be performed. The changes were divided into small manageable chunks as part of the following PRs against a feature branch.

    • Add Shard Indexing Pressure Settings. #716

    • Add Shard Indexing Pressure Tracker. #717

    • Refactor IndexingPressure to allow extension. #718

    • Add Shard Indexing Pressure Store #838

    • Add Shard Indexing Pressure Memory Manager #945

    • Add ShardIndexingPressure framework level construct and Stats #1015

    • Add Indexing Pressure Service which acts as orchestrator for IP #1084

    • Add plumbing logic for IndexingPressureService in Transport Actions. #1113

    • Add shard indexing pressure metric/stats via rest end point. #1171

    • Add shard indexing pressure integration tests. #1198 Signed-off-by: Saurabh Singh <sisurab@amazon.com> Co-authored-by: Saurabh Singh <sisurab@amazon.com> Co-authored-by: Rabi Panda <adnapibar@gmail.com>

  • Replace securemock with mock-maker (test support), update Mockito to 3.12.4 (#1332) (#1354)

    Andriy Redko - Mon, 11 Oct 2021 11:29:50 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • [BUG] ConcurrentSnapshotsIT#testAssertMultipleSnapshotsAndPrimaryFailOver fails intermittently (#1311) (#1322)

    Andriy Redko - Thu, 30 Sep 2021 19:59:11 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Support for Heap after GC stats (#1265) (#1309)

    Andriy Redko - Wed, 29 Sep 2021 17:20:06 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Switch to newSnapshot testing method to use helper

    Nicholas Walter Knize - Sun, 26 Sep 2021 00:00:35 -0500

    Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

  • [Revert] Translog Pruning Setting Deprecation Removal (#1294)

    Nick Knize - Sat, 25 Sep 2021 23:04:26 -0500

    INDEX_PLUGINS_REPLICATION_TRANSLOG_RETENTION_LEASE_PRUNING_ENABLED_SETTING was

    deprecated, then deprecation was removed. This adds deprecation back so that the setting can be moved to the plugin in the next minor release. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

  • Update Jackson to 2.12.5 (#1247) (#1270)

    Andriy Redko - Wed, 22 Sep 2021 10:52:55 -0400

    Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

  • Add support to generate code coverage report with JaCoCo (#1236)

    Tianli Feng - Mon, 20 Sep 2021 11:06:35 -0700

    • Apply JaCoCo Gradle plugin to the root project, and to all the sub-projects through BuildPlugin.
    • Add 3 Gradle tasks: codeCoverageReport, codeCoverageReportForUnitTest, codeCoverageReportForIntegrationTest to generate code coverage report for all the tests, unit tests, and integration tests correspondingly.
    • Attach Gradle codeCoverageReport task to check task.
    • Remove outdated code of giving JaCoCo files permission when Java security manager enabled Signed-off-by: Tianli Feng <ftianli@amazon.com>

  • [Version] Increment 1.x to 1.2 (#1239)

    Nick Knize - Tue, 14 Sep 2021 06:48:05 -0400

    Signed-off-by: Nicholas Walter Knize <nknize@apache.org>