[Remove] Deprecated Zen1 Discovery #1216

nknize · 2021-09-03T21:16:18Z

Zen1 discovery was deprecated in Legacy 7.x for eventual removal. OpenSearch 1.x
carries this deprecation. This PR completely removes all support for Zen1
discovery in favor of Zen2 in the 2.x versioning line.

nknize · 2021-09-03T21:16:51Z

start gradle check

opensearch-ci-bot · 2021-09-03T21:21:47Z

✅ Gradle Wrapper Validation success ec3fc8607114d82a4b7e93e3599a7d051d020c68

opensearch-ci-bot · 2021-09-03T21:21:47Z

✅ DCO Check Passed ec3fc8607114d82a4b7e93e3599a7d051d020c68

opensearch-ci-bot · 2021-09-03T21:28:39Z

✅ Gradle Precommit success ec3fc8607114d82a4b7e93e3599a7d051d020c68

opensearch-ci-bot · 2021-09-03T21:54:24Z

✅ Gradle Check success ec3fc8607114d82a4b7e93e3599a7d051d020c68
Log 490

Reports 490

nknize · 2021-09-06T18:37:41Z

start gradle check

opensearch-ci-bot · 2021-09-06T18:43:06Z

✅ DCO Check Passed ed8702c61348985dbf84843b57120b07725b73a8

opensearch-ci-bot · 2021-09-06T18:43:06Z

✅ Gradle Wrapper Validation success ed8702c61348985dbf84843b57120b07725b73a8

opensearch-ci-bot · 2021-09-06T18:50:49Z

✅ Gradle Precommit success ed8702c61348985dbf84843b57120b07725b73a8

opensearch-ci-bot · 2021-09-06T19:05:22Z

❌ Gradle Check failure ed8702c61348985dbf84843b57120b07725b73a8
Log 492

Reports 492

nknize · 2021-09-06T21:36:22Z

Jenkins flakiness...

./gradlew :qa:full-cluster-restart:v1.1.0#oldClusterTest -Dtests.seed=B5D94F8B26901B69

» [2021-09-06T18:53:58.574874Z] [BUILD] Starting OpenSearch process
» [2021-09-06T18:54:02,473][INFO ][o.o.n.Node               ] [v1.1.0-1] version[1.1.0-SNAPSHOT], pid[248346], build[tar/ba9c8c08446b51bd3b6f9dacdecea149478b0ed2/2021-09-06T18:43:08.660107Z], OS[Linux/5.4.0-1045-aws/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
» [2021-09-06T18:54:02,481][INFO ][o.o.n.Node               ] [v1.1.0-1] JVM home [/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/search/distribution/bwc/minor/build/bwc/checkout-1.x/distribution/archives/linux-tar/build/install/opensearch-1.1.0-SNAPSHOT/jdk], using bundled JDK [true]
» [2021-09-06T18:54:02,482][INFO ][o.o.n.Node               ] [v1.1.0-1] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/search/qa/full-cluster-restart/build/testclusters/v1.1.0-1/tmp, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=logs, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms512m, -Xmx512m, -ea, -esa, -XX:TieredStopAtLevel=1, -XX:MaxDirectMemorySize=268435456, -Dopensearch.path.home=/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/search/distribution/bwc/minor/build/bwc/checkout-1.x/distribution/archives/linux-tar/build/install/opensearch-1.1.0-SNAPSHOT, -Dopensearch.path.conf=/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/search/qa/full-cluster-restart/build/testclusters/v1.1.0-1/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
» [2021-09-06T18:54:07,155][INFO ][o.o.i.r.ReindexPlugin    ] [v1.1.0-1] ReindexPlugin reloadSPI called
» [2021-09-06T18:54:07,167][INFO ][o.o.i.r.ReindexPlugin    ] [v1.1.0-1] Unable to find any implementation for RemoteReindexExtension
...
> Task :qa:full-cluster-restart:v1.1.0#oldClusterTest FAILED

Unable to reproduce:

BUILD SUCCESSFUL in 3m 10s
161 actionable tasks: 8 executed, 153 up-to-date

Restarting check...

nknize · 2021-09-06T21:36:28Z

start gradle check

nknize · 2021-09-06T21:46:33Z

More link checker flakiness...

opensearch-ci-bot · 2021-09-06T22:12:59Z

✅ Gradle Check success ed8702c61348985dbf84843b57120b07725b73a8
Log 493

Reports 493

opensearch-ci-bot · 2021-09-29T21:21:18Z

Can one of the admins verify this patch?

opensearch-ci-bot · 2021-10-26T15:04:34Z

✅ Gradle Wrapper Validation success ed8702c61348985dbf84843b57120b07725b73a8

opensearch-ci-bot · 2021-10-26T15:05:13Z

✅ DCO Check Passed ed8702c61348985dbf84843b57120b07725b73a8

opensearch-ci-bot · 2021-10-26T15:32:24Z

✅ Gradle Precommit success ed8702c61348985dbf84843b57120b07725b73a8

opensearch-ci-bot · 2021-12-02T22:29:45Z

✅ Gradle Wrapper Validation success 21ae020f92f3655280be614aef2c920446fe6330

opensearch-ci-bot · 2021-12-02T22:38:25Z

✅ Gradle Precommit success 21ae020f92f3655280be614aef2c920446fe6330

opensearch-ci-bot · 2021-12-02T23:15:01Z

❌ Gradle Check failure 21ae020f92f3655280be614aef2c920446fe6330
Log 1308

Reports 1308

Zen1 discovery was deprecated in Legacy 7.x for eventual removal. OpenSearch 1.x carries this deprecation. This commit completely removes all support for Zen1 discovery in favor of Zen2. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

opensearch-ci-bot · 2021-12-02T23:32:26Z

✅ Gradle Wrapper Validation success 47ee204

opensearch-ci-bot · 2021-12-02T23:41:30Z

✅ Gradle Precommit success 47ee204

opensearch-ci-bot · 2021-12-03T00:13:09Z

✅ Gradle Check success 47ee204
Log 1309

Reports 1309

nknize · 2021-12-03T02:04:45Z

@opensearch-project/opensearch-core this is ready for review if anyone wants to have a look. Full zen1 removal which was deprecated in the fork.

adnapibar

LGTM - Thanks Nick!
Just a couple of comments. I also see a few more references of zen1, but I guess we can clean those up in a subsequent PR.

adnapibar · 2021-12-03T05:22:04Z

server/src/main/java/org/opensearch/cluster/coordination/Coordinator.java

@@ -670,10 +660,6 @@ void becomeCandidate(String method) {
            peerFinder.activate(coordinationState.get().getLastAcceptedState().nodes());
            clusterFormationFailureHelper.start();

-            if (getCurrentTerm() == ZEN1_BWC_TERM) {


Should we be completely removing this constant ? It's used here as well

good catch... pushing additional commit to remove all remaining references

Removing caused a node join failure so I'll take care of that in a follow up to isolate the issue.

Failing test:

./gradlew ':server:test' --tests "org.opensearch.node.NodeTests.testCloseOnLeakedIndexReaderReference" -Dtests.seed=47F5C424B97CB691 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=fr-BE -Dtests.timezone=Africa/Maputo -Druntime.java=17

server/src/main/java/org/opensearch/rest/action/admin/indices/RestResizeHandler.java

opensearch-ci-bot · 2021-12-03T16:54:30Z

✅ Gradle Wrapper Validation success d858ffa5f50a3e525bb1d46eda1f385a7f28a832

opensearch-ci-bot · 2021-12-03T17:02:04Z

✅ Gradle Wrapper Validation success 602ef840a4948083c353f72b63b1fd6fac3e16c1

opensearch-ci-bot · 2021-12-03T17:04:01Z

❌ Gradle Precommit failure d858ffa5f50a3e525bb1d46eda1f385a7f28a832
Log 1693

opensearch-ci-bot · 2021-12-03T17:05:29Z

✅ Gradle Precommit success 602ef840a4948083c353f72b63b1fd6fac3e16c1

opensearch-ci-bot · 2021-12-03T17:08:24Z

❌ Gradle Check failure d858ffa5f50a3e525bb1d46eda1f385a7f28a832
Log 1316

Reports 1316

opensearch-ci-bot · 2021-12-03T17:17:20Z

❌ Gradle Check failure 602ef840a4948083c353f72b63b1fd6fac3e16c1
Log 1317

Reports 1317

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

opensearch-ci-bot · 2021-12-03T17:44:39Z

✅ Gradle Wrapper Validation success 18324ac

opensearch-ci-bot · 2021-12-03T17:55:54Z

✅ Gradle Precommit success 18324ac

reta · 2021-12-03T18:01:31Z

server/src/main/java/org/opensearch/node/Node.java

@@ -293,6 +292,11 @@
    }, Setting.Property.NodeScope);

    private static final String CLIENT_TYPE = "node";
+    public static final Setting<TimeValue> INITIAL_STATE_TIMEOUT_SETTING = Setting.positiveTimeSetting(


That looks a bit off: why discovery setting on node? May be it is better keep old DiscoverySettings class with this single setting (for now)?

It's a NodeScope setting refactored from DiscoverySettings and the only remaining setting for node discovery so I think it's appropriately refactored here to node level without the overhead of an inner class?

Sorry, should have been more clear, NodeScope is totally fine, I think the pretty useful convention we have is that that Node settings are prefixed with node., Search settings with search., Index settings with index., the overhead of inner class is super minimal but it would be just natural to look for discovery. under Discovery - related classes, just my two cents.

oic what you're saying. API wise? sure we can make a public inner class. And I think "overhead" should be 1:1? Instead of loading DiscoverySettings.class (which is now removed) we're adding a new Node$DiscoverySettings.class.

👍 I refactored DiscoverySettings as an internal class to Node. I think that's a nice clean suggestion.

Thank you @nknize !

server/src/test/java/org/opensearch/action/admin/cluster/node/stats/NodeStatsTests.java

opensearch-ci-bot · 2021-12-03T18:26:27Z

✅ Gradle Check success 18324ac
Log 1318

Reports 1318

…rnal class Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

opensearch-ci-bot · 2021-12-03T20:44:25Z

✅ Gradle Wrapper Validation success a6f36a2

opensearch-ci-bot · 2021-12-03T20:55:55Z

✅ Gradle Precommit success a6f36a2

reta · 2021-12-03T20:56:27Z

server/src/main/java/org/opensearch/node/Node.java

@@ -294,6 +293,14 @@

    private static final String CLIENT_TYPE = "node";

+    public static class DiscoverySettings {


opensearch-ci-bot · 2021-12-03T21:24:33Z

✅ Gradle Check success a6f36a2
Log 1320

Reports 1320

nknize added v2.0.0 Version 2.0.0 distributed framework labels Sep 3, 2021

nknize force-pushed the remove/zen1 branch from ed8702c to 21ae020 Compare December 2, 2021 22:24

nknize requested a review from a team as a code owner December 2, 2021 22:24

[Remove] Deprecated Zen1 Discovery

47ee204

Zen1 discovery was deprecated in Legacy 7.x for eventual removal. OpenSearch 1.x carries this deprecation. This commit completely removes all support for Zen1 discovery in favor of Zen2. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

nknize force-pushed the remove/zen1 branch from 21ae020 to 47ee204 Compare December 2, 2021 23:30

adnapibar reviewed Dec 3, 2021

View reviewed changes

nknize requested a review from adnapibar December 3, 2021 17:04

restore rest resize handler

18324ac

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

nknize force-pushed the remove/zen1 branch from 602ef84 to 18324ac Compare December 3, 2021 17:42

reta reviewed Dec 3, 2021

View reviewed changes

server/src/test/java/org/opensearch/action/admin/cluster/node/stats/NodeStatsTests.java Show resolved Hide resolved

refactor INITIAL_STATE_TIMEOUT_SETTING to Node.DiscoverySettings inte…

a6f36a2

…rnal class Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

reta reviewed Dec 3, 2021

View reviewed changes

reta approved these changes Dec 3, 2021

View reviewed changes

dblock approved these changes Dec 3, 2021

View reviewed changes

nknize merged commit 4db97aa into opensearch-project:main Dec 3, 2021

adnapibar mentioned this pull request Dec 7, 2021

Remove LegacyVersion.V_6.x Constants and Deprecated Code in OpenSearch 2.0 #1674

Closed

19 tasks

owaiskazi19 mentioned this pull request Feb 3, 2022

[CI] Test failure in "org.opensearch.cluster.coordination.Zen1IT > testZen2NodesJoiningZen1Cluster" #2046

Closed

VachaShah mentioned this pull request Mar 2, 2022

[Backport 1.x] Avoid logging duplicate deprecation warnings multiple times #2315

Merged

saratvemulapalli mentioned this pull request Mar 15, 2022

[Documentation] Breaking changes in 2.0 #2480

Closed

10 tasks

		@@ -294,6 +293,14 @@

		private static final String CLIENT_TYPE = "node";

		public static class DiscoverySettings {

[Remove] Deprecated Zen1 Discovery #1216

[Remove] Deprecated Zen1 Discovery #1216

Conversation

nknize commented Sep 3, 2021

nknize commented Sep 3, 2021

opensearch-ci-bot commented Sep 3, 2021

opensearch-ci-bot commented Sep 3, 2021

opensearch-ci-bot commented Sep 3, 2021

opensearch-ci-bot commented Sep 3, 2021

nknize commented Sep 6, 2021

opensearch-ci-bot commented Sep 6, 2021

opensearch-ci-bot commented Sep 6, 2021

opensearch-ci-bot commented Sep 6, 2021

opensearch-ci-bot commented Sep 6, 2021

nknize commented Sep 6, 2021

nknize commented Sep 6, 2021

nknize commented Sep 6, 2021

opensearch-ci-bot commented Sep 6, 2021

opensearch-ci-bot commented Sep 29, 2021

opensearch-ci-bot commented Oct 26, 2021

opensearch-ci-bot commented Oct 26, 2021

opensearch-ci-bot commented Oct 26, 2021

opensearch-ci-bot commented Dec 2, 2021

opensearch-ci-bot commented Dec 2, 2021

opensearch-ci-bot commented Dec 2, 2021

opensearch-ci-bot commented Dec 2, 2021

opensearch-ci-bot commented Dec 2, 2021

opensearch-ci-bot commented Dec 3, 2021

nknize commented Dec 3, 2021

adnapibar left a comment

Choose a reason for hiding this comment

adnapibar Dec 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

reta Dec 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

opensearch-ci-bot commented Dec 3, 2021

Choose a reason for hiding this comment

opensearch-ci-bot commented Dec 3, 2021

adnapibar Dec 3, 2021 •

edited

Loading

reta Dec 3, 2021 •

edited

Loading