Skip to content

Commit

Permalink
Use java.lang.ref.Cleaner to close cloned IndexInputs (opensearch-pro…
Browse files Browse the repository at this point in the history
…ject#6351)

As detailed [in this issue][1], Lucene does not close the cloned
IndexInput instances, so we are using the Cleaner mechanism from the JDK
to close any unclosed clones.

A single static Cleaner instance to ensure any unclosed clone of an
IndexInput is closed. This instance creates a single daemon thread on
which it performs the cleaning actions. For an already-closed
IndexInput, the cleaning action is a no-op. For an open IndexInput, the
close action will decrement a reference count.

[1]: opensearch-project#5243 (comment)

Signed-off-by: Andrew Ross <andrross@amazon.com>
  • Loading branch information
andrross committed Feb 17, 2023
1 parent c772596 commit dae1566
Show file tree
Hide file tree
Showing 6 changed files with 331 additions and 74 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
*/
package org.opensearch.snapshots;

import com.carrotsearch.randomizedtesting.annotations.ThreadLeakFilters;
import com.carrotsearch.randomizedtesting.generators.RandomPicks;
import org.hamcrest.MatcherAssert;
import org.opensearch.action.admin.cluster.node.stats.NodesStatsResponse;
Expand All @@ -27,21 +28,23 @@
import org.opensearch.common.util.FeatureFlags;
import org.opensearch.index.Index;
import org.opensearch.index.IndexNotFoundException;
import org.opensearch.index.store.remote.file.CleanerDaemonThreadLeakFilter;
import org.opensearch.monitor.fs.FsInfo;
import org.opensearch.repositories.fs.FsRepository;

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;

import static org.hamcrest.Matchers.contains;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.greaterThan;
import static org.hamcrest.Matchers.is;
import static org.hamcrest.Matchers.contains;
import static org.hamcrest.Matchers.notNullValue;
import static org.opensearch.action.admin.cluster.node.stats.NodesStatsRequest.Metric.FS;
import static org.opensearch.common.util.CollectionUtils.iterableAsArrayList;

@ThreadLeakFilters(filters = CleanerDaemonThreadLeakFilter.class)
public final class SearchableSnapshotIT extends AbstractSnapshotIntegTestCase {

@Override
Expand Down
Loading

0 comments on commit dae1566

Please sign in to comment.