Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit file handles used via remote_segments open #6023

Closed
jcsp opened this issue Aug 15, 2022 · 2 comments · Fixed by #7042
Closed

Limit file handles used via remote_segments open #6023

jcsp opened this issue Aug 15, 2022 · 2 comments · Fixed by #7042
Labels
area/cloud-storage Shadow indexing subsystem kind/enhance New feature or request

Comments

@jcsp
Copy link
Contributor

jcsp commented Aug 15, 2022

Goal is to ensure that even if SI cache is large and client does a very large number of random reads that prompt segments to be opened, we impose a bound on the total number of file handles used by the process.

@jcsp
Copy link
Contributor Author

jcsp commented Sep 13, 2022

Related: #6112

That PR adds a per-partition limit. Not as good as a global limit, but an improvement on the previous situation of having no limit at all + hitting bad_alloc.

@jcsp
Copy link
Contributor Author

jcsp commented Nov 3, 2022

This is covered by #7042 -- we are mainly limiting the reader count for memory, but it'll help file handles too.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cloud-storage Shadow indexing subsystem kind/enhance New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant