[BUG] Cleanup unreferenced files when any operation failed due to disk full.

[RS146BIJAY]

Describe the bug

As of now, we cleanup unreferenced files whenever last write is performed by merge and it caused disk to get full and shard to fail. Incase some other operation performs the last write and caused disk to get 100% full and a merge is ongoing, merge will just get aborted and no cleanup will be performed. Since when closing the shard it is the other operation causing disk full and not segment merge.

Related component

Storage

To Reproduce

There is no specific way in which we can reproduce this issue. Got to know about this from a flaky test case. In order to specifically reproduce this, we need some way to allow the merge and some other operation to happen in parallel. It should be the other operation (other than merge) which should cause disk to go 100% full.

Expected behavior

Cleanup unreferenced files when any operation failed due to disk full.

Additional Details

No response

[peternied]

[Triage - attendees 1 2 3 4 5 6 7 8]
@RS146BIJAY Thanks for filing, we'd gladly review a pull request to address this issue.

[RS146BIJAY]

Raised PR for this change: #12128

[shourya035]

@RS146BIJAY Feel free to close this issue or re-raise the PR if you are still working on this