s3 multipart upload something has issue

[cometta]

Problem description

i tested with open("s3://","wb") to upload many files each one with size above 400mb into s3. Sometime it fail with below error
botocore.exceptions.ClientError: An error occurred (NoSuchUpload) when calling the CompleteMultipartUpload operation: The specified multipart upload does not exist. The upload ID might be invalid, or the multipart upload might have been aborted or completed

and if i reupload it, it will be successful. Anyone else experience this? It only happen for multipart files, if the file is small size then no issue at all.

i'm using smart_open[all]==4.2.0

[mpenkov]

Please post a reproducible example. Without it, we can't help you.

[ddelange]

We also encounter this error very rarely after 100k+ sequential writes to s3. Fix seems to retry completing the multipart upload when this error occurs: peak/s5cmd#470

botocore.errorfactory.NoSuchUpload: An error occurred (NoSuchUpload) when calling the CompleteMultipartUpload op eration: The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted  or completed.