Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[aws-cpp-sdk-s3-crt] support for the exponential back-off retry strategy #2615

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

grrtrr
Copy link
Contributor

@grrtrr grrtrr commented Aug 2, 2023

The S3CrtClient not longer has a configurable retry strategy.

By default, the S3CrtClient uses the 'standard' AWS retry strategy with default parameters (5 maximum retries, scale factor of 25msec and default/full jitter options).

In porting from S3Client to S3CrtClient deployments, support for exponential back-off is necessary, in order to retry for a longer period.

This is based on the truncated exponential backoff in awslabs/aws-c-io#587.

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The `S3CrtClient` not longer has a configurable retry strategy.

By default, the `S3CrtClient` uses the 'standard' AWS retry strategy with
default parameters (5 maximum retries, scale factor of 25msec and default/full
jitter options).

In porting from `S3Client` to `S3CrtClient` deployments, support for
exponential back-off is necessary, als in order to retry for a longer
period.

This is based on the truncated exponential backoff in
awslabs/aws-c-io#587.
@jmklix
Copy link
Member

jmklix commented Aug 14, 2023

Can you add an enum flag to opt in to this new retry strategy?

@grrtrr
Copy link
Contributor Author

grrtrr commented Aug 14, 2023

@jmklix - how would that look like in the bigger picture of the remaining configuration?
For instance, setting exponentialBackoffTotalRetries to 0 disables retries.

@jmklix
Copy link
Member

jmklix commented Aug 15, 2023

It would look like this:

enum class AwsS3CrtRetryStrategy
  {
    DEFAULT=0, /* defaults to standard now but may change */
    STANDARD,
    EXPONENTIAL_BACKOFF,
    NO_RETRY=-1,
  };

@grrtrr
Copy link
Contributor Author

grrtrr commented Aug 15, 2023

It would look like this:

Ok thanks. It may take a few weeks to implement this, I don't have time to get into that at the moment.

@@ -480,6 +492,7 @@ void S3CrtClient::init(const S3Crt::ClientConfiguration& config,
{
aws_tls_connection_options_clean_up(&nonConstTlsOptions);
}
aws_retry_strategy_release(s3CrtConfig.retry_strategy);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aws_retry_strategy_release does nothing when given a NULL parameter, so safe to use with defaults.

@grrtrr
Copy link
Contributor Author

grrtrr commented Dec 27, 2023

@jmklix - sorry it has taken a while. I have added an enum flag as requested, specifically

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants