-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added information about Lambda Interruption #199
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the submission, @halilduygulu!
Do you have a linkable source source for this (even AWS support comment forums-level)?
@@ -1055,6 +1055,7 @@ Lambda | |||
- 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. | |||
- 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. | |||
- 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. | |||
- ❗Currently (October,2016) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggestion: "(as of October, 2016)"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks
@@ -1055,6 +1055,7 @@ Lambda | |||
- 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. | |||
- 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. | |||
- 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. | |||
- ❗Currently (October,2016) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not 100% clear on what "for failure recovery purposes" means - is it 2-3 minutes while recovering from failure?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As i sad it was a support ticket response. I don't have link to it because it is in our account, but i can copy paste the exact response here.
I've received an update from the engineering team.
They replied that customer may see their stream stop processing for three minute or less very infrequently due to certain failure recovery modes in our system. Their stream processing should then resume and recover from the delay.
They are working improvements to our system that should eliminate these sorts of pauses, we don't have a solid ETA on when that will be available.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also it happened to my 2 functions 4 times in total last week.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a huge difference between "Lambda functions can stop working" and "streams can stop processing" though. This should belong in the Kinesis section then, imo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an excellent addition, thank you @halilduygulu - a few requests related to the undocumented nature of the behavior.
@@ -1055,6 +1055,7 @@ Lambda | |||
- 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. | |||
- 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. | |||
- 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. | |||
- ❗Currently (as of October, 2016) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this isn't documented officially, would you mind doing the following?
- Adding a "undocumented" scroll icon after the serious gotcha red exclamation point? (📜)
- Convert "October, 2016" to be a link pointing to your support ticket text? ( https://github.com/open-guides/og-aws/pull/199/files/c99bddb4ee2437587f1e188d47be2bb1da01f81d#r83529126 )
@ThanosBaskous I made changes, could you check again. Thanks for reviews. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fixes, @halilduygulu! I have one more request to change something that was in the original review - sorry that I didn't catch it the first time around.
@@ -1055,6 +1055,7 @@ Lambda | |||
- 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. | |||
- 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. | |||
- 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. | |||
- ❗📜 Currently [as of October, 2016](https://github.com/open-guides/og-aws/pull/199/files/c99bddb4ee2437587f1e188d47be2bb1da01f81d#r83529126) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please replace "dev" with "development.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done and thanks for accepting.
Thanks @halilduygulu! Since this doesn't have a topic, assigning to @jlevy for secondary review. |
👍 Thanks! |
Added information about Lambda Interruption which happened on my project this week. The information comes from aws support ticket and lambda engineering team according to support staff.