From 71b3cbc0fd03b01389ffdf166fe65a8ccdc8d226 Mon Sep 17 00:00:00 2001 From: Halil Duygulu Date: Fri, 14 Oct 2016 10:24:07 +0300 Subject: [PATCH 1/6] Added some Redshift tips. --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 2fac26fa..599203a5 100644 --- a/README.md +++ b/README.md @@ -1252,6 +1252,9 @@ Redshift - ❗ Never resize a live cluster. The resize operation takes hours depending on the dataset size. In rare cases, the operation may also get stuck and you'll end up having a non-functional cluster. The safer approach is to create a new cluster from a snapshot, resize the new cluster and shut down the old one. - Redshift has reserved keywords which are not present in Postgres (see full list [here](https://docs.aws.amazon.com/redshift/latest/dg/r_pg_keywords.html)). Watch out for DELTA ([Delta Encodings](https://docs.aws.amazon.com/redshift/latest/dg/c_Delta_encoding.html)). - Redshift does not support many Postgres functions, most notably several date/time-related and aggregation functions. See the [full list here](https://docs.aws.amazon.com/redshift/latest/dg/c_unsupported-postgresql-functions.html). +- 🔹 If you need to change sort key or dist key of a table, you need to create a new table with new key and move your data to new table with insert into new_table select * from old_table. [Choosing Sort Key](http://docs.aws.amazon.com/redshift/latest/dg/t_Sorting_data.html) +- ❗🚪 When moving data with insert into x select from y; you need to have at least twice of y table size free space on your harddisk. Redshift first copies data to disk then to new table. +- 🔹 Vacuum delete only does not block copy commands, but vacuum reindex can block if you execute copy command every minute or so. EMR --- From d433e5c62c7a14b454cda8af3e31aeb6fe795eea Mon Sep 17 00:00:00 2001 From: Halil Duygulu Date: Fri, 14 Oct 2016 11:15:04 +0300 Subject: [PATCH 2/6] Lambda stopping issue added. --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 599203a5..cee2890d 100644 --- a/README.md +++ b/README.md @@ -1055,6 +1055,7 @@ Lambda - 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. - 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. - 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. +- ❗Currently (October,2016) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. 🚧 [*Please help expand this incomplete section.*](CONTRIBUTING.md) From c99bddb4ee2437587f1e188d47be2bb1da01f81d Mon Sep 17 00:00:00 2001 From: Halil Duygulu Date: Fri, 14 Oct 2016 11:28:32 +0300 Subject: [PATCH 3/6] Lambda stopping issue added. --- README.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/README.md b/README.md index cee2890d..18829e9f 100644 --- a/README.md +++ b/README.md @@ -1253,9 +1253,6 @@ Redshift - ❗ Never resize a live cluster. The resize operation takes hours depending on the dataset size. In rare cases, the operation may also get stuck and you'll end up having a non-functional cluster. The safer approach is to create a new cluster from a snapshot, resize the new cluster and shut down the old one. - Redshift has reserved keywords which are not present in Postgres (see full list [here](https://docs.aws.amazon.com/redshift/latest/dg/r_pg_keywords.html)). Watch out for DELTA ([Delta Encodings](https://docs.aws.amazon.com/redshift/latest/dg/c_Delta_encoding.html)). - Redshift does not support many Postgres functions, most notably several date/time-related and aggregation functions. See the [full list here](https://docs.aws.amazon.com/redshift/latest/dg/c_unsupported-postgresql-functions.html). -- 🔹 If you need to change sort key or dist key of a table, you need to create a new table with new key and move your data to new table with insert into new_table select * from old_table. [Choosing Sort Key](http://docs.aws.amazon.com/redshift/latest/dg/t_Sorting_data.html) -- ❗🚪 When moving data with insert into x select from y; you need to have at least twice of y table size free space on your harddisk. Redshift first copies data to disk then to new table. -- 🔹 Vacuum delete only does not block copy commands, but vacuum reindex can block if you execute copy command every minute or so. EMR --- From 994ca7b864e0f7f26df773d64795bfffc01d727e Mon Sep 17 00:00:00 2001 From: Halil Duygulu Date: Sat, 15 Oct 2016 12:45:49 +0300 Subject: [PATCH 4/6] date typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 18829e9f..e6715318 100644 --- a/README.md +++ b/README.md @@ -1055,7 +1055,7 @@ Lambda - 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. - 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. - 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. -- ❗Currently (October,2016) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. +- ❗Currently (as of October, 2016) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. 🚧 [*Please help expand this incomplete section.*](CONTRIBUTING.md) From 25df5050517c24998bc8b548b7782bd20d32ba27 Mon Sep 17 00:00:00 2001 From: Halil Duygulu Date: Mon, 17 Oct 2016 09:00:27 +0300 Subject: [PATCH 5/6] Added link and undocumented icon. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e6715318..af43952c 100644 --- a/README.md +++ b/README.md @@ -1055,7 +1055,7 @@ Lambda - 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. - 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. - 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. -- ❗Currently (as of October, 2016) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. +- ❗📜 Currently [as of October, 2016](https://github.com/open-guides/og-aws/pull/199/files/c99bddb4ee2437587f1e188d47be2bb1da01f81d#r83529126) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. 🚧 [*Please help expand this incomplete section.*](CONTRIBUTING.md) From cc1c153098a88fc06a2922a6b8faa023a0705c7c Mon Sep 17 00:00:00 2001 From: Halil Duygulu Date: Tue, 18 Oct 2016 09:36:45 +0300 Subject: [PATCH 6/6] dev -> development --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index af43952c..e6b77d2f 100644 --- a/README.md +++ b/README.md @@ -1055,7 +1055,7 @@ Lambda - 🔸Lambda is a new technology. As of mid 2016, only a few companies are using it for large-scale production applications. - 🔸Managing lots of Lambda functions is a workflow challenge, and tooling to manage Lambda deployments is still immature. - 🔸AWS’ official workflow around managing function [versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html) is painful. -- ❗📜 Currently [as of October, 2016](https://github.com/open-guides/og-aws/pull/199/files/c99bddb4ee2437587f1e188d47be2bb1da01f81d#r83529126) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda dev team. They are working to prevent this in the future. +- ❗📜 Currently [as of October, 2016](https://github.com/open-guides/og-aws/pull/199/files/c99bddb4ee2437587f1e188d47be2bb1da01f81d#r83529126) Lambda functions can sometimes stop working for 2-3 minutes for failure recovery purposes according to a support ticket answer from Lambda development team. They are working to prevent this in the future. 🚧 [*Please help expand this incomplete section.*](CONTRIBUTING.md)