ci: move mirrors to their standalone bucket #63485

pietroalbini · 2019-08-12T08:03:46Z

Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with
CI toolchains. This is problematic for multiple reasons:

CI IAM credentials are allowed to both edit and delete those files.
A malicious user gaining access to those credentials would be able to
change our mirrored dependencies, possibly compromising the compiler.
Contents of the rust-lang-ci2 bucket are disposable except for the
mirrors' content. When we implement backups for S3 buckets we'd have
to replicate just that part of the bucket, complicating the backup
logic and increasing the chance of mistakes. A standalone bucket will
be way easier to backup.

This commit switches our CI to use the new rust-lang-ci-mirrors bucket.

r? @alexcrichton

Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with CI toolchains. This is problematic for multiple reasons: - CI IAM credentials are allowed to both edit and delete those files. A malicious user gaining access to those credentials would be able to change our mirrored dependencies, possibly backdooring the compiler. - Contents of the rust-lang-ci2 bucket are disposable except for the mirrors' content. When we implement backups for S3 buckets we'd have to replicate just that part of the bucket, complicating the backup logic and increasing the chance of mistakes. A standalone bucket will be way easier to backup. This commit switches our CI to use the new rust-lang-ci-mirrors bucket.

alexcrichton · 2019-08-12T14:26:29Z

@bors: r+

bors · 2019-08-12T14:26:30Z

📌 Commit eb832b2 has been approved by alexcrichton

pietroalbini · 2019-08-12T15:49:22Z

On second thought, let's not backport this to 1.37: promotion is happening really soon, and I don't want anything breaking. Let's get it in in 1.38 though.

@bors p=10

bors · 2019-08-12T16:32:36Z

⌛ Testing commit eb832b2 with merge eba9bae...

@alexcrichton

ci: move mirrors to their standalone bucket Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with CI toolchains. This is problematic for multiple reasons: - CI IAM credentials are allowed to both edit and delete those files. A malicious user gaining access to those credentials would be able to change our mirrored dependencies, possibly compromising the compiler. - Contents of the rust-lang-ci2 bucket are disposable except for the mirrors' content. When we implement backups for S3 buckets we'd have to replicate just that part of the bucket, complicating the backup logic and increasing the chance of mistakes. A standalone bucket will be way easier to backup. This commit switches our CI to use the new rust-lang-ci-mirrors bucket. r? @alexcrichton

bors · 2019-08-12T19:13:41Z

💔 Test failed - checks-azure

rust-highfive · 2019-08-12T19:13:42Z

The job x86_64-msvc-cargo of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-12T16:33:55.5690828Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-12T16:33:55.5690972Z 
2019-08-12T16:33:55.5691106Z   git checkout -b <new-branch-name>
2019-08-12T16:33:55.5691438Z 
2019-08-12T16:33:55.5691607Z HEAD is now at eba9bae19 Auto merge of #63485 - pietroalbini:new-mirror-bucket, r=alexcrichton
2019-08-12T16:33:55.6060858Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-12T16:33:55.6168268Z ==============================================================================
2019-08-12T16:33:55.6168374Z Task         : Bash
2019-08-12T16:33:55.6168441Z Description  : Run a Bash script on macOS, Linux, or Windows

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

pietroalbini · 2019-08-12T19:22:29Z

@bors retry

Spurious crates.io download failure (expired cert)

bors · 2019-08-12T19:27:21Z

⌛ Testing commit eb832b2 with merge debcee23ac879e9ade0a93dac60f28f641cd2b23...

Mark-Simulacrum · 2019-08-12T19:41:50Z

@bors retry prioritize stable

@alexcrichton

…lexcrichton ci: move mirrors to their standalone bucket Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with CI toolchains. This is problematic for multiple reasons: - CI IAM credentials are allowed to both edit and delete those files. A malicious user gaining access to those credentials would be able to change our mirrored dependencies, possibly compromising the compiler. - Contents of the rust-lang-ci2 bucket are disposable except for the mirrors' content. When we implement backups for S3 buckets we'd have to replicate just that part of the bucket, complicating the backup logic and increasing the chance of mistakes. A standalone bucket will be way easier to backup. This commit switches our CI to use the new rust-lang-ci-mirrors bucket. r? @alexcrichton

@ghost

Rollup of 11 pull requests Successful merges: - #62760 (Deduplicate error messages in `librsctc_mir`) - #62849 (typeck: Prohibit RPIT types that inherit lifetimes) - #63383 (`async fn` lifetime elision tests) - #63421 (Implement Clone, Display for ascii::EscapeDefault) - #63459 (syntax: account for CVarArgs being in the argument list.) - #63475 (Bring back suggestion for splitting `<-` into `< -`) - #63485 (ci: move mirrors to their standalone bucket) - #63486 (Document `From` trait for `BinaryHeap`) - #63488 (improve DiagnosticBuilder docs) - #63493 (Remove unneeded comment in src/libcore/hash/mod.rs) - #63499 (handle elision in async fn correctly) Failed merges: r? @ghost

@alexcrichton

…lexcrichton ci: move mirrors to their standalone bucket Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with CI toolchains. This is problematic for multiple reasons: - CI IAM credentials are allowed to both edit and delete those files. A malicious user gaining access to those credentials would be able to change our mirrored dependencies, possibly compromising the compiler. - Contents of the rust-lang-ci2 bucket are disposable except for the mirrors' content. When we implement backups for S3 buckets we'd have to replicate just that part of the bucket, complicating the backup logic and increasing the chance of mistakes. A standalone bucket will be way easier to backup. This commit switches our CI to use the new rust-lang-ci-mirrors bucket. r? @alexcrichton

pietroalbini · 2019-08-13T15:55:19Z

1.38 beta is happening right now and this is not going to land before it. Let's backport it instead.

@ghost

Rollup of 10 pull requests Successful merges: - #62760 (Deduplicate error messages in `librsctc_mir`) - #62849 (typeck: Prohibit RPIT types that inherit lifetimes) - #63383 (`async fn` lifetime elision tests) - #63421 (Implement Clone, Display for ascii::EscapeDefault) - #63459 (syntax: account for CVarArgs being in the argument list.) - #63475 (Bring back suggestion for splitting `<-` into `< -`) - #63485 (ci: move mirrors to their standalone bucket) - #63486 (Document `From` trait for `BinaryHeap`) - #63488 (improve DiagnosticBuilder docs) - #63493 (Remove unneeded comment in src/libcore/hash/mod.rs) Failed merges: r? @ghost

@alexcrichton

…lexcrichton ci: move mirrors to their standalone bucket Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with CI toolchains. This is problematic for multiple reasons: - CI IAM credentials are allowed to both edit and delete those files. A malicious user gaining access to those credentials would be able to change our mirrored dependencies, possibly compromising the compiler. - Contents of the rust-lang-ci2 bucket are disposable except for the mirrors' content. When we implement backups for S3 buckets we'd have to replicate just that part of the bucket, complicating the backup logic and increasing the chance of mistakes. A standalone bucket will be way easier to backup. This commit switches our CI to use the new rust-lang-ci-mirrors bucket. r? @alexcrichton

@alexcrichton

…lexcrichton ci: move mirrors to their standalone bucket Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with CI toolchains. This is problematic for multiple reasons: - CI IAM credentials are allowed to both edit and delete those files. A malicious user gaining access to those credentials would be able to change our mirrored dependencies, possibly compromising the compiler. - Contents of the rust-lang-ci2 bucket are disposable except for the mirrors' content. When we implement backups for S3 buckets we'd have to replicate just that part of the bucket, complicating the backup logic and increasing the chance of mistakes. A standalone bucket will be way easier to backup. This commit switches our CI to use the new rust-lang-ci-mirrors bucket. r? @alexcrichton

@alexcrichton

…lexcrichton ci: move mirrors to their standalone bucket Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with CI toolchains. This is problematic for multiple reasons: - CI IAM credentials are allowed to both edit and delete those files. A malicious user gaining access to those credentials would be able to change our mirrored dependencies, possibly compromising the compiler. - Contents of the rust-lang-ci2 bucket are disposable except for the mirrors' content. When we implement backups for S3 buckets we'd have to replicate just that part of the bucket, complicating the backup logic and increasing the chance of mistakes. A standalone bucket will be way easier to backup. This commit switches our CI to use the new rust-lang-ci-mirrors bucket. r? @alexcrichton

@ghost

Rollup of 17 pull requests Successful merges: - #62760 (Deduplicate error messages in `librsctc_mir`) - #62849 (typeck: Prohibit RPIT types that inherit lifetimes) - #63383 (`async fn` lifetime elision tests) - #63421 (Implement Clone, Display for ascii::EscapeDefault) - #63459 (syntax: account for CVarArgs being in the argument list.) - #63475 (Bring back suggestion for splitting `<-` into `< -`) - #63485 (ci: move mirrors to their standalone bucket) - #63486 (Document `From` trait for `BinaryHeap`) - #63488 (improve DiagnosticBuilder docs) - #63493 (Remove unneeded comment in src/libcore/hash/mod.rs) - #63499 (handle elision in async fn correctly) - #63501 (use `ParamName` to track in-scope lifetimes instead of Ident) - #63508 (Do not ICE when synthesizing spans falling inside unicode chars) - #63511 (ci: add a check for clock drift) - #63512 (Provide map_ok and map_err method for Poll<Option<Result<T, E>>>) - #63529 (RELEASES.md: ? is one of three Kleene operators) - #63530 (Fix typo in error message.) Failed merges: r? @ghost

@ghost

Rollup of 17 pull requests Successful merges: - #62760 (Deduplicate error messages in `librsctc_mir`) - #62849 (typeck: Prohibit RPIT types that inherit lifetimes) - #63383 (`async fn` lifetime elision tests) - #63421 (Implement Clone, Display for ascii::EscapeDefault) - #63459 (syntax: account for CVarArgs being in the argument list.) - #63475 (Bring back suggestion for splitting `<-` into `< -`) - #63485 (ci: move mirrors to their standalone bucket) - #63486 (Document `From` trait for `BinaryHeap`) - #63488 (improve DiagnosticBuilder docs) - #63493 (Remove unneeded comment in src/libcore/hash/mod.rs) - #63499 (handle elision in async fn correctly) - #63501 (use `ParamName` to track in-scope lifetimes instead of Ident) - #63508 (Do not ICE when synthesizing spans falling inside unicode chars) - #63511 (ci: add a check for clock drift) - #63512 (Provide map_ok and map_err method for Poll<Option<Result<T, E>>>) - #63529 (RELEASES.md: ? is one of three Kleene operators) - #63530 (Fix typo in error message.) Failed merges: r? @ghost

@ghost

[beta] Rollup backports Cherry-picked: * RELEASES.md: ? is one of three Kleene operators #63529 * Do not ICE when synthesizing spans falling inside unicode chars #63508 * ci: move mirrors to their standalone bucket #63485 r? @ghost

@alexcrichton

…ichton ci: move libc mirrors to the rust-lang-ci-mirrors bucket Finishing up rust-lang#63485. Already moved the objects. r? @alexcrichton

pietroalbini added beta-nominated Nominated for backporting to the compiler in the beta channel. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Aug 12, 2019

rust-highfive assigned alexcrichton Aug 12, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 12, 2019

This comment has been minimized.

Sign in to view

pietroalbini force-pushed the new-mirror-bucket branch from fb127a1 to eb832b2 Compare August 12, 2019 08:31

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 12, 2019

alexcrichton added the beta-accepted Accepted for backporting to the compiler in the beta channel. label Aug 12, 2019

pietroalbini removed T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. beta-accepted Accepted for backporting to the compiler in the beta channel. beta-nominated Nominated for backporting to the compiler in the beta channel. labels Aug 12, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 12, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 12, 2019

Centril mentioned this pull request Aug 12, 2019

Rollup of 3 pull requests #63503

Closed

Centril mentioned this pull request Aug 13, 2019

Rollup of 11 pull requests #63515

Closed

Centril mentioned this pull request Aug 13, 2019

Rollup of 10 pull requests #63517

Closed

pietroalbini added beta-accepted Accepted for backporting to the compiler in the beta channel. beta-nominated Nominated for backporting to the compiler in the beta channel. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Aug 13, 2019

Mark-Simulacrum mentioned this pull request Aug 14, 2019

Rollup of 11 pull requests #63540

Closed

Centril mentioned this pull request Aug 14, 2019

Rollup of 16 pull requests #63541

Closed

Centril mentioned this pull request Aug 14, 2019

Rollup of 17 pull requests #63544

Merged

bors merged commit eb832b2 into rust-lang:master Aug 14, 2019

pietroalbini deleted the new-mirror-bucket branch August 20, 2019 21:47

pietroalbini mentioned this pull request Aug 21, 2019

[beta] Rollup backports #63771

Merged

pietroalbini removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Aug 21, 2019

pietroalbini mentioned this pull request Aug 21, 2019

ci: move libc mirrors to the rust-lang-ci-mirrors bucket #63772

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: move mirrors to their standalone bucket #63485

ci: move mirrors to their standalone bucket #63485

pietroalbini commented Aug 12, 2019 •

edited

Loading

This comment has been minimized.

alexcrichton commented Aug 12, 2019

bors commented Aug 12, 2019

pietroalbini commented Aug 12, 2019

bors commented Aug 12, 2019

bors commented Aug 12, 2019

rust-highfive commented Aug 12, 2019

pietroalbini commented Aug 12, 2019

bors commented Aug 12, 2019

Mark-Simulacrum commented Aug 12, 2019

pietroalbini commented Aug 13, 2019

ci: move mirrors to their standalone bucket #63485

ci: move mirrors to their standalone bucket #63485

Conversation

pietroalbini commented Aug 12, 2019 • edited Loading

This comment has been minimized.

alexcrichton commented Aug 12, 2019

bors commented Aug 12, 2019

pietroalbini commented Aug 12, 2019

bors commented Aug 12, 2019

bors commented Aug 12, 2019

rust-highfive commented Aug 12, 2019

pietroalbini commented Aug 12, 2019

bors commented Aug 12, 2019

Mark-Simulacrum commented Aug 12, 2019

pietroalbini commented Aug 13, 2019

pietroalbini commented Aug 12, 2019 •

edited

Loading