Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Windows provisioners stability #733

Merged
merged 9 commits into from
May 5, 2020
Merged

Improve Windows provisioners stability #733

merged 9 commits into from
May 5, 2020

Conversation

vmapetr
Copy link
Contributor

@vmapetr vmapetr commented Apr 16, 2020 •
edited
Loading

Description

This PR improves stability for following Windows provisioners:

  1. Install-CloudFoundryCli.ps1
  2. Install-Go.ps1
  3. Install-Kind.ps1
  4. Install-MysqlCli.ps1
  5. Install-Rust.ps1
  6. Install-SQLPowerShellTools.ps1
  7. Update-AndroidSDK.ps1

Most of default Invoke-WebRequest occurrences was replace with custom Start-DownloadWithRetry function that support retries logic. Also several improvements in syntax was made.

How it was tested:
Validation image was generated - Windows2019.

Related issue:

Issue: https://github.com/actions/virtual-environments-internal/issues/384

Check list

  • Related issue / work item is attached
  • Tests are written (if applicable)
  • Documentation is updated (if applicable)
  • Changes are tested and related VM images are successfully generated

@vmapetr vmapetr marked this pull request as ready for review April 16, 2020 16:46
@vmapetr vmapetr changed the title V mapetr/windows/add retries for provisioners pr Improve Windows provisioners stability Apr 16, 2020
@maxim-lobanov
Copy link
Contributor

/azp run windows2016, windows2019

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@maxim-lobanov maxim-lobanov merged commit 33f6bf1 into actions:master May 5, 2020
miketimofeev pushed a commit to miketimofeev/virtual-environments that referenced this pull request May 14, 2020
@vmapetr
Improve Windows provisioners stability (actions#733)
62f34bb 
* Improve Install-CloudFoundryCli.ps1

* Improve Install-Go.ps1

* Improve Install-Kind.ps1

* Improve Install-MysqlCli.ps1

* Improve Install-Rust.ps1

* Improve Install-SQLPowerShellTools.ps1

* Improve Update-AndroidSDK.ps1

* Fix issue with resolve
AlenaSviridenko added a commit that referenced this pull request May 26, 2020
@miketimofeev @bergmeister
@josteink @Darleev @AlenaSviridenko @vsafonkin @nikita-bykov @al-cheb @459680 @vmapetr @sima-zhu @phozzy @jthelin @dmitry-shibanov @simpleprovider @Ailrun @dibir-magomedsaygitov
Add Ubuntu 20 image (#883)
021e7bc 
* add ubuntu 20 yml

* add ubuntu 20.04

* add toolcache

* add config

* install ansible from oficial canonical repo

* remove azure-cli

* remove azure-devops-cli

* libcurl3 -> libcurl4

* libcurl4

* install python2

* install clang9 directly from apt-get

* remove swift

* fix libcurl in the right place

* remove docker-moby

* remove dotnetcore-sdk

* remove haskell

* remove azul jdk

* remove leingen

* remove php

* remove sbt

* fix lsb release

* remove mongodb

* remove >> for go installation

* install only python3

* remove pip version

* remove add-apt-repository -y ppa:apt-fast/stable

* remove android

* remove powershell

* remove python from tests

* add java back

* change heroku installation

* get back leingen

* get back android

* remove set -e from java installation

* get back apt-fast

* remove git-lfs install from packagecloud

* add pip installation for python2

* get back set -e to java-tools.sh

* change ansible installation

* add pip to python

* Revert "install clang9 directly from apt-get"

This reverts commit e7d965f.

* Revert "change heroku installation"

This reverts commit 4fb3c0c.

* Revert "remove php"

This reverts commit d64ce16.

* change CRLF to LF for php installation

* get back heroku to template

* get back haskell

* get back dotnet

* install pwsh via snap

* remove netcore 3.0 & 2.2

* add azpowershell

* Upgrade to Az 3.8

* Update Validate-AzureModules.ps1

* Fix formatting in doc-string for Rust for Windows-image.

Ensure output in ` images/win/Windows2019-Readme.md` looks proper.

[Live in master right now](https://github.com/actions/virtual-environments/blob/master/images/win/Windows2019-Readme.md#rust-64-bit), it looks kinda bad.

* added workaround for default DNS

* revert changes

* Don\'t create backup sed file (#792)

Co-authored-by: Sergey Dolin <v-sedoli@micorosoft.com>

* Updated README with new rules of tool installation. (#793)

* Update README with new rules of tool installation.

* fixed typo

* fix notes

* another minor fix

* fixed articles

* fixed license line

* Add Python toolcache installation from Github releases for Ubuntu (#704)

Change logic to install Python to Ubuntu images to use GitHub releases from https://github.com/actions/python-versions

* Software list: "do not change" label. (#796)

* added_advises

* change place for hided stirng.

* changed to more direct message.

* Fail fast in image generation if not enough disk space for customers (#767)

* added checking free disk space
Co-authored-by: Nikita Bykov <v-nibyko@microsoft.com>

* Add FS-iSCSITarget-Server (#805)

* Updating readme file for win19 version 20200426.1

* Add Python tools installation from Github releases for Windows (#705)

* Add Toolset provision for Windows
Co-authored-by: MaksimZhukov <v-mazhuk@microsoft.com>

* Updating readme file for macOS version 20200425.1 (#801)

* Updating readme file for win16 version 20200426.1

* change min free space to 18000Mb (#814)

* Added java 13 package in VM (#795)

* Added java 13 package in VM

* Fixed var name

* Make changes from 11 to 13

* added java 13 for validate-javatool

* Fix documentation for containertools

* pre-create .composer folder to preserve permission

* Fix minor glitch in installer message

echo "cmake is already installed"

* added workaround for default DNS (#819)

* workaround reverted

* Improve Windows provisioners stability (#733)

* Improve Install-CloudFoundryCli.ps1

* Improve Install-Go.ps1

* Improve Install-Kind.ps1

* Improve Install-MysqlCli.ps1

* Improve Install-Rust.ps1

* Improve Install-SQLPowerShellTools.ps1

* Improve Update-AndroidSDK.ps1

* Fix issue with resolve

* Improve Windows Visual Studio provisioners (#758)

* Add retries to DotnetSDK provisioner

* Improve VS provisioners

* Fixes in syntax

* Fixes in syntax

* Rename Install-VS function

* Remove bootstrapperName parameter from Install-VisualStudio

* Small fix

* Improve Windows browsers provisioners stability (#725)

* Improve Chrome provisioner

* Improve Edge provisioner

* Improve Firefox provisioner

* Resolve conflicts in Install-Chrome.ps1

* Add retry logic for javatools (#802)

* Add retry logic for javatools

* Add retry logic for Selenium server download (#807)

* Add retry logic for selenium

* Refactoring

* Add retries for Selenium release info

* Minor fix

* Fix kubectl installation (#799)

* Fix kubectl installation

* Fix kubectl installation

* Add --client flag

* Fix for ubuntu 16.04

* Move kubectl apt packages installation to basic.sh

* Combined separated scripts for kubectl installation to one

* Minor fix

* Remove apt-transport-https package from basic.sh

* Delete helm init

* Disable crash reporting (#826)

* disable jit debugger

* override settings asfter vs installation

* Fix maven environment variable (#831)

* Updating readme file for macOS version 20200430.1 (#846)

* Add nodejs toolcache for windows (#840)

* add installation of nodejs

Co-authored-by: Dmitry Shibanov <v-dmshib@microsoft.com>

* Add nodejs tool cache for linux. (#839)

* add nodejs

* 4 node versions

Co-authored-by: Dmitry Shibanov <v-dmshib@microsoft.com>

* Hide packages that are no longer required (#854)

* Updating readme file for macOS version 20200430.1

* hide AutomaticRemove info

* added SSDT workload to VS 2019 (#844)

* Install Go to hostedtoolcache directory on Ubuntu image (#848)

* Move go location to the toolcache

* Install Go to hostedtoolcache directory on Windows image (#849)

* Move installed go versions to toolcache directory

* Updating readme file for win19 version 20200430.2

* Updating readme file for ubuntu16 version 20200430.2

* Install Windows 10 SDK (10.0.14393.795) via standalone installer (#862)

* add win10 sdk 10.0.14393

* markdown changes

* Add msys2 installation and validation (#632)

* SQL Server Data-Tier Application Framework latest upgrade. (#863)

* DACFx_latest_version_install

* Fix the cleanup script. (#856)

* Fix the cleanup script.The result is not returned to the variable because of the output redirect

* Added the cancel timeout.

* Update Haskell tool Stack (#860)

* Update Haskell tool Stack

* Update haskell script to install stack from get.haskellstack.org

* Updating readme file for win16 version 20200505.1

* Updating readme file for ubuntu18 version 20200430.1

* Switch Expand-Archive to 7zip on Windows image generation (#861)

* switch expand-archive to 7zip on windows

* [Ubuntu] Add Azcopy10 (#871)

* Add azcopy10

* Add spaces after azcopy

* add condition to ansible

* deprecated software Ubuntu 20.04

* install powershell Ubuntu 20.04

* use function in ansible

* move preparemetadata.sh to installers folder

* update preimagedata.sh ubuntu20

* add confition for dotnetsdk

* scripts/installers/2004/android.sh

* python.sh ubuntu 20.04

* simplify conditions

* remove toolcache.json

* get back note in readme

* remove workaround for git-lfs

* Az 3.8.0 Ubuntu 20.04

* fix kubernetes-tools in template

* add helpers script to os.sh

* remove old android, old cabal

* remove platform tools

* remove java7 for ubuntu20

* stay only go 1.14 Ubuntu 20.04

* add helper to python.sh

* add os.sh helper to azpwsh and java

* remove apis from documented

* remove addons google-api

* Install Az latest module

* get back swift

* set java 11 as default

* change offer and sku

* Revert "change offer and sku"

This reverts commit d5d92a5.

* Unite mercurial scripts

* combine basic scripts for ubuntu 18 & 20

* add os.sh to basic.sh

* Revert "set java 11 as default"

This reverts commit aada5b1.

* set Java11 by default, set Java8 for Androidsdk

* rework php script

* remove rid of php.sh files

* move installation to for loop

* add disk space validation

Co-authored-by: Christoph Bergmeister [MVP] <c.bergmeister@gmail.com>
Co-authored-by: Jostein Kjønigsen <jostein@kjonigsen.net>
Co-authored-by: Darii Nurgaleev <Darleev@github.com>
Co-authored-by: Sergey Dolin <v-sedoli@microsoft.com>
Co-authored-by: Sergey Dolin <v-sedoli@micorosoft.com>
Co-authored-by: Alena Sviridenko <alenasviridenko@github.com>
Co-authored-by: Vladimir Safonkin <vsafonkin@gmail.com>
Co-authored-by: Dariy Nurgaleev <50947177+Darleev@users.noreply.github.com>
Co-authored-by: Nikita Bykov <49442273+nikita-bykov@users.noreply.github.com>
Co-authored-by: Aleksandr Chebotov <47745270+al-cheb@users.noreply.github.com>
Co-authored-by: Image generation service account <no-reply@microsoft.com>
Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
Co-authored-by: phozzy <arthur.fayzullin@gmail.com>
Co-authored-by: Jorgen Thelin <jthelin@microsoft.com>
Co-authored-by: Dmitry Shibanov <shibanov-1997@inbox.ru>
Co-authored-by: Dmitry Shibanov <v-dmshib@microsoft.com>
Co-authored-by: cage200 <patruadrian@yahoo.com>
Co-authored-by: Junyoung Clare Jang <jjc9310@gmail.com>
Co-authored-by: Dibir Magomedsaygitov <61747324+dibir-magomedsaygitov@users.noreply.github.com>
Co-authored-by: Aleksandr Chebotov <v-aleche@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sergey-akhalkov sergey-akhalkov sergey-akhalkov approved these changes

@vsafonkin vsafonkin vsafonkin approved these changes

@AlenaSviridenko AlenaSviridenko AlenaSviridenko approved these changes

@maxim-lobanov maxim-lobanov maxim-lobanov approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@vmapetr @maxim-lobanov @sergey-akhalkov @vsafonkin @AlenaSviridenko