Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install system dependencies in swiflty-install.sh #54

Merged
merged 12 commits into from
Jul 21, 2023
2 changes: 1 addition & 1 deletion docker/install-test.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8

# dependencies
RUN apt-get update && apt-get install -y curl
RUN apt-get update --fix-missing && apt-get install -y curl
RUN echo 'export PATH="$HOME/.local/bin:$PATH"' >> $HOME/.profile
2 changes: 1 addition & 1 deletion docker/test.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8

# dependencies
RUN apt-get update && apt-get install -y curl build-essential
RUN apt-get update --fix-missing && apt-get install -y curl build-essential
COPY ./scripts/install-libarchive.sh /
RUN /install-libarchive.sh

Expand Down
183 changes: 142 additions & 41 deletions install/swiftly-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
# Unless the --disable-confirmation flag is set, this script will allow the runner to
# configure either of those two directory paths.
#
# Unless the --no-install-system-deps flag is set, this script will attempt to install Swift's
# system dependencies using the system package manager.
#
# curl is required to run this script.

set -o errexit
Expand All @@ -48,6 +51,47 @@ read_input_with_default () {
fi
}

yn_prompt () {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally thought we needed another y/n prompt, so factored it out into these functions. We didn't end up needing it, but it still seemed useful so I left it in.

if [[ "$1" == "true" ]]; then
echo "(Y/n)"
else
echo "(y/N)"
fi
}

# Read a y/n input.
# First argument is the default value (must be "true" or "false").
#
# Sets READ_INPUT_RETURN to "true" for an input of "y" or "Y", "false" for an input
# of "n" or "N", or the default value for a blank input
#
# For all other inputs, a message is printed and the user is prompted again.
read_yn_input () {
while [[ true ]]; do
read_input_with_default "$1"

case "$READ_INPUT_RETURN" in
"y" | "Y")
READ_INPUT_RETURN="true"
return
;;

"n" | "N")
READ_INPUT_RETURN="false"
return
;;

"$1")
return
;;

*)
echo "Please input either \"y\" or \"n\", or press ENTER to use the default."
;;
esac
done
}

# Replaces the actual path to $HOME at the beginning of the provided string argument with
# the string "$HOME". This is used when printing to stdout.
# e.g. "home/user/.local/bin" => "$HOME/.local/bin"
Expand All @@ -71,9 +115,68 @@ bold () {
echo "$(tput bold)$1$(tput sgr0)"
}

# Fetch the list of required system dependencies from the apple/swift-docker
# repository and attempt to install them using the system's package manager.
#
# $docker_platform_name, $docker_platform_version, and $package manager need
# to be set before calling this function.
install_system_deps () {
if [[ "$(id --user)" != "0" ]] && ! has_command sudo ; then
echo "Warning: sudo not installed and current user is not root, skipping system dependency installation."
return
elif ! has_command "$package_manager" ; then
echo "Warning: package manager \"$package_manager\" not found, skipping system dependency installation."
return
fi

dockerfile_url="https://raw.githubusercontent.com/apple/swift-docker/main/nightly-main/$docker_platform_name/$docker_platform_version/Dockerfile"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The system dependencies can change on each release, so I decided to just use the latest set of them to ensure a user can always install the latest swift version successfully. I thought about unioning the oldest releases' dependencies with the newest one's, but that seemed overly complex for an uncommon case.

dockerfile="$(curl --silent --retry 3 --location --fail $dockerfile_url)"
if [[ "$?" -ne 0 ]]; then
echo "Error enumerating system dependencies, skipping installation of system dependencies."
fi

# Find the line number of the RUN command associated with installing system dependencies.
beg_line_num=$(printf "$dockerfile" | grep -n --max-count=1 "$package_manager.*install" | cut -d ":" -f1)

# Starting from there, find the first line that starts with an & or doesn't end in a backslash.
relative_end_line_num=$(printf "$dockerfile" |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started using lowercase variable names for non-environment variables per https://stackoverflow.com/a/673940/16745421. In a later commit, I'll go back and update the existing variables to match this.

tail --lines=+"$((beg_line_num + 1))" |
grep -n --max-count=1 --invert-match '[[:space:]]*[^&].*\\$' | cut -d ":" -f1)
end_line_num=$((beg_line_num + relative_end_line_num))

# Read the lines between those two, deleting any spaces and backslashes.
readarray -t package_list < <(printf "$dockerfile" | sed -n "$((beg_line_num + 1)),${end_line_num}p" | sed -r 's/[\ ]//g')

# If the installation command from the Dockerfile included some cleanup as part of a second command, drop that.
if [[ "${package_list[-1]}" =~ ^\&\& ]]; then
unset 'package_list[-1]'
fi

install_args=(--quiet)
if [[ "$DISABLE_CONFIRMATION" == "true" ]]; then
install_args+=(-y)
fi

# Disable errexit since failing to install system dependencies is not swiftly installation-fatal.
set +o errexit
if [[ "$(id --user)" == "0" ]]; then
"$package_manager" install "${install_args[@]}" "${package_list[@]}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a chance on ubuntu that you need to call apt-get update first?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, especially for a fresh docker image. That said, I think it would be better for users to do that outside of the script, since in most cases calling update shouldn't be needed, and we don't want to do it unnecessarily since it can involve downloading a decent amount of data.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a package install fails is it worthwhile pointing out to the user they might want to run apt-get update

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea, done

else
sudo "$package_manager" install "${install_args[@]}" "${package_list[@]}"
fi
if [[ "$?" -ne 0 ]]; then
echo "System dependency installation failed."
if [[ "$package_manager" == "apt-get" ]]; then
echo "You may need to run apt-get update before installing system dependencies."
fi
fi
set -o errexit
}

SWIFTLY_INSTALL_VERSION="0.1.0"

MODIFY_PROFILE="true"
SWIFTLY_INSTALL_SYSTEM_DEPS="true"

for arg in "$@"; do
case "$arg" in
Expand All @@ -89,6 +192,7 @@ FLAGS:
-y, --disable-confirmation Disable confirmation prompt.
--no-modify-profile Do not attempt to modify the profile file to set environment
variables (e.g. PATH) on login.
--no-install-system-deps Do not attempt to install Swift's required system dependencies.
-h, --help Prints help information.
--version Prints version information.
EOF
Expand All @@ -103,6 +207,10 @@ EOF
MODIFY_PROFILE="false"
;;

"--no-install-system-deps")
SWIFTLY_INSTALL_SYSTEM_DEPS="false"
;;

"--version")
echo "$SWIFTLY_INSTALL_VERSION"
exit 0
Expand Down Expand Up @@ -134,23 +242,31 @@ case "$ID" in
fi
PLATFORM_NAME="amazonlinux2"
PLATFORM_NAME_FULL="amazonlinux2"
docker_platform_name="amazonlinux"
docker_platform_version="2"
package_manager="yum"
;;

"ubuntu")
docker_platform_name="ubuntu"
package_manager="apt-get"
case "$UBUNTU_CODENAME" in
"jammy")
PLATFORM_NAME="ubuntu2204"
PLATFORM_NAME_FULL="ubuntu22.04"
docker_platform_version="22.04"
;;

"focal")
PLATFORM_NAME="ubuntu2004"
PLATFORM_NAME_FULL="ubuntu20.04"
docker_platform_version="20.04"
;;

"bionic")
PLATFORM_NAME="ubuntu1804"
PLATFORM_NAME_FULL="ubuntu18.04"
docker_platform_version="18.04"
;;

*)
Expand All @@ -167,6 +283,9 @@ case "$ID" in
fi
PLATFORM_NAME="ubi9"
PLATFORM_NAME_FULL="ubi9"
docker_platform_name="rhel-ubi"
docker_platform_version="9"
package_manager="yum"
;;

*)
Expand Down Expand Up @@ -243,6 +362,7 @@ while [ -z "$DISABLE_CONFIRMATION" ]; do
printf " %40s: $(bold $(replace_home_path $HOME_DIR))\n" "Data and configuration files directory"
printf " %40s: $(bold $(replace_home_path $BIN_DIR))\n" "Executables installation directory"
printf " %40s: $(bold $MODIFY_PROFILE)\n" "Modify login config ($(replace_home_path $PROFILE_FILE))"
printf " %40s: $(bold $SWIFTLY_INSTALL_SYSTEM_DEPS)\n" "Install system dependencies"
echo ""
echo "Select one of the following:"
echo "1) Proceed with the installation (default)"
Expand All @@ -265,30 +385,17 @@ while [ -z "$DISABLE_CONFIRMATION" ]; do
read_input_with_default "$BIN_DIR"
BIN_DIR="$(expand_home_path $READ_INPUT_RETURN)"

if [[ "$MODIFY_PROFILE" == "true" ]]; then
MODIFY_PROFILE_PROMPT="(Y/n)"
else
MODIFY_PROFILE_PROMPT="(y/N)"
fi
echo "Modify login config ($(replace_home_path $PROFILE_FILE))? $MODIFY_PROFILE_PROMPT"
read_input_with_default "$MODIFY_PROFILE"

case "$READ_INPUT_RETURN" in
"y" | "Y")
MODIFY_PROFILE="true"
;;

"n" | "N")
MODIFY_PROFILE="false"
;;
echo "Modify login config ($(replace_home_path $PROFILE_FILE))? $(yn_prompt $MODIFY_PROFILE)"
read_yn_input "$MODIFY_PROFILE"
MODIFY_PROFILE="$READ_INPUT_RETURN"

*)
;;
esac
echo "Install system dependencies? $(yn_prompt $SWIFTLY_INSTALL_SYSTEM_DEPS)"
read_yn_input "$SWIFTLY_INSTALL_SYSTEM_DEPS"
SWIFTLY_INSTALL_SYSTEM_DEPS="$READ_INPUT_RETURN"
;;

*)
echo "Cancelling installation"
echo "Cancelling installation."
exit 0
;;
esac
Expand All @@ -300,23 +407,11 @@ if [[ -d "$HOME_DIR" ]]; then
else
echo "Existing swiftly installation detected at $(replace_home_path $HOME_DIR), overwrite? (Y/n)"

while [[ true ]]; do
read_input_with_default "y"
case "$READ_INPUT_RETURN" in
"y" | "Y")
break
;;

"n" | "N" | "q")
echo "Cancelling installation"
exit 0
;;

*)
echo "Please input \"y\" or \"n\"."
;;
esac
done
read_yn_input "true"
if [[ "$READ_INPUT_RETURN" == "false" ]]; then
echo "Cancelling installation."
exit 0
fi
fi

rm -r $HOME_DIR
Expand All @@ -343,10 +438,6 @@ echo "$JSON_OUT" > "$HOME_DIR/config.json"
# Verify the downloaded executable works. The script will exit if this fails due to errexit.
SWIFTLY_HOME_DIR="$HOME_DIR" SWIFTLY_BIN_DIR="$BIN_DIR" "$BIN_DIR/swiftly" --version > /dev/null

echo ""
echo "swiftly has been succesfully installed!"
echo ""

ENV_OUT=$(cat <<EOF
export SWIFTLY_HOME_DIR="$(replace_home_path $HOME_DIR)"
export SWIFTLY_BIN_DIR="$(replace_home_path $BIN_DIR)"
Expand All @@ -367,6 +458,16 @@ if [[ "$MODIFY_PROFILE" == "true" ]]; then
fi
fi

if [[ "$SWIFTLY_INSTALL_SYSTEM_DEPS" != "false" ]]; then
echo ""
echo "Installing Swift's system dependencies via $package_manager (note: this may require root access)..."
install_system_deps
fi

echo ""
echo "swiftly has been succesfully installed!"
echo ""

if ! has_command "swiftly" || [[ "$HOME_DIR" != "$DEFAULT_HOME_DIR" || "$BIN_DIR" != "$DEFAULT_BIN_DIR" ]] ; then
echo "Once you log in again, swiftly should be accessible from your PATH."
echo "To begin using swiftly from your current shell, first run the following command:"
Expand Down
49 changes: 49 additions & 0 deletions install/test-util.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,52 @@ test_fail () {
test_pass () {
exit 0
}

get_os () {
if [[ -f "/etc/os-release" ]]; then
OS_RELEASE="/etc/os-release"
elif [[ -f "/usr/lib/os-release" ]]; then
OS_RELEASE="/usr/lib/os-release"
else
echo "Error: could not detect OS information"
exit 1
fi

source "$OS_RELEASE"

case "$ID" in
"amzn")
echo "amazonlinux2"
;;

"ubuntu")
case "$UBUNTU_CODENAME" in
"jammy")
echo "ubuntu2204"
;;

"focal")
echo "ubuntu2004"
;;

"bionic")
echo "ubuntu1804"
;;

*)
echo "Unsupported Ubuntu version: $PRETTY_NAME"
exit 1
;;
esac
;;

"rhel")
echo "rhel-ubi9"
;;

*)
echo "Unsupported platform: $PRETTY_NAME"
exit 1
;;
esac
}
3 changes: 1 addition & 2 deletions install/tests/custom-home-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ cleanup () {
}
trap cleanup EXIT

# Make sure that the "~" character is handled properly.
printf "2\n\$HOME/${CUSTOM_HOME_DIR_NAME}\n\$HOME/${CUSTOM_HOME_DIR_NAME}/bin\ny\n1\n" | ./swiftly-install.sh
printf "2\n\$HOME/${CUSTOM_HOME_DIR_NAME}\n\$HOME/${CUSTOM_HOME_DIR_NAME}/bin\ny\nn\n1\n" | ./swiftly-install.sh

# .profile should be updated to update PATH and SWIFTLY_HOME_DIR/SWIFTLY_BIN_DIR.
bash --login -c "swiftly --version"
Expand Down
2 changes: 1 addition & 1 deletion install/tests/custom-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ cleanup () {
}
trap cleanup EXIT

printf "2\n$CUSTOM_HOME_DIR\n$CUSTOM_BIN_DIR\ny\n1\n" | ./swiftly-install.sh
printf "2\n$CUSTOM_HOME_DIR\n$CUSTOM_BIN_DIR\ny\nn\n1\n" | ./swiftly-install.sh

# .profile should be updated to update PATH and SWIFTLY_HOME_DIR/SWIFTLY_BIN_DIR.
bash --login -c "swiftly --version"
Expand Down
2 changes: 1 addition & 1 deletion install/tests/custom-tilde-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ cleanup () {
trap cleanup EXIT

# Make sure that the "~" character is handled properly.
printf "2\n~/${CUSTOM_HOME_DIR_NAME}\n~/${CUSTOM_HOME_DIR_NAME}/bin\ny\n1\n" | ./swiftly-install.sh
printf "2\n~/${CUSTOM_HOME_DIR_NAME}\n~/${CUSTOM_HOME_DIR_NAME}/bin\ny\nn\n1\n" | ./swiftly-install.sh

# .profile should be updated to update PATH and SWIFTLY_HOME_DIR/SWIFTLY_BIN_DIR.
bash --login -c "swiftly --version"
Expand Down
Loading