[DigitalOcean] droplet integration

[asaiacai]

This adds digital ocean droplets to the sky.

Tested (run the relevant ones):

• Code formatting: bash format.sh
• All smoke tests: pytest tests/test_smoke.py --do

[moinnadeem]

@asaiacai I would be really interested in using this work -- do you know when you might be able to land this PR?

[asaiacai]

Hello! I’m aiming for this to land in next two weeks providing testing goes smoothly. Stay tuned!

[asaiacai]

@Michaelvll i think this PR is finally ready, once you have a free moment! I've currently disabled tests for gpu droplets as they are still in early access, but I think I got the remaining tests to work on normal CPU droplets fine. Let me know if you find any issues.

[asaiacai]

also this catalog update is required to pass the sky serve requests since I added cheaper 2vcpu instances.

skypilot-org/skypilot-catalog#83

[cblmemo]

Thanks for this amazing work @asaiacai ! This would be very helpful. Left some discussions ;)

[cblmemo]

Is do a widely adopted term..? I personally found it a little bit confusing with the verb. Does sth like "skypilot-nightly[digitalocean]" works, like paperspace? cc @Michaelvll @concretevitamin for a look here.

[cblmemo]

Actually, after a second glance, it seems like this is the official abbreviation used in their python package. Lets keep the original name, but maybe we should make the cloud argument to digitalocean, i.e. sky launch --cloud digitalocean?

[cblmemo]

Why don't we support it? I saw relevant code snippet in the ray yaml file.

[asaiacai]

I included a startup script that installs docker, but ssh comes up in parallel before the startup script finishes running resulting in the docker environment setup failing. Docker does eventually get setup on the machine. If I for example, reattempt to launch on the same cluster after waiting for like 10 seconds, I can successfully use the docker environment, but the times i tested it never worked on freshly provisioned instances. Let me know what would make sense here.

[cblmemo]

Is it possible to use a base image with docker installed? Or, can we wait for the docker to finish setup for DO cloud?

skypilot/sky/provision/docker_utils.py

Lines 336 to 343 in e870839

	def _check_docker_installed(self):
	no_exist = 'NoExist'
	cleaned_output = self._run(
	f'command -v {self.docker_cmd} || echo {no_exist!r}')
	if no_exist in cleaned_output or 'docker' not in cleaned_output:
	logger.error(
	f'{self.docker_cmd.capitalize()} not installed. Please use an '
	f'image with {self.docker_cmd.capitalize()} installed.')