Installation
Contents
Installation¶
Install SkyPilot using pip:
$ # SkyPilot requires python >= 3.6.
$ # Recommended: use a new conda env to avoid package conflicts.
$ conda create -y -n sky python=3.7
$ conda activate sky
$ # Choose an extra (default: [aws])
$ pip install skypilot
$ # pip install "skypilot[gcp]"
$ # pip install "skypilot[azure]"
$ # pip install "skypilot[all]"
SkyPilot currently supports three major cloud providers: AWS, GCP, and Azure. If you
only have access to certain clouds, use any combination of
"[aws,azure,gcp]" (e.g., "[aws,gcp]") to reduce the
dependencies installed.
You may also install SkyPilot from source.
$ git clone https://github.com/skypilot-org/skypilot.git
$ cd skypilot
$ pip install ".[all]"
Note
For Macs, macOS >= 10.15 is required to install SkyPilot. Apple Silicon-based devices (e.g. Apple M1) must run conda install grpcio=1.43.0 prior to installing SkyPilot.
Note
As an alternative to installing SkyPilot on your laptop, we also provide a Docker image as a quick way to try out SkyPilot. See instructions below on running SkyPilot in a container.
Cloud account setup¶
Configure access to at least one cloud:
AWS
To get the AWS access key required by aws configure, please go to the AWS IAM Management Console and click on the “Access keys” dropdown (detailed instructions here). The Default region name [None]: and Default output format [None]: fields are optional and can be left blank to choose defaults.
$ # Install boto
$ pip install boto3
$ # Configure your AWS credentials
$ aws configure
Note: If you are using AWS IAM Identity Center (AWS SSO), you will need pip install awscli>=1.27.10. See here for instructions on how to configure AWS SSO.
GCP
$ pip install google-api-python-client
$ conda install -c conda-forge google-cloud-sdk
$ gcloud init
$ # Run this if you don't have a credentials file.
$ # This will generate ~/.config/gcloud/application_default_credentials.json.
$ gcloud auth application-default login
If running conda install -c conda-forge google-cloud-sdk produces the error “RemoveError: ‘requests’ is a dependency of conda and cannot be removed from conda’s operating environment”, try conda update --force conda first and rerun the command.
Note: if you encounter Authorization Error (Error 400: invalid_request) with the url generated by gcloud auth login, try installing the latest version of the Google Cloud SDK (e.g., with conda install -c conda-forge google-cloud-sdk) on your local machine (which opened the browser) and rerun the command.
Note
If you are using multiple GCP projects, list all the configs by gcloud config configurations list and activate one by gcloud config configurations activate <CONFIG_NAME> (See GCP docs).
Azure
$ # Login
$ az login
$ # Set the subscription to use
$ az account set -s <subscription_id>
Hint: run az account subscription list to get a list of subscription IDs under your account.
Verifying cloud setup
After configuring the desired clouds, you can optionally run sky check to verify that credentials are correctly set up:
$ sky check
This will produce a summary like:
Checking credentials to enable clouds for SkyPilot.
AWS: enabled
GCP: enabled
Azure: enabled
SkyPilot will use only the enabled clouds to run tasks. To change this, configure cloud credentials, and run sky check.
Requesting quotas for first time users¶
If your cloud account has not been used to launch instances before, the respective quotas are likely set to zero or a low limit. This is especially true for GPU instances.
Please follow Requesting Quota Increase to check quotas and request quota increases before proceeding.
Quick alternative: trying in Docker¶
As a quick alternative to installing SkyPilot on your laptop, we also provide a Docker image with SkyPilot and its dependencies installed for users to quickly try out SkyPilot. You can simply run:
$ docker run -td --name sky --rm -v "$HOME/.sky:/root/.sky:rw" -v "$HOME/.aws:/root/.aws:rw" -v "$HOME/.config/gcloud:/root/.config/gcloud:rw" -v "$HOME/.azure:/root/.azure:rw" public.ecr.aws/a9w6z7w5/sky:latest
$ docker exec -it sky /bin/bash
If your cloud CLIs are already setup, your credentials will be mounted to the container and you can proceed to Quickstart. Else you can follow the instructions in Cloud account setup inside the container to setup your cloud accounts.
If you exit from the shell in the container, the container will keep running
in the background. You can start a new shell with docker exec -it sky /bin/bash.
Once you are done with experimenting with sky, remember to delete any clusters and storage resources you may have created using the following commands:
# Run inside the container
$ sky down -a -y
$ sky storage delete -a -y
Finally, you can stop the container with:
$ docker stop sky
Enabling shell completion¶
SkyPilot supports shell completion for Bash (Version 4.4 and up), Zsh and Fish. This is only available for click versions 8.0 and up (use pip install click==8.0.4 to install).
To enable shell completion after installing SkyPilot, you will need to modify your shell configuration.
SkyPilot automates this process using the --install-shell-completion option, which you should call using the appropriate shell name or auto:
$ sky --install-shell-completion auto
$ # sky --install-shell-completion zsh
$ # sky --install-shell-completion bash
$ # sky --install-shell-completion fish
Shell completion may perform poorly on certain shells and machines.
If you experience any issues after installation, you can use the --uninstall-shell-completion option to uninstall it, which you should similarly call using the appropriate shell name or auto:
$ sky --uninstall-shell-completion auto
$ # sky --uninstall-shell-completion zsh
$ # sky --uninstall-shell-completion bash
$ # sky --uninstall-shell-completion fish