Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud storage file mounts for gs:// #269

Closed
Michaelvll opened this issue Jan 31, 2022 · 3 comments · Fixed by #102
Closed

Cloud storage file mounts for gs:// #269

Michaelvll opened this issue Jan 31, 2022 · 3 comments · Fixed by #102
Labels
enhancement New feature or request

Comments

@Michaelvll
Copy link
Collaborator

Our task is still not available to mount gs:// storage. Can we check the GCLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS in the environment, then set the environment on the remote instance and send the credential to it?

@Michaelvll Michaelvll added the enhancement New feature or request label Jan 31, 2022
@Michaelvll Michaelvll changed the title Cloud storage for gs:// Cloud storage file mounts for gs:// Jan 31, 2022
@michaelzhiluo
Copy link
Collaborator

This is addressed in #102.

@gmittal gmittal linked a pull request Feb 1, 2022 that will close this issue
19 tasks
@michaelzhiluo
Copy link
Collaborator

It might take a while for Sky Init to be merged, so I will push a hotfix for this, which will be later overwritten by Sky Init.

@concretevitamin
Copy link
Member

@Michaelvll The current sky credentials design is that the user must have certain credential files locally (hinted by sky init). It's not enough to just have environment vars.

The reason for not doing env vars is because authentication in the clouds has much more cases than just files + env vars. For AWS, their metadata service means an empty VM has access to a user's AWS credentials. For GCP, it's also not simple: ref1, ref2. This is a topic for more investigation.

tl;dr Give master a try now to see if the tasks can access gs://private-bucket? If not, running the hint in sky init should work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants