Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate if there is a way to install Hail 0.2 via a user script #491

Closed
rtitle opened this issue Jul 18, 2018 · 1 comment
Closed

Investigate if there is a way to install Hail 0.2 via a user script #491

rtitle opened this issue Jul 18, 2018 · 1 comment

Comments

@rtitle
Copy link
Collaborator

rtitle commented Jul 18, 2018

Leo uses Dataproc 1.1, Spark 2.0.2, and Hail 0.1. We'd like to support Dataproc 1.2, Spark 2.2.0, and Hail 0.2. That work is tracked in the below tickets.

#193
#144

HOWEVER, it may be possible to run Hail 0.2 on a Spark 2.0.2 cluster. This ticket is to see if we can create a user script that allows usage of Hail 0.2, but without other changes to the Leo/Spark environment (in other words, allows using Hail 0.2 on Leo clusters as they exist today).

@rtitle
Copy link
Collaborator Author

rtitle commented Jul 31, 2018

Decision: since there is no urgent user need for this, we will instead devote our time on full support for:

  • Python 3.5
  • Spark 2.3.0 (?)
  • Hail 0.2
  • Pandas recent

since these are all interrelated. See #488 for context around upgrading pandas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant