Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a Dockerfile for running Airflow #29

Open
MthwRobinson opened this issue Dec 19, 2020 · 0 comments
Open

Create a Dockerfile for running Airflow #29

MthwRobinson opened this issue Dec 19, 2020 · 0 comments

Comments

@MthwRobinson
Copy link
Contributor

The goal of this story is to create a docker image that (1) installs the data ingest package and (2) runs the Airflow scheduler and webserver. Optionally, the solution can use docker compose to set up a Postgres database to use as the backend for Airflow. Currently, Airflow is configured to use SQLite as a backend. Postgres would enable us to take advantage of multithreading. Another option would be to use the RDS, but I'd prefer to shut that down to save money.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant