Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Future steps #1

Open
Jithsaavvy opened this issue Aug 25, 2022 · 1 comment
Open

Future steps #1

Jithsaavvy opened this issue Aug 25, 2022 · 1 comment
Assignees

Comments

@Jithsaavvy
Copy link
Owner

  • Implement data management pipeline for data extraction, validation, data version control etc.
  • Use cloud storage services like Amazon S3 bucket to store data, artifacts, predictions.
  • Even though exception handling is implemented in the code, it is equally important to write separate test cases for different scenarios.
  • Orchestrate the entire workflow as automated pipeline by means of orchestration tools like Airflow, KubeFlow. As this is a small personal project with a static dataset, the pipeline can be created using normal function calls. But, it is crucial and predominant to replace them with orchestration tools for large, scalable and real-time workflows.
  • Implement Continuous Training (CT) pipeline along with CI/CD.
@Jithsaavvy Jithsaavvy self-assigned this Aug 25, 2022
@Jithsaavvy
Copy link
Owner Author

Added test cases using pytest and automated a new workflow which runs all tests for every commit.

Future steps - remaining

  • Implement data management pipeline for data extraction, validation, data version control, etc.
  • Use cloud storage services like Amazon S3 bucket to store data, artifacts, and predictions.
  • Orchestrate the entire workflow as an automated pipeline by means of orchestration tools like Airflow, and KubeFlow. As this is a small personal project with a static dataset, the pipeline can be created using normal function calls. But, it is crucial and predominant to replace them with orchestration tools for large, scalable, and real-time workflows.
  • Implement Continuous Training (CT) pipeline along with CI/CD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant