Monorepo Blueprints

A place to develop monorepo patterns, but can also be used for polyrepos!

Overview

Unlike other examples and demos out in the wild, we have loftier goals. If you don't see your goals being met, I wouldn't bother digging much further since technologies and versions change so much that anything here would almost be useless similar to what my experience has been while looking for best practices and more indepth examples that can be used as on-going blueprints rather than for one time presentations and shallow demos.

Quickstart

1. Download latest bazelisk
2. Download latest skaffold
3. Download latest minikube
4. Todo automate for local development outside of Google Cloud Workstations
5. Clone this repo to your local workstation or Google Cloud Workstations
6. Run make quickstart followed by eval $(minikube -p minikube docker-env)
7. Start with skaffold?
8. Run bazel build //... or bazel test //...

Blueprint Capability

Continuous Development

• Skaffold for continuous development
  • Skaffold for development
  • Skaffold for CI
  • Skaffold for CD
• Bazel
  • Local build and test
  • Local build and test cache
  • CI build and test
  • CI build and test remote cache
  • CI remote build executor (RBE)
• Pants

Continuous Integration

• Github Actions
• Cloud Build
• CircleCI
• Harness

Continuous Deployment

• Argo CD
• Cloud Deploy

Progressive Delivery

• Argo Rollouts
• Cloud Deploy

Framework Best Practice

• React - Frontend
• FastAPI - Backend
• Gin - Backend
• Springboot - Backend
• Typer - CLI

Motivation

• Salesforce
• Uber
• Twitter
• BazelCon - bzlmod
• DevOps Toolkit - skaffold

Goals

1. Quick, reliable and convenient builds for SDLC.
2. Manage dependencies across all projects.
3. Manage polyglot builds.
4. Manage blueprints for rapid application development.
5. Constant evaluation of the technology stack and scale for thousands of engineers AND feature development.

Objectives

1. Ensure efficient, correct, fast and repeatable builds
2. Ensure multi-language builds
3. Ensure high quality test coverage builds
4. Ensure hermetic builds

Roadmap

• Setup basic Bazel builds and simple examples
• Setup automated Bazel build + test using Github Actions
• Setup e2e Python example
  • Test multi-version python
• Setup e2e Golang example
  • Test multi-version golang
• Setup e2e Springboot example
  • Test multi-version springboot
• Setup e2e React example
• OCI container support
  • Test container structure
  • Make sure we are not regressing to using Dockerfile image builds
• Define promotion strategy supporting multi-release cadance
• Research and setup container & kubernetes development with Skaffold
• Research build systems like GitHub Actions, Cloud Build, Harness.io, etc
• Research remote execution services like Buildkite, BuildBuddy, etc
• Research testing best practice
• Document VSCode and PyCharm IDE development environment
• Tooling for boilerplate project creation
  • CookieCutter or Copier
• Port example into this code base microservices-demo

Directory Convention

monorepo/
|-- README.md
|-- Makefile
|-- skaffold.yaml
|-- WORKSPACE.bazel
|-- MODULE.bazel
|-- BUILD.bazel
|-- bazel-*/**                    -> bazel local directories not checked in
|-- fixes/**                      -> bazel temporary fixes
|-- docs/**                       -> please contribute!
|-- libs/                         -> common used libraries
|   |-- BUILD.bazel
|   |-- README.md
|   |-- base/
|   |-- calculator/
|   |-- echo/
|   `-- ...
|-- projects/                     -> all projects created here
|   |-- BUILD.bazel
|   |-- README.md
|   |-- bazel/
|   |-- base_project/
|   |-- py_calculator_cli_app/
|   |-- py_calculator_flask_app/
|   |-- echo_fastapi_app/
|   |-- py_helloworld_cli_app/
|   |-- py_helloworld_v2_cli_app/
|   `-- ...
|-- third_party/                  -> 3rd party dependencies
|   |-- README.md
|   |-- python/
|   `-- ...
`-- tools/                        -> bazel specific tooling
    |-- README.md
    |-- pytest
    `-- workspace_status.sh

References

• https://github.com/aspect-build/bazel-examples/tree/main
• https://s.itho.me/ccms_slides/2022/9/27/8dc835fc-b6f6-4656-84d4-53df725d1d6e.pdf
• https://nubenetes.com/kubernetes-based-devel/
• https://blog.getambassador.io/skaffold-vs-telepresence-comparing-kubernetes-inner-development-loop-tools-c8abd70545e5
• https://sixfeetup.com/blog/common-kubernetes-concerns