docs: add ADR 057 App Wiring Part I

docs/architecture/README.md

@@ -79,6 +79,7 @@ When writing ADRs, follow the same best practices for writing RFCs. When writing
 * [ADR 039: Epoched Staking](./adr-039-epoched-staking.md)
 * [ADR 040: Storage and SMT State Commitments](./adr-040-storage-and-smt-state-commitments.md)
 * [ADR 046: Module Params](./adr-046-module-params.md)
+* [ADR 057: App Wiring Part I](./adr-057-app-wiring-1.md)
 
 ### Draft
 

docs/architecture/adr-057-app-wiring-1.md

@@ -0,0 +1,273 @@
+# ADR 057: App Wiring Part I
+
+## Changelog
+
+* 2022-05-04: Initial Draft
+
+## Status
+
+PROPOSED Partially Implemented
+
+## Abstract
+
+In order to make it easier to build Cosmos SDK modules and apps, we propose a new app wiring system based on
+dependency injection and declarative app configurations to replace the current `app.go` code.
+
+## Context
+
+A number of factors have made the SDK and SDK apps in their current state hard to maintain. A symptom of the current
+state of complexity is [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/c3edbb22cab8678c35e21fe0253919996b780c01/simapp/app.go)
+which contains almost 100 lines of imports and is otherwise over 600 lines of mostly boilerplate code that is
+generally copied to each new project. (Not to mention the additional boilerplate which gets copied in `simapp/simd`.)
+
+The large amount of boilerplate needed to bootstrap an app has made it hard to release independently versioned go
+modules for Cosmos SDK modules as described in [ADR 053: Go Module Refactoring](./adr-053-go-module-refactoring.md).
+
+In addition to being very verbose and repetitive, `app.go` also exposes a large surface area for breaking changes
+as most modules instantiate themselves with positional parameters which forces breaking changes anytime a new parameter
+(even an optional one) is needed.
+
+Several attempts were made to improve the current situation including [ADR 033: Internal-Module Communication](./adr-033-protobuf-inter-module-comm.md)
+and [a proof-of-concept of a new SDK](https://github.com/allinbits/cosmos-sdk-poc). The discussions around these
+designs led to the current solution described here.
+
+## Decision
+
+In order to improve the current situation, a new "app wiring" paradigm has been designed to replace `app.go` which
+involves:
+* declaration configuration of the modules in an app which can be serialized to JSON or YAML
+* a dependency-injection (DI) framework for instantiating apps from the that configuration
+
+### Dependency Injection
+
+When examining the code in `app.go` most of the code simply instantiates modules with dependencies provided either
+by the framework (such as store keys) or by other modules (such as keepers). It is generally pretty obvious given
+the context what the correct dependencies actually should be, so dependency-injection is an obvious solution. Rather
+than making developers manually resolve dependencies, a module will tell the DI container what dependency it needs
+and the container will figure out how to provide it.
+
+We explored several existing DI solutions in golang and felt that the reflection-based approach in [uber/dig](https://github.com/uber-go/dig)
+was closest to what we needed but not quite there. Assessing what we needed for the SDK, we designed and built
+the Cosmos SDK [container module](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/container), which has the following
+features:
+* dependency resolution and provision through functional constructors, ex: `func(need SomeDep) (AnotherDep, error)`
+* dependency injection `In` and `Out` structs which support `optional` dependencies
+* grouped-dependencies (many-per-container) through the `AutoGroupType` tag interface
+* module-scoped dependencies via `ModuleKey`s (where each module gets a unique dependency)
+* one-per-module dependencies through the `OnePerModuleType` tag interface
+* sophisticated debugging information and container visualization via GraphViz
+
+Here are some examples of how these would be used in an SDK module:
+* `StoreKey` could be a module-scoped dependency which is unique per module
+* a module's `AppModule` instance (or the equivalent) could be a `OnePerModuleType`
+* CLI commands could be provided with `AutoGroupType`s
+
+Note that even though dependency resolution is dynamic and based on reflection, which could be considered a pitfall
+of this approach, the entire dependency graph should be resolved immediately on app startup and only gets resolved
+once (except in the case of dynamic config reloading which is a separate topic). This means that if there are any
+errors in the dependency graph, they will get reported immediately on startup so this approach is only slightly worse
+than fully static resolution in terms of error reporting and much better in terms of code complexity.
+
+### Declarative App Config
+
+In order to compose modules into an app, a declarative app configuration will be used. This configuration is based off
+of protobuf and its basic structure is very simple:
+
+```protobuf
+package cosmos.app.v1;
+
+message Config {
+  repeated ModuleConfig modules = 1;
+}
+
+message ModuleConfig {
+  string name = 1;
+  google.protobuf.Any config = 2;
+}
+```
+
+(See also https://github.com/cosmos/cosmos-sdk/blob/6e18f582bf69e3926a1e22a6de3c35ea327aadce/proto/cosmos/app/v1alpha1/config.proto)
+
+The configuration for every module is itself a protobuf message and modules will be identified and loaded based
+on the protobuf type URL of their config object (ex. `cosmos.bank.module.v1.Module`). Modules are given a unique short `name`
+to share resources across different versions of the same module which might have a different protobuf package
+versions (ex. `cosmos.bank.module.v2.Module`).
+
+An example app config in YAML might look like this:
+```yaml
+modules:
+  - name: baseapp
+    config:
+      "@type": cosmos.baseapp.module.v1.Module
+      begin_blockers: [staking, auth, bank]
+      end_blockers: [bank, auth, staking]
+      init_genesis: [bank, auth, staking]
+  - name: auth
+    config:
+      "@type": cosmos.auth.module.v1.Module
+      bech32_prefix: "foo"
+  - name: bank
+    config:
+      "@type": cosmos.bank.module.v1.Module
+  - name: staking
+    config:
+      "@type": cosmos.staking.module.v1.Module
+```
+
+In the above example, there is a hypothetical `baseapp` module which contains the information around ordering of
+begin blockers, end blockers, and init genesis. Rather than lifting these concerns up to the module config layer,
+they are themselves handled by a module which could allow a convenient way of swapping out different versions of
+baseapp (for instance to target different versions of tendermint), without needing to change the rest of the config.
+The `baseapp` module would then provide to the server framework (which sort of sits outside the ABCI app) an instance
+of `abci.Application`.
+
+In this model, an app is *modules all the way down* and the dependency injection/app config layer is very much
+protocol-agnostic and can adapt to even major breaking changes at the protocol layer.
+
+### Module & Protobuf Registration
+
+In order for the two components of dependency injection and declarative configuration to work together as described,
+we need a way for modules to actually register themselves and provide dependencies to the container.
+
+One additional complexity that needs to be handled at this layer is protobuf registry initialization. Recall that
+in both the current SDK `codec` and the proposed [ADR 054: Protobuf Semver Compatible Codegen](https://github.com/cosmos/cosmos-sdk/pull/11802),
+protobuf types need to be explicitly registered. Given that the app config itself is based on protobuf and
+uses protobuf `Any` types, protobuf registration needs to happen before the app config itself can be decoded. Because
+we don't know which protobuf `Any` types will be needed a priori and modules themselves define those types, we need
+to decode the app config in separate phases:
+1. parse app config JSON/YAML as raw JSON and collect required module type URLs (without doing proto JSON decoding)
+2. build a [protobuf type registry](https://pkg.go.dev/google.golang.org/protobuf@v1.28.0/reflect/protoregistry) based
+   on file descriptors and types provided by each required module
+3. decode the app config as proto JSON using the protobuf type registry
+
+Because in [ADR 054: Protobuf Semver Compatible Codegen](https://github.com/cosmos/cosmos-sdk/pull/11802), each module
+should use `internal` generated code which is not registered with the global protobuf registry, this code should provide
+an alternate way to register protobuf types with a type registry. In the same way that `.pb.go` files currently have a
+`var File_foo_proto protoreflect.FileDescriptor` for the file `foo.proto`, generated code should have a new member
+`var Types_foo_proto TypeInfo` where `TypeInfo` is an interface or struct with all the necessary info to register both
+the protobuf generated types and file descriptor.
+
+So a module must provide dependency injection providers and protobuf types, and takes as input its module
+config object which uniquely identifies the module based on its type URL.
+
+With this in mind, we define a global module register which allows module implementations to register themselves
+with the following API:
+
+```go
+// Register registers a module with the provided type name (ex. cosmos.bank.module.v1.Module)
+// and the provided options.
+func Register(configTypeName protoreflect.FullName, option ...Option) { ... }
+
+type Option { /* private methods */ }
+
+// Provide registers dependency injection provider functions which work with the
+// cosmos-sdk container module. These functions can also accept an additional
+// parameter for the module's config object.
+func Provide(providers ...interface{}) Option { ... }
+
+// Types registers protobuf TypeInfo's with the protobuf registry.
+func Types(types ...TypeInfo) Option { ... }
+```
+
+Ex:
+```go
+func init() {
+	module.Register("cosmos.bank.module.v1.Module",
+		module.Types(
+			types.Types_tx_proto,
+            types.Types_query_proto,
+            types.Types_types_proto,
+	    ),
+	    module.Provide(
+			provideBankModule,
+	    )
+	)
+}
+
+type inputs struct {
+	container.In
+
+	AuthKeeper auth.Keeper
+	DB ormdb.ModuleDB
+}
+
+type outputs struct {
+	Keeper bank.Keeper
+	Handler app.Handler // app.Handler is a hypothetical type which replaces the current AppModule
+}
+
+func provideBankModule(config types.Module, inputs) (outputs, error) { ... }
+```
+
+Note that in this module, a module configuration object *cannot* register different dependency providers based on the
+configuration. This is intentional because it allows us to know globally which modules provide which dependencies. This
+can help us figure out issues with missing dependencies in an app config if the needed modules are loaded at runtime.
+In cases where required modules are not loaded at runtime, it may be possible to guide users to the correct module if
+through a global Cosmos SDK module registry.
+
+### New `app.go`
+
+With this setup, `app.go` might now look something like this:
+
+```go
+package main
+
+import (
+	// Each go package which registers a module must be imported just for side-effects
+	// so that module implementations are registered.
+	_ "github.com/cosmos/cosmos-sdk/x/auth/module"
+	_ "github.com/cosmos/cosmos-sdk/x/bank/module"
+	_ "github.com/cosmos/cosmos-sdk/x/staking/module"
+	"github.com/cosmos/cosmos-sdk/core/app"
+)
+
+// go:embed app.yaml
+var appConfigYaml []byte
+
+func main() {
+	app.Run(app.ParseYamlConfig(appConfigYaml))
+}
+```
+
+### Application to existing SDK modules
+
+So far we have described a system which is largely agnostic to the specifics of the SDK such as store keys, `AppModule`,
+`BaseApp`, etc. A second app wiring ADR will be created which outlines the details of how this app wiring system will
+be applied to the existing SDK in a way that:
+1. is as easy to apply to existing modules as possible,
+2. while also making it possible to improve existing APIs and minimize long-term technical debt
+
+## Consequences
+
+### Backwards Compatibility
+
+Modules which work with the new app wiring system do not need to drop their existing `AppModule` and `NewKeeper`
+registration paradigms. These two methods can live side-by-side for as long as is needed.
+
+### Positive
+
+* wiring up new apps will be simpler, more succinct and less error-prone
+* it will be easier to develop and test standalone SDK modules without needing to replicate all of simapp
+* it may be possible to dynamically load modules and upgrade chains without needing to do a coordinated stop and binary
+  upgrade using this mechanism
+
+### Negative
+* it may be confusing when a dependency is missing although error messages, the GraphViz visualization, and global
+  module registration may help with that
+
+### Neutral
+
+* it will require work and education
+
+## Further Discussions
+
+As mentioned above, a second app wiring ADR will be created to describe more specifics than there is space to go
+into here. Further discussions will also happen within the Cosmos SDK Framework Working Group and in https://github.com/cosmos/cosmos-sdk/discussions/10582.
+
+## References
+
+* https://github.com/cosmos/cosmos-sdk/blob/c3edbb22cab8678c35e21fe0253919996b780c01/simapp/app.go
+* https://github.com/allinbits/cosmos-sdk-poc
+* https://github.com/uber-go/dig
+* https://pkg.go.dev/github.com/cosmos/cosmos-sdk/container
+* https://github.com/cosmos/cosmos-sdk/pull/11802