Consolidate Migrations

[landagen]

This feature aims to improve management of migrations by providing the ability to consolidate multiple migrations into a single migration allowing the consolidated migrations to be removed from the project.

This can be accomplished by allowing migrations to be grouped into folders. Each folder would consist of one or more migrations and a single model snapshot. New migrations should be creatable between two model snapshots. When a new migration is created between two snapshots the intermediate migrations can be deleted from the project. This would require that all deployments must not reference an intermediate migration. The new snapshot migration would also designate the intermediate migrations it now handles (i.e. an array of strings that match the migration names). This will allow the history table to be managed properly. Folders are not strictly necessary for this to work but provide additional organization. This feature only really requires the ability to have multiple model snapshots, but without folders it may be necessary to designate which snapshot a migration references.

A common use case would be to group by version. At the beginning of a new version of a system, a new group/snapshot would be created. All migrations would be placed in this group. Either prior to production deployment or post deployment the version could be consolidated to be a single migration. As new versions are promoted and deployed, versions no longer referenced by a deployment can be consolidated and removed. The only migrations necessary to maintain would be the migrations referenced by a deployment.

One challenge would be custom SQL in the intermediate migrations. The best way to handle this would be to add all custom SQL statements to the consolidated migration and then for the migration to be edited where appropriate. Another option would be to add a target ID for the custom SQL. This would allow custom SQL that target the same id to be consolidated to only use the latest custom SQL statement. You would only use the same ID if the SQL statement were a full replacement for the previous migration. As an example, SQL migrations of a view are typically full replacements of the view and therefore previous migrations targeting that same view are unnecessary.

[roji]

@landagen overall this sounds like a duplicate of #2174; that issue is a high-level one that doesn't go into specifics (e.g like grouping by directory), but seems to cover what you're trying to describe.

[landagen]

@roji, after reviewing it, I agree. I will close this issue and add my notes to the linked issue.