Skip to content

Commit

Permalink
Replicate data from staging, not integration
Browse files Browse the repository at this point in the history
Since the env sync was ported over to govuk-helm-charts, not all
apps have backups saved to s3 on integration (e.g. note the lack
of `backup` operation [here](https://github.com/alphagov/govuk-helm-charts/blob/cae5e10baf0f67423bbebf66454da737da787d37/charts/db-backup/values.yaml#L767-L773)).
This means that the data replication scripts fail, as they're
looking for a bucket that does not exist.

In contrast, all data is backed up to s3 on staging, so it should
be safe to pull from there by default. The only downside is that
integration access is given to devs by default, but that access to
staging and production required Production Admin access. This is
something that can hopefully be ironed out in the move to using
govuk-helm-charts for local development, over the coming months.
For now it seems better to fix the script for prod-admin users
than to have it broken for everyone.

NB I've also removed the unnecessary duplication of replication
examples where one used a readonly role and one used a poweruser
role. We should use principle of least privilege, so only show
the readonly example.
  • Loading branch information
ChrisBAshton committed Jul 12, 2024
1 parent d1d6271 commit a930b80
Show file tree
Hide file tree
Showing 4 changed files with 5 additions and 9 deletions.
2 changes: 1 addition & 1 deletion bin/replicate-mongodb.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ case "$app" in
;;
esac

folder="govuk-integration-database-backups/$instance"
folder="govuk-staging-database-backups/$instance"

archive_dir="${replication_dir}/mongodb"
archive_file="${database}.gz"
Expand Down
2 changes: 1 addition & 1 deletion bin/replicate-mysql.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ app="${1//_/-}"

replication_dir="${GOVUK_DOCKER_REPLICATION_DIR:-${GOVUK_DOCKER_DIR:-${GOVUK_ROOT_DIR:-$HOME/govuk}/govuk-docker}/replication}"

bucket="govuk-integration-database-backups"
bucket="govuk-staging-database-backups"
archive_dir="${replication_dir}/mysql"
archive_file="${app//-/_}_production.dump.gz"
archive_path="${archive_dir}/${archive_file}"
Expand Down
2 changes: 1 addition & 1 deletion bin/replicate-postgresql.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ app="${1//_/-}"

replication_dir="${GOVUK_DOCKER_REPLICATION_DIR:-${GOVUK_DOCKER_DIR:-${GOVUK_ROOT_DIR:-$HOME/govuk}/govuk-docker}/replication}"

bucket="govuk-integration-database-backups"
bucket="govuk-staging-database-backups"

archive_dir="${replication_dir}/postgresql"
archive_file="${app//-/_}_production.dump.gz"
Expand Down
8 changes: 2 additions & 6 deletions docs/how-tos.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ gem "govuk_publishing_components", path: "../govuk_publishing_components"

## How to: replicate data locally

There may be times when a full database is required locally. The following scripts in the `bin` directory allow replicating data from integration:
There may be times when a full database is required locally. The following scripts in the `bin` directory allow replicating data from staging:

- `replicate-elasticsearch.sh`
- `replicate-mongodb.sh APP-NAME`
Expand All @@ -46,11 +46,7 @@ There may be times when a full database is required locally. The following scri
You will need to assume-role into AWS using the [gds-cli](https://docs.publishing.service.gov.uk/manual/access-aws-console.html) before running the scripts. For example, to replicate data for Content Publisher, run:

```
# as an AWS PowerUser...
gds aws govuk-integration-poweruser --assume-role-ttl 180m ./bin/replicate-postgresql.sh content-publisher
# as an AWS User...
gds aws govuk-integration-readonly --assume-role-ttl 180m ./bin/replicate-postgresql.sh content-publisher
gds aws govuk-staging-readonly --assume-role-ttl 180m ./bin/replicate-postgresql.sh content-publisher
```

All the scripts, other than `replicate-elasticsearch.sh`, take the name of the app to replicate data for.
Expand Down

0 comments on commit a930b80

Please sign in to comment.