Migration
Migrating to v7 from v6
All of the datasets need to be moved to a layer as part of the v7 migration.
The migration script carries this out, along with other operations.
To execute it, you'll need to decide:
- Which layer the existing datasets should be moved to.
- What the full complement of layers in your rAPId instance should be.
Prerequisites
Local requirements
You will need the ability to run Batect
, the requirements for which are listed here.
Steps:
Make the infrastructure changes
The v7 infrastructure changes need to be applied to your rAPId instance.
- Add the
layers
variable to the rAPId cluster module.layers
will be a list of the layers you wish to use in your rAPId instance. You can omit this if you just want to use thedefault
layer. - Change the rAPId module source to:
git@github.com:no10ds/rapid.git//infrastructure/modules/rapid
- Update both the application_version and ui_version variables to
v7.0.4
Apply these changes.
Clone the repo
To do this, run:
git clone -b v7.0.4 git@github.com:no10ds/rapid.git
Set your environment variables
Within the rAPId repo, set the following variables in the .env
file to match those of your rAPId instance and AWS account:
# rAPId instance variables
AWS_REGION=
DATA_BUCKET=
RESOURCE_PREFIX=
# AWS environment variables
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_SESSION_TOKEN=
Run the migration script
You can now run the script and specify your layer configuration. Examples for it are below:
Example 1:
You do not wish to use the layer functionality:
- The existing datasets can be moved to a
default
layer - The full complement of layers can just consist of one, called
default
.
To do this, you would run:
make migrate-v7 layer=default all-layers=default
Example 2:
You wish to use the layer functionality and largely have raw data already in your rAPId instance:
- The existing datasets can be moved to a
raw
layer. - The full complement of layers in your rAPId instance can mirror your architecture and be:
raw
,curated
andpresentation
To do this, you would run:
make migrate-v7 layer=raw all-layers=raw,curated,presentation