Back up and restore a development database and schemas

When you are developing a configuration that uses an Information Store or Chart Store, you might populate the database with demonstration or test data and charts. You can use the tooling provided to back up the Information Store database that is associated with a config and linked to the schemas within your configuration. The Information Store database is contained in a Docker volume. Because Docker is not designed for permanent data storage, you can back up an Information Store to your local file system.

Backup location

The backups are stored in the /backups directory. A sub-directory is created for each config name, with another sub-directory for each backup name. The backup file is called ISTORE.bak for SQL Server or ISTORE.pgb for Postgres.

For example:

- backups
  - <config_name>
    - <backup_name>
      - ISTORE.bak
    - <backup_name>
      - ISTORE.bak

Creating a release and backup

Use the deploy script with the release option to create a backup. The release change set mechanism is used to create a release of your current schema files and a backup of the database. This ensures that the schema files and your database remain synchronized. For more information about the change set feature, see Database scripts and configuration.

The following usage pattern shows how you create a backup:

deploy -c <config-name> -r

The release change set is created in the <config-name>/change-sets directory, with a name in the following structure: <number>-release-<date>-<time>.

The backup is created in the backups/<config-name>/<number>-release-<date>-<time> directory.

Restoring from a release

Use the deploy script with the release option to restore the system to a previous release. The schema files from the release change set are restored in your configuration and the associated release backup is restored.

If any change sets have been created since the previous release, the change sets are deleted when you restore to a previous release. For more information about the change set feature, see Database scripts and configuration.

The following usage pattern shows how you create a backup:

deploy -c <config-name> -r <release-change-set-number>

For example, to restore release change set 1:

deploy -c <config-name> -r 1

Adding data after a release

If you add data to your Information Store after you create a release change set, you need to create a new backup. The process that you need to follow depends on the current state of your environment:

  • If you have made changes to the schema files, repeat the previous process to create a new release and associated backup.
  • If you have not made any changes to the schema files, and do not have any change sets since the last release, then use the following instructions to take a backup and later restore.

Creating a backup

Use the deploy script to create your backup. The following usage pattern shows how you create a backup:

deploy -c <config_name> -t backup -b <backup_name>

If you do not provide a backup name, the backup is created in a directory called default.

For example, to create a backup called test-1 for the config-development config:

deploy -c config-development -t backup -b test-1

Restoring from backup

Use the deploy script to restore from a backup. The following usage pattern shows how you restore from a backup:

deploy -c <config_name> -t restore -b <backup_name>

For example, to restore a backup called test-1 for the config-development config:

deploy -c config-development -t restore -b test-1

Using external database backups

You can use the backup-database-utility to create and restore backups from outside of analyze-deployment-tooling. This means that you can store your database backups separately from your configs by using S3-like storage or a shared drive.

The backup-database-utility supports using S3-like storage or filesystem storage.

Backup and restore with S3-like storage

The s3cmd option allows you to backup to any S3-like system supported by the S3cmd tool. For example: AWS S3, Azure Blob Storage, Google Cloud Storage, and DigitalOcean Spaces.

The following example describes how to configure AWS S3 to store your backups:

  1. In AWS, create a user with access to the following:
    • s3:ListAllMyBuckets
    • s3:GetBucketLocation
    • s3:ListBucket
    • s3:GetObject
    • s3:PutObject
  2. Create an access key ID and secret access key for the user by following the steps in AWS Account and Access Keys.
  3. Configure your local system to access your AWS S3 storage. You can either assign the user's access key ID and secret values to the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Additionally, you can run backup-database-utility -t configure-s3cmd that creates the .s3cfg configuration file in the root of the project where you can specify the access key ID and secret. > Important: This file contains secrets in plain text. You must store this file appropriately.
  4. Create an S3 bucket to store your backups. For example, analyze-deployment-tooling-backups

To create a backup, run the command with the following arguments:

backup-database-utility -c <config_name> -s s3cmd -p "s3://analyze-deployment-tooling-backups/" -t backup -b <backup_name>

If a backup with that name already exists for the config, it will be copied to the location specified for -p.

For example, to create a backup named fraud for the config-development config in the analyze-deployment-tooling-backups bucket:

backup-database-utility -c config-development -s s3cmd -p "s3://analyze-deployment-tooling-backups/" -t backup -b fraud

You can restore the above backup by using the -t restore flag. For example:

backup-database-utility -c config-development -s s3cmd -p "s3://analyze-deployment-tooling-backups/" -t restore -b fraud

Backup and restore with filesystem storage

The filesystem option allows you to backup to a shared network drive or other filesystem outside of analyze-deployment-tooling or your shared configs.

To create a backup, run the command with the following arguments:

backup-database-utility -c <config_name> -s filesystem -p <path> -t backup -b <backup_name>

If a backup with that name already exists for the config, it will be copied to the location specified for -p. The utility attempts to create the folder if it does not already exist.

For example, to create a backup named fraud for the config-development config in the user's OneDrive folder:

backup-database-utility -c config-development -s filesystem -p "/mnt/c/Users/<user_name>/OneDrive/backups" -t backup -b fraud

You can restore the above backup by using the -t restore flag. For example:

backup-database-utility -c config-development -s filesystem -p "/mnt/c/Users/<user_name>/OneDrive/backups" -t restore -b fraud