Synchronize configurations

You can synchronize configurations between the i2 Analyze deployment toolkit and the configuration development environment. This enables you to develop the i2 Analyze configuration by using the config development environment, and deploy with it as part of the i2 Analyze deployment toolkit. This can be useful in the following situations:

You want to create a new deployment that uses the i2 Analyze deployment toolkit and you want to use the config development environment to develop the configuration for it.
You have an existing deployment that uses the i2 Analyze deployment toolkit and you want to use the config development environment to further develop the configuration.

When you synchronize a configuration, you must do so with a complete i2 Analyze deployment toolkit. You can synchronize from an existing deployment toolkit configuration (import) or from an existing configuration development config (export).

The configuration from the i2 Analyze deployment toolkit is the source of truth. When you have an existing deployment toolkit, the instructions direct you to back up the configuration before you synchronize it with the config development environment.

You complete the synchronization by using the manage-toolkit-configuration script.

Examples

The following use cases demonstrate the process of synchronizing configurations between the config development environment and an i2 Analyze deployment toolkit:

New deployment: You want to create a new deployment that uses the i2 Analyze deployment toolkit and you want to use the config development environment to develop the configuration for it.
Existing deployment: You have an existing deployment that uses the i2 Analyze deployment toolkit and you want to convert to use the config development environment to further develop the configuration. After you develop the configuration, you will still deploy i2 Analyze using the deployment toolkit.

New deployment

This section describes the process of exporting a configuration from the config development environment to an i2 Analyze deployment toolkit.

Use the config development environment to create and develop a config.
For more information, see Configuration development environment. When you are happy with the developed configuration, you can complete the process that enables you to use it with the i2 Analyze deployment toolkit.
Install the i2 Analyze deployment toolkit.
For more information, see Installing i2 Analyze.
Update the path-configuration.json file inside the analyze-deployment-tooling directory to reference the path where you installed the i2 Analyze toolkit:
Open a terminal on your host machine.
- On Windows, use a WSL terminal. Press Start -> wsl
Navigate to the analyze-deployment-tooling/scripts directory.
Run the configure-paths script with the external task.
For example:
```
configure-paths -t external -n <name> -p <path>
```
Where <name> is a name used to reference the external toolkit and <path> is the path where you installed the toolkit.
Reopen the analyze-deployment-tooling folder in a dev container.
Press F1 (or Cmd+Shift+P in MacOS), type Dev Containers: Rebuild and Reopen in Container and select it
Export the configuration from the config development environment to the i2 Analyze deployment toolkit.
1. Run the manage-toolkit-configuration command with the create task to create a base configuration in the i2 Analyze deployment toolkit.
```
manage-toolkit-configuration -c config-development -n toolkit -t create
```
  Where:
  - -c is the name of the config.
  - -n is the name of the i2 Analyze deployment toolkit that was added to the path-configuration.json file.
  Note: The base configuration that is created is suitable for the deployment pattern that is specified in the variables.conf file of the specified config.
2. Run the manage-toolkit-configuration command with the export task to export the config dev configuration into the deployment toolkit.
```
manage-toolkit-configuration -c config-development -n toolkit -t export
```
  Where:
  - -c is the name of the config to export.
  - -p is the absolute path to the root of the i2 Analyze deployment toolkit that was added to the path-configuration.json file.
You can then deploy your i2 Analyze deployment toolkit configuration. For more information about deploying i2 Analyze, see Deploying i2 Analyze. Start at step 3 from the linked documentation. There are a number of environment specific configuration files that are not populated in the config development environment.

Existing deployment

This section assumes that you are starting with a configuration from an i2 Analyze deployment toolkit. For more information about deploying i2 Analyze, see Deploying i2 Analyze.

The config development environment uses fixed names for schemas and configuration files. Before you can import your deployment toolkit configuration into the config development environment, your configuration must be updated to use the expected file names. The manage-toolkit-configuration command can rename the files for you and update any references to them.

You can also decide whether or not to use the config development environment configuration set in your deployment toolkit configuration. If you decide to use the new config set, all settings for the deployment must be specified in the analyze-settings.properties file. If you choose not to use the new configuration set, any additions that you make to the analyze-settings file in the config development environment are not exported.

Back up your i2 Analyze deployment toolkit configuration.
Fore more information about backing up your configuration, see Back up and restore the configuration.
Use the config development environment to create an empty config to import your configuration in to.
For more information, see Configuration development environment. You do not need to specify any schema files or deploy the config.
Update the path-configuration.json file inside analyze-deployment-tooling directory to reference the path where you installed the i2 Analyze toolkit:
Open a terminal on your host machine.
- On Windows, use a WSL terminal. Press Start -> wsl
Navigate to the analyze-deployment-tooling/scripts directory.
Run the configure-paths script with the external task.
For example:
```
configure-paths -t external -n <name> -p <path>
```
Where <name> is a name used to reference the external toolkit and <path> is the path where you installed the toolkit.
Reopen the analyze-deployment-tooling folder in a dev container.
Press F1 (or Cmd+Shift+P in MacOS), type Dev Containers: Rebuild and Reopen in Container and select it
Run the manage-toolkit-configuration command with the prepare task to rename the schema and configuration files and update the references:
```
manage-toolkit-configuration -c config-development -n toolkit -t prepare
```
Where:
- -c is the config name you are working with.
- -n is the name of the toolkit to act on.
To use the config development environment configuration set, complete the following 2 steps. Otherwise, move to the next step.
1. Copy the contents of templates/toolkit-config-mod directory to the /toolkit/configuration/fragments/common/WEB-INF/classes directory of the i2 Analyze deployment toolkit.
2. Review the settings that are used in the DiscoClientSettings, DiscoServerSettingsCommon, and ApolloServerSettingsMandatory properties files of i2 Analyze deployment toolkit and move any settings that you want to continue using into the /toolkit/configuration/fragments/common/WEB-INF/classes/analyze-settings.properties file.
  Note: You do not need to move any of the Resource settings from the ApolloServerSettingsMandatory file that specify the names of XML files. You can remove the DiscoClientSettings and DiscoServerSettingsCommon properties files after you move any settings.
Run the manage-toolkit-configuration command with the import task to import the configuration into the config development environment:
```
manage-toolkit-configuration -c config-development -n toolkit -t import
```
Where:
- -c is the config name to import the configuration into.
- -n is the name of the toolkit to act on.
Use the config development environment to develop the configuration.
For more information, see Developing the configuration.
1. In the config development environment, a user.registry.xml is required. You can start from the example provided in pre-reqs/i2analyze/toolkit/examples/security and align it to your security schema or copy in an existing one.
If you want to use your developed configuration in your i2 Analyze deployment toolkit, you can use the export task to export the configuration into your toolkit. Otherwise, continue to use the config development environment.
Run the manage-toolkit-configuration command with the export task. For example:
```
manage-toolkit-configuration -c config-development -n toolkit -t export
```
Where:
- -c is the config name you have created to synchronize the configuration to.
- -n is the name of the toolkit to act on.
You must update the i2 Analyze deployment toolkit deployment with the changes to the configuration.

Connectors

In the config development environment, i2 Connect connectors are run on Docker images and can only be access from within the Docker network. This means that the connectors and URL references to the connectors cannot be synchronized between configurations.

The connector IDs are displayed when you run the import and export tasks to enable you to provide the connectors in each environment. For more information about deploying connectors, see:

Extensions

In the config development environment, i2 Analyze extensions are built and deployed using Maven.

To add an extension to your i2 Analyze deployment toolkit, copy the JAR from i2a-extensions/<extension-name>/target/<extension-name>-<version>.jar and follow the Developer Essentials deployment steps.

Note: You must also copy the JAR files from any extensions that the extension depends on to your deployment toolkit.

To import extensions into the config dev environment, complete the instructions in Developing extensions.

Plugins

Plugins can be synchronized between analyze-deployment-tooling configurations and deployment toolkit configurations.

When you import a configuration that contains plugins, those plugins are imported to the i2a-plugins directory in the config dev environment and referenced in the plugin-references.json file of the configuration.

When you export a configuration that references plugins, the referenced plugins are added to the exported configuration from the i2a-plugins directory.

If a plugin with the same name already exists in the i2a-plugins directory, it is not overwritten or added to the plugin-references.json file.

Prometheus and Grafana

In the config development environment, Prometheus and Grafana are run on Docker images. You can modify the configs/<config-name>/configuration/prometheus/prometheus.yml file and the Grafana dashboards in configs/<config-name>/configuration/grafana/dashboards folder to further develop your metrics configuration and once finished this can be copied to the locations that are required by Prometheus and Grafana. For more information, see Configure server monitoring.

manage-toolkit-configuration command

Usage:
  manage-toolkit-configuration -c <config_name> -n <toolkit_path_name> -t { create | prepare | import | export } [-v]

Options:
  -c <config_name>             Name of the config to use.
  -n <toolkit_path_name>       The name of the path to the root of an i2 Analyze deployment toolkit as shown in the path-configuration.json file.
  -t {create}                  Creates a configuration in the i2 Analyze deployment toolkit that can be imported into the config development environment.
  -t {prepare}                 Prepares an existing i2 Analyze deployment toolkit configuration to be imported into the config development environment.
  -t {export}                  Export a config development environment configuration to an i2 Analyze deployment toolkit configuration.
  -t {import}                  Import an i2 Analyze deployment toolkit configuration to a config development environment configuration.
  -v                           Verbose output.
  -h                           Display the help.