Source references

In the records that you return to users from your services, you can include source references that contain information about the external source from which the data came. Entity and link records both support the inclusion of source reference objects.

Setting a source reference

To add a source reference to an entity or link, call the setSourceReference() function from within the service's acquire callback.

For example, see the calls to complaintEntity.setSourceReference() and victimOfLink.setSourceReference() in the following code:

const exampleSourceRef = {
  name: 'NYPD Complaint Dataset',
  type: 'Open source data',
  description: 'A collection of open source data about NYPD complaints',
};

addService(
  {
    id: 'getVictims',
    name: 'NYPD Connector: Get all victims',
    description: 'A service that retrieves all the victim data.',
  },
  ({ result }) => {
    const complaintId = `Complaint: 1`;
    const complaintEntity = result.addEntity(Complaint, complaintId);
    complaintEntity.setSourceReference(exampleSourceRef);

    const victimId = `Victim: 1`;
    const victimEntity = result.addEntity(Person, victimId);
    victimEntity.setSourceReference(exampleSourceRef);

    const victimOfLinkId = `VictimOf: 1`;
    const victimOfLink = result.addLink(Victimof, victimOfLinkId, victimEntity, complaintEntity);
    victimOfLink.setSourceReference(exampleSourceRef);
  }
);

Static file serving

The i2 Connect SDK supports serving static files locally from your connector, which is useful for including images, documents, or other resources with your entity and link records. This feature allows connectors to store files locally and reference them in source references.

Setting up static file serving

Before using static file serving, you need to configure your environment and register storage in your connector.

1. Configure storage location

Set the root location where files will be stored:

A. ADT deployment

If using ADT to deploy your connectors then you have 2 options:

Note: You can only use 1 of these options, they can't be used together.

  1. Set the environment variable CONNECTOR_STORAGE_DIRECTORY to some path on your system e.g /home/user1/storage. The storage will be mapped to a subdirectory of this path using the connector ID for the specific connectors being brought up e.g
For connectors with ID's of connector1 and connector2, the path for each will end up:

-> /home/user1/storage/connector1
-> /home/user1/storage/connector2

We will then pass these paths into the respective connectors under the environment variable CONNECTOR_STORAGE_ROOT_LOCATION, so you don't need to set this manually for deployment with ADT.

  1. Set the environment variable CONNECTOR_STORAGE_VOLUME which will create a docker volume containing a subdirectory using the connectorId per connector. Much the same as the above step, the paths to the connectors will be passed through to the respective connectors via CONNECTOR_STORAGE_ROOT_LOCATION automatically.

Refer to the Official ADT documentation that also mentions the above.

B. Local deployment

For this you'll have to get the environment variable to apply specifically to the connector you want this to apply to.

export CONNECTOR_STORAGE_ROOT_LOCATION='/home/user1/storage/connector1'

Note: Use an absolute path for the storage location.

2. Register storage in your connector

Import and call the registerStorage() function at the top level of your connector:

import { registerStorage } from '@i2analyze/i2connect';

const storage = registerStorage();

3. Optional: Add authentication

You can configure additional authentication for file access by providing an auth handler. The auth handler receives the fileId, the identifier of the file being accessed, and requestInformation, the information about the user who made the request.

const storage = registerStorage({
  authHandler: (fileId, requestInformation) => {
    // Access the fileId and user information
    const userCanAccessFile1 = requestInformation.i2AnalyzeUserCacPermissions.includes('example-permission');
    if (fileId === 'file1' && !userCanAccessFile1) {
      // If the user is not authorized, throw an error
      throw new Error('User does not have permission to access this file');
    }
  },
});

If authentication fails, throw an error with a descriptive message. The server will return a 403 response with your error message.

Writing files to storage

Use the writeFile() function to store files locally:

addService(
  {
    id: 'getAll',
    name: 'NYPD Connector: Get all',
    description: 'A service that retrieves all data.',
  },
  async ({ result }) => {
    // Store a file
    await storage.writeFile({ fileId: 'file1', fileContent: 'This is a test file.' });
  }
);

Files are stored at ${CONNECTOR_STORAGE_ROOT_LOCATION}/${fileId}, for example /home/user1/storage/example-connector/report-image-123.png.

Note: File extensions in the fileId affect browser behavior when users access the files. If the fileId includes a recognized extension (e.g., .png, .pdf, .jpg), browsers will handle the file according to that type. Files without extensions may be downloaded rather than displayed inline, even if they contain image data.

Setting static source references

Use setStaticSourceReference() to reference locally stored files in your entities and links:

// For images
entity.setStaticSourceReference({
  fileId: 'image123.png',
  name: 'Evidence Photo',
  fileType: 'image',
  description: 'Photo taken at crime scene',
});

// For documents
entity.setStaticSourceReference({
  fileId: 'document456.pdf',
  name: 'Police Report',
  fileType: 'file',
  description: 'Official incident report',
});

Note: The fileId must match the fileId used when writing the file to storage.

Note: The fileType helps determine which property is set on the source reference. image sets the imageUrl field and file sets the location field on the underlying setSourceReference() function.

Configuring the base URI for a connector

When using setStaticSourceReference we construct a URI that points to the resource in question for a connector. We attempt to fetch this value from request headers. However, connectors can be configured on systems with custom proxies, which can map the URI to what they need, we may not be aware of what decisions have been made. This could potentially result in an incorrect resource reference. To resolve this we have introduced some overrides in case a situation like this occurs.

ADT deployment

You can set the environment variable CONNECTORS_BASE_URI to something like https://test.eia:3000/connectors. ADT is a more controlled environment so this will automatically append the configured proxy path ('example-connector/api' in the above example) and pass an environment variable of FRONT_END_URI to the connectors. This will prevent all header detection and use this path to construct the resourcePath.

Non ADT deployment

Set the environment variable FRONT_END_URI to something like https://test.eia:3000. If you have a custom proxy with path mappings then they need to be added to point exactly to the connectors. Therefore it could be something like https://test.eia:3000/custom/path instead.

If you don't set FRONT_END_URI then we will attempt header detection and you will end up with something like https:test.eia:3000. Again, if you have any proxy path mappings then you need to configure this. You can do so with the environment variable FRONT_END_URI_PATH. This will append that value to the detected protocol, host and port.

NOTE: You can't use both FRONT_END_URI and FRONT_END_URI_PATH. FRONT_END_URI is an override, we will ignore all header detection making FRONT_END_URI_PATH useless in the case that it is used.