New York Police Department (NYPD) Connector

The NYPD connector connects to the NYPD Complaint Data and marshals the data into entities, links and properties. The dataset contains information on the felony, misdemeanor, and violation crimes reported to the New York City Police Department.

There are a number of fields in the dataset, of which the following were used:

Column Name	Type
CMPLNT_NUM (Complaint Number)	Number
ADDR_PCT_CD (Precinct Code)	Number
BORO_NM (Borough Name)	Plain Text
CMPLNT_FR_DT (Complaint From Date)	Date & Time
CMPLNT_FR_TM (Complaint From Time)	Plain Text
CMPLNT_TO_DT (Complaint To Date)	Date & Time
CMPLNT_TO_TM (Complaint To Time)	Plain Text
CRM_ATPT_CPTD_CD (Crime Attempted-Completed Code)	Plain Text
HADEVELOPT (Housing Development)	Plain Text
JURISDICTION_CODE (Jurisdiction Code)	Number
JURIS_DESC (Jurisdiction Description)	Plain Text
KY_CD (Key Code)	Number
LAW_CAT_CD (Law Category Code)	Plain Text
LOC_OF_OCCUR_DESC (Location of Occurrence Description)	Plain Text
OFNS_DESC (Offense Description)	Plain Text
PARKS_NM (Park Name)	Plain Text
PATROL_BORO (Patrol Borough)	Plain Text
PD_CD (PD Code)	Number
PD_DESC (PD Description)	Plain Text
PREM_TYPE_DESC (Premise Type Description)	Plain Text
RPT_DT (Report Date)	Date & Time
STATION_NAME (Station Name)	Plain Text
SUSP_AGE_GROUP (Suspect's Age Group)	Plain Text
SUSP_RACE (Suspect's Race)	Plain Text
SUSP_SEX (Suspect's Sex)	Plain Text
TRANSIT_DISTRICT (Transit District)	Number
VIC_AGE_GROUP (Victim's Age Group)	Plain Text
VIC_RACE (Victim's Race)	Plain Text
VIC_SEX (Victim's Sex)	Plain Text
Latitude	Number
Longitude	Number

Data model

The NYPD schema models the NYPD Complaint Dataset by using its relevant fields. Each row of data can be represented by three entities (Complaint, Person and Location) and a number of appropriate links between them alongside properties extracted from each field.

The schema (and charting schemes) for the NYPD connector can be found in the schema directory of this repository.

Where Property Type is the name of the schema property, Logical Type is the property's data type, and Derived From is the field of the external dataset where the property is derived from.

Entity: Complaint

Represents a crime complaint.

Property Type	Logical Type	Derived From
Complaint Number	SINGLE_LINE_STRING	CMPLNT_NUM
Complaint Start Date	DATE	CMPLNT_FR_DT
Complaint End Date	DATE	CMPLNT_TO_DT
Complaint Start Time	TIME	CMPLNT_FR_TM
Complaint End Time	TIME	CMPLNT_TO_TM
Crime Status	SUGGESTED_FROM	CRM_ATPT_CPTD_CD
Jurisdiction Code	INTEGER	JURISDICTION_CODE
Jurisdiction Description	SINGLE_LINE_STRING	JURIS_DESC
Offence Classification Code	SINGLE_LINE_STRING	KY_CD
Level of Offence	SUGGESTED_FROM	LAW_CAT_CD
Offence Description	SINGLE_LINE_STRING	OFNS_DESC
Internal Classification Code	INTEGER	PD_CD
Classification Description	SINGLE_LINE_STRING	PD_DESC
Event Date	DATE	RPT_DT
Location of Occurrence	SUGGESTED_FROM	LOC_OF_OCCUR_DESC

Entity: Location

Represents the location of a reported crime.

Property Type	Logical Type	Derived From
Precinct Code	INTEGER	ADDR_PCT_CD
Borough Name	SINGLE_LINE_STRING	BORO_NM
Housing Development	SINGLE_LINE_STRING	HADEVELOPT
Park Name	SINGLE_LINE_STRING	PARKS_NM
Patrol Borough	SINGLE_LINE_STRING	PATROL_BORO
Premises Description	SINGLE_LINE_STRING	PREM_TYP_DESC
Station Name	SINGLE_LINE_STRING	STATION_NAME
Transit District	INTEGER	TRANSIT_DISTRICT
Coordinates	GEOSPATIAL	Latitude & Longitude

Entity: Person

Represents a person somehow involved in a reported crime. Since each record of the NYPD Complaint Dataset has information on both victims and suspects, two Person entities are created from a single record.

Property Type	Logical Type	Derived From
Age Group	SUGGESTED_FROM	SUSP_AGE_GROUP or VIC_AGE_GROUP
Race	SINGLE_LINE_STRING	SUSP_RACE or VIC_RACE
Sex	SUGGESTED_FROM	SUSP_SEX or VIC_SEX

Links

Establishes some connection between a Complaint, a Location and a Person.

Link Type	Link Ends
Located At	Complaint -> Location
Suspect Of	Person -> Complaint
Victim Of	Person -> Complaint

Setup

These instructions are for setting up and running the NYPD connector.

If you are not familiar with deploying i2 Analyze with the i2 Connect gateway and have not previously done so, you must do so now.

If you want to configure your connector to use an Information Store schema, you must deploy i2 Analyze with the Information Store and the i2 Connect gateway.

1. Add connector to topology

In your topology.xml file in toolkit\configuration\environment, add a new <connector-id> element for the NYPD connector:

<wars>
  <war ... >
    ...
    <connector-ids>
      <connector-id value="nypd-connector"/>
    </connector-ids>
    ...
  </war>
</wars>

Additionally, add a new <connector> to the topology:

<ns1:topology ...>
  ...
  <connectors>
    <connector base-url="http://localhost:9081" name="NYPD Connector" id="nypd-connector"/>
  </connectors>
</ns1:topology>

Ensure that you are using the same port specified in the nypd\nypd-connector\src\main\resources\application.properties file in this repository (9081 by default), and that the value of the id attribute is the same as the value attribute of its corresponding <connector-id>.

2. Configure the schema

Choose whether you want to configure the NYPD schema as a connector schema, a gateway schema, or an Information Store schema.

Connector schema

By default, your connector has the NYPD schema configured as a connector schema. schemaUrl, chartingSchemesUrl, and schemaShortName are defined in the NYPD connector's config.json file, in nypd\nypd-connector\src\main\resources.

Additionally, ensure the following:

The connector's config.json does not contain a gatewaySchema property;
The NYPD <connector> element in your i2 Analyze topology does not contain a gateway-schema attribute.

See for more information on configuring a connector schema.

Gateway schema

If you want to set up the NYPD schema as a gateway schema, follow the guidelines for configuring a gateway schema using the NYPD schema and charting scheme found in the schema directory of this repository.

Additionally, ensure the following:

The connector's config.json in nypd\nypd-connector\src\main\resources does not contain the schemaUrl, chartingSchemesUrl and schemaShortName properties. These properties exist on the configuration by default; if they are present, remove them.
The NYPD <connector> element in your i2 Analyze topology does not contain a schema-short-name attribute.

Information Store schema

If you want to set up the NYPD schema as an Information Store schema:

Copy the NYPD schema and charting scheme found in the schema directory of this repository to the configuration/fragments/common/WEB-INF/classes directory in your i2 Analyze configuration.
Open the ApolloServerSettingsMandatory.properties file in the same directory, and update the following properties to point to the relevant files
```
SchemaResource=nypd-complaint-data-schema.xml
ChartingSchemesResource=nypd-complaint-data-schema-charting-schemes.xml
```

Additionally, ensure the following:

The connector's config.json in nypd\nypd-connector\src\main\resources does not contain the following properties that exist by default; if they are present, remove them:
- schemaUrl
- chartingSchemesUrl
- schemaShortName
- gatewaySchema
The NYPD <connector> element in your i2 Analyze topology does not contain the following attributes:
- schema-short-name
- gateway-schema

3. Acquire Socrata token

In order to query the external data source, a Socrata app token is required. If you do not already have a Socrata app token, you will need to generate one. Instructions on how to generate this token can be found here.

In the NYPD connector's application.properties file at connector\nypd\nypd-connector\src\main\resources, add your token.

server.port=9081
socrata.url=https://data.cityofnewyork.us/resource/7x9x-zpz6.json

# API Token. Create a Socrata account and create an API Token. Paste it here
socrata.api.token=

3. Run the NYPD connector

To run the connector, navigate to connector\nypd\nypd-connector in your terminal and run the application using the following command:

mvnw spring-boot:run

For more information on running this repository's Java connectors, see running example connectors in Java.

4. Deploy and start i2 Analyze

Deploy and start the Liberty server.

setup -t deployLiberty
setup -t startLiberty