Write a report of the project in detail along with what they do it:

Curbing Crimes in Urban Areas Using Emerging Computing Technologies

We will design an approach to detect and classify crimes based on IoT-enabled technologies and open city data in an accurate and automatic manner. We build our detection and classification model based on features extracted from multiple sources and context, including video surveillance, sensed data, historical crime data, and 911 reports. We exam three types of crimes in our approach: gunshots, robbery, and property theft.

In the case of gunshot, criminal and victims are involved in the crime scene. Our classifiers extract and analyze scenario and context information from camera videos, acoustic gunshot sensors, location of the incident, timeframe of the incident, historical occurrence of similar incidents, and prior and current social media data. In the case of robbery, our classifier extracts scenario-based features from camera videos such as moving speed of target, acoustic sensor, neighborhood information, historical police reports, and 911 data. In the case of property theft, our classifier uses features and objects extracted from videos, past occurrence frequency of similar crimes, location information, time information including season information.

A hybrid deep learning method (HDL) networks can be repurposed from one application to another, therefore, handling heterogeneous raw input data. Our system has different modal of features, and we need learn representation features from each modality and use fused high level representation to build classifiers. The hybrid deep learning also works well when some modalities are absent. Wedetect and classify crime behaviors into gunshot, robbery, and property theft. Scenario features, such as crowd motion, gait speed, facial features, objects, license plates, sound, light, will be extracted from video feeds and sensors. Scenario features together with open city data, situational data such as location and time will be used in classifiers to detect and classify crime incidents. We will implement the following three important deep learning tasks

Designing a context-aware classifier to automatically detect and classify crime behaviors

To efficiently and accurately classify the observed crime-incidents, a hybrid deep-learning (HDL) algorithm will be investigated in this work. HDL is featured with the following techniques: (1) HDL is capable of learning from multiple data with heterogeneous modalities simultaneously; those data will jointly will provide classifier with a panoramic and full-spectrum description about the occurring incidents.  (2) HDL is equipped with different deep-learning algorithms to learn from heterogeneous modality. In this work, Deep Convolutional Neural Network (DCNN)will be used to learn from visual media such as video and images because it demonstrates superior performance (high accuracy and short training time) on matrix-oriented feature-learning; Recurrent Neural Network (RNN)will be used to learn from streaming data such as acoustic signal or vibration signals because RNN exhibits dynamic temporal behavior (enabled by the directed cycle inside RNN); Deep Boltzmann Machine (DBM)will be used to learn from textual information such as scenario features, social network, and personal and situational factors, etc.  (3) Deep learning algorithms always learn the upper-level features from lower ones, the input data with heterogeneous modality will eventually fuse at upper layers with somewhat homogeneous modality. Therefore, a unified algorithm like DBM can be used in upper-level feature-learning.

Scoring and ranking threats of detected crimes:

The crime incidents will be scored and ranked by rubrics declared in the Uniform Crime Reporting Program (UCR, https://www.fbi.gov/about-us/cjis/ucr/ucr) and the National Crime Victimization Survey Program (NCVS,http://www.bjs.gov), after being categorized into one of three classes: gun-shot, robbery, and property theft. UCR and NCVS are two statistical programs to measure the magnitude, nature, and impact of various crimes in U.S., and administrated by FBI and U.S. Department of Justice respectively. Although deep learning classifier can be employed as a completely unsupervised machine learning technique, we will acquire labeled training data from the UCR and NCVS program database, and from video-sharing websites such as YOUTUBE and LiveVideo to increase accuracy of classifier. Additionally, we have Chattanooga Police Department and Campus Police at the University of Tennessee at Chattanooga serve on our advisory board (see the attached support letters), and they will provide expert advice from domain knowledge, which are essential in accuracy and effectiveness of our system.