Impact of Data Quality on the Performance of Neural Network Incident Detection Models

Abstract

One of the challenges in using field data for the development of neural network incident detection models is to be able to train models that can handle the noisy nature of the loop detector data. The noise in the field data, which may be the result of either a systematic or random error, can have an adverse effect on the performance of an incident detection model, especially in terms of false alarm rate. This paper describes a number of procedures for evaluating the impact of data quality on the performance of a neural network incident detection model that was trained and tested on field data (comprising speed, flow and occupancy measurements) collected from a number of freeways in Melbourne, Australia. Since this model was developed for implementation in an actual system, the paper also reports on a number of techniques and procedures for evaluating the model’s performance in the case of missing or incorrect data either during training or after implementation. In addition to the original research findings reported in this paper, the described procedures are also of interest to practitioners since they address many issues relevant to the implementation of incident detection systems and the quality of the detector data. These issues include evaluating the impact of detector failures, communications malfunction and missing or incorrect data on the model’s performance. In addition, the paper also describes how the same procedures can be used to evaluate the impact of speed data on the model’s performance (ie. the impact of using dual-loop instead of single- loop detectors).