Researchers at the Federal Highway Administration are using extremely large datasets to identify and understand complex transportation issues that can impact efficiency, cost, and safety. New automated tools for data extraction and analysis are needed to help make these massive datasets accessible to the widest possible range of researchers, academia, and industry. 

The second Strategic Highway Research Program’s (SHRP2) naturalistic driving study (NDS) demonstrates the immense scale of data gathered in recent years. The study focused on driver behavior and addressed the notion that it is possible to obtain information on what people do when they drive on a day-to-day basis. The study involved more than 3,000 volunteer drivers and their vehicles, each of which was equipped with four cameras, GPS, and various sensors. Over a two-year period, NDS researchers gathered more than 1.2 million hours and more than two petabytes of data, the majority of which came from video captured by onboard cameras.

The massive size of the NDS video data resulted in a bottleneck that made traditional methods for identifying and extracting features of interest in video inadequate. Traditionally, researchers manually logged the location in the video where each feature of interest was found; however, it is estimated that it would take almost 600 technicians a full year to manually analyze all the video in the NDS.

EAR Investment in Computer Vision

Computer vision, which uses artificial intelligence algorithms to perform visual perception tasks, offers a more efficient method for analyzing video data. The Exploratory Advanced Research (EAR) Program has funded computer vision research on SHRP2 safety data at the Department of Energy’s (DOE) Oak Ridge National Laboratory (ORNL) via an interagency agreement. One of the most successful ORNL efforts focused on continuing development of a highly flexible data processing framework for ingesting image data and coordinating the application of multiple facial recognition models to process that data. This framework includes a variety of machine learning tools, ranging from face detection and characterization to body pose attributes. These tools can be used to detect facial attributes, such as looking forward and checking mirrors, and expressions or body movements, such as turning or reaching. ORNL calls this data processing framework “Face Recognition from Oak Ridge,” or FaRO. 

Innovations Using FaRO Computer Vision Algorithm

There have been several collateral innovations that have resulted from the development of FaRO. ORNL has used FaRO internally for enhancing unmanned aerial system capability modeling via a portable system that performs on-device object detection and avoidance. FaRO is also used by the DOE’s Office of Energy Efficiency and Renewable Energy for controlling access to the network—an application that uses industrial cameras to perform real-time video analytics of biometrics data. Other organizations, including a university, a commercial entity, and Federal intelligence agencies are now interested in using FaRO. The software is available through an open-source license and is freely distributed. Resultantly, the return on investment of these spinoff uses will continue to increase the value of the public funds invested. 

EAR Supports Cutting-Edge Research/Looking Ahead

FHWA’s goals are both short and long term. In the short term, FHWA wants to extract value from the NDS data. In the long term, FHWA wants to ensure that the data being collected will improve transportation safety. The EAR Program seeks to leverage advances in science and engineering that could lead to breakthroughs for critical current and emerging issues in highway transportation by supporting a community of experts from different disciplines who have the talent and interest to research solutions but who likely would not do so without EAR Program funding.

Mary Huie is the manager of the Technology Transfer Program at FHWA’s Turner-Fairbank Highway Research Center. She holds a B.S. in civil engineering from the Catholic University of America in Washington, DC.

Adil Anis is a program manager working as a contractor with FHWA’s Technology Transfer Program. He holds a B.S. in computer engineering from George Mason University in Fairfax, VA.

For more information, see https://highways.dot.gov/research/research-programs/exploratory-advanced-research/exploratory-advanced-research-overview or contact Mary Huie, 202-493-3460, mary.huie@dot.gov.