The research consists of developing a four-layered processing framework to detect and track facial features (if possible); to detect the head in the remaining frames; to replace the head with an avatar in all frames; and to evaluate the confidence of the identity masking. A graphical user interface (GUI) will be developed that a person can use to verify success.

The implementation of the four-layered processing consists of nine tasks:

• Collect samples of videos from the second Strategic Highway Research Program (SHRP2) 24-car (SHRP2-24) study dataset and retrain the face detector.
• Track and extract facial features from the SHRP2-24 data by looking at relative movements of tracked facial points.
• Track and extract face and head pose from the SHRP2-24 data using the random sample consensus (RANSAC) method and a three-dimensional (3D) face model.
• Develop and apply the capability to track eye gaze without an infrared (IR) camera and with low-resolution video, which will be developed and applied to the SHRP2-24 dataset using an adaptive appearance-based eye gaze estimation method.
• Develop a method to interpolate head position in frames that the tracker missed using dense-trajectory-based interpolation methods and apply it to the SHRP2-24 data.
• Develop the capability to synthesize facial motion on a computer-generated avatar.
• Develop the capability to render avatars over the videos at the appropriate head location for identity masking.
• Develop a method to detect (not mask) nonfacial elements that obscure the face using fine-grained alpha-mask extraction.
• Develop a GUI tool to obtain confidence of masking over the entire video, which will also allow a user to quickly view the lowest confidence frames in the masked video.

Goals:

To develop an automated, complete, and irreversible identity-masking system for processing realistic, low-resolution video in the larger second SHRP2 safety video dataset that shall preserve head pose, eye and mouth movement, facial features, and eye gaze information. The masking could be applied to the larger SHRP2 safety video data of drivers and passengers as well as similar naturalistic driving data studies.

• Exploratory Advanced Research

• FY 2002-2022 / Exploratory Advanced Research