First-person vision goggles are poised to become the next evolution in drone use for inspecting superstructures.

First-person vision (FPV) goggles. The phrase may garner blank stares from many people who are not yet familiar with the technology. Even those who are familiar with it may have limited knowledge. Aren’t they those gadgets that can be used with a cell phone? Or, aren’t they what kids use when playing video games? Well, yes, they are most commonly known as toys or tech accessories for computer-generated virtual reality and video games. However, they have the potential to be much more than a toy when paired with a drone to help supplement bridge inspections.

Connected directly to a drone’s video feed, FPV goggles give bridge inspectors a first-person visual that is equivalent to being about 3 feet (0.9 meters) away from the components they are inspecting. This is accomplished by positioning the drone 6 feet (1.8 meters) from the component and utilizing the two times digital zoom. The visual image is so clear and immersive it gives the inspector a relative image size equivalent to an 18-foot (5.5-meter), 1080 pixels high-definition television screen. The technology has potential to enable inspectors to visually examine bridge components easily and with enough precision to see even hairline cracks in the field. It offers these benefits without requiring standard inspection techniques, such as using ladders, climbing equipment, or bridge inspection vehicles (snoopers), which can be costly and potentially hazardous. For video footage of drone operations under a bridge deck with the corresponding view from the drone camera, visit https://youtu.be/_kE2Vi2rdtc.

This technique of using a drone with FPV goggles for bridge inspections is the result of a number of factors that have changed or developed recently. Notably, these include the release of the Federal Aviation Administration (FAA) Part 107 rule regarding civil operation of small unmanned aircraft systems (UAS), the Federal Highway Administration’s support of the use of drones as tools for supplementing visual inspections of bridges, and recommendations issued by the Minnesota Department of Transportation (MnDOT) based on its Unmanned Aerial Vehicle Bridge Inspection Demonstration Project. FHWA’s Technology Transfer (T2) program and recent approval of the FHWA Minnesota Division’s T2 funding for UAS bridge inspections continue the momentum.

FAA Part 107 Rule

On June 21, 2016, the FAA released its final rule, adding the new Part 107 to the FAA Title 14 Code of Federal Regulations (14 CFR) to allow for routine civil operation of small UAS (including drones) in the National Airspace System and to provide safety rules for those operations. The rule simplification comes under the remote pilot certification in Part 107, which allows for the operation of a small UAS by passing the FAA remote pilot knowledge test and then operating under Part 107 regulations. The rule makes it relatively easy for qualified bridge inspectors to become certified to operate drones to supplement bridge inspections without extensive training time and expense.

For more information, visit FAA’s website at www.faa.gov/uas.

FHWA Support Of Drone Use

FHWA encourages the advancement of new technologies by State and local public agencies (LPAs) in their design, construction, inspection, and maintenance to support an efficient and effective transportation system. As drones have emerged as leading technology in the area of industrial construction, FHWA’s Office of Bridges and Structures has allowed the use of drones to supplement bridge inspection procedures and to potentially improve routine bridge inspection processes and documentation.

For more information, see “Ready for Takeoff” in the Winter 2018 issue of Public Roads.

Minnesota DOT Demonstration Project

In July 2015, MnDOT released the findings of its demonstration project using an unmanned aircraft for bridge inspection. The report, Unmanned Aerial Vehicle (UAV) Bridge Inspection Demonstration Project, states, “The use of UAVs to aid bridge inspection should be considered for routine inspections to improve the quality of the inspection by obtaining information and detail that may not be readily obtained without expensive access methods. [UAVs] should also be considered where they can increase safety for inspection personnel and the traveling public.”

Overall, the study team observed that the safety risk to inspection personnel and the public is very low, and drones can provide a cost-effective way to obtain additional detailed information that may not be visible during routine inspections with limited accessibility. For more information, visit www.dot.state.mn.us/research/TS/2015/201540.pdf.

FHWA Technology Transfer Program

The financial springboard for this project was supplied by the FHWA Director of Field Services’ T2 program. The program’s objective is to fund research, development, technology and innovation transfer, and appropriate outreach and communications activities in accordance with the Fixing America’s Surface Transportation (FAST) Act.

Annually, FHWA employees are encouraged to submit proposals for technologically based ideas that will assist the State and local transportation agencies with advancing and incorporating technologies into their work or practices.

Minnesota’s T2 Funding Request

The FHWA Minnesota Division combined staff experience as bridge designers, construction engineers, and bridge inspection program administrators for LPAs to develop a 2017 T2 funding request to explore consumer drones as platforms for routine bridge inspections, targeting LPA deployment.

The division submitted a T2 proposal for “UAS Bridge Inspection” based on its idea to investigate economical, recreational drones for potential use in routine bridge inspection. The goal was to focus on less expensive drones that would more easily fit into the inspection crew budgets of LPAs.

On April 10, 2017, the FHWA Director of Field Services approved the T2 funding for the Minnesota proposal in the amount of $2,250 for the purchase of a drone and peripherals for demonstration and technology sharing with LPAs.

Drone Selection and Testing

First, staff at the Minnesota Division researched and identified a number of drones that met the majority of their needs for demonstration. After analysis of the specifications, the division team chose one for testing. Several drones with similar capabilities are currently on the market, but reviewing multiple drones was not part of this project. As technology continues to advance, there will be even more drones that could be evaluated for use. However, the basis of this article applies to many different drone types and manufacturers.

The selected drone has the following technical specifications:

• 4K video (high-definition display resolution with about 4,000 pixels) with 30 frames per second
• 12 megapixel still photos
• Camera tilts up to 30 degrees (+30 degrees) above horizontal
• Two flight modes without GPS
• Availability at major retailers online and in big-box stores
• Forward collision avoidance and downward sensing

After assembling the drone and components, the Minnesota team tested the drone on two bridges in the State, one in St. Louis County and one in Washington County. The testing was very basic with the goal of verifying that inspectors could operate the recreational drone under a bridge deck without GPS assistance and obtain good-quality video and photos of bridge components. Because the project focused on use for LPAs located outside major metropolitan areas, the Minnesota team tested bridges in rural areas of the State.

During the project, the team identified several major questions related to counties and the use of drones for their routine bridge inspection.

• Can a bridge crew operate and fly a drone under FAA Part 107 rules and perform the basic inspection tasks? The main items of operations to consider for drone use in bridge inspection are to operate in class G airspace at a maximum altitude of 400 feet (122 meters) above ground level, and not to conduct drone operations over traffic or people.

With good preflight preparation, inspections can be performed under requirements of Part 107 by a bridge inspection crew that has a drone operator with a remote pilot certification.

• Can the drone operate effectively without GPS while under the bridge deck? Yes. The drone used has a total of three flight modes, and two can be used without GPS.
• Are the images captured by the camera sufficient documentation of bridge component condition? Yes. With the ability to take 4K video and 12 megapixel still photos, a bridge inspector can capture high-resolution documentation of components.
• Is the drone camera’s tilt range of +30 degrees sufficient to supplement visual inspection of the underside of bridge components? Yes. The camera tilt of +30 degrees is sufficient to get a good visual supplement of the bridge components from below.

Overall, the initial testing showed that the approximately $1,000 drone ($2,300 full system) did meet the basic needs to supplement routine bridge inspection practices.

A Revelation: FPV Goggles

After the initial testing of the consumer drone for bridge inspection, as detailed above, was positive, the Minnesota Division Office noticed that the drone manufacturer had released a pair of high-definition FPV goggles that connected wirelessly to compatible devices, including drones. The Minnesota team agreed that this new technology could drastically change and improve the use of drones in bridge inspections. The team decided to submit a second T2 funding request to purchase the goggles for testing. The FHWA Director of Field Services approved the second funding request. The purchase of the FPV goggles was the next step in the evolution of drone use to supplement routine bridge inspections.

The FPV goggles, costing $450 at the time of purchase, are wirelessly linked to the drone’s camera, and receive high-definition video feed at 1080 pixels and 30 frames per second. The video feed presents on two separate displays with a resolution of 1920 by 1080 pixels and is transmitted at a speed with a latency of 110 milliseconds.

When a bridge inspector uses the connected drone and goggles, the visual impression is described by the manufacturer as being equivalent to an 18-foot (5.5-meter) high-definition, large screen television at a distance of about 9 feet (3 meters). The goggles may be combined with a digital zoom of up to two times, enabling the image equivalent of standing as close as 3 feet (0.9 meters) from the objects being inspected.

One member of the team describes the experience as “like sitting in the sixth row of a movie theater.” For a video demonstration, see https://youtu.be/0eP8NDZqVg8.

Beyond the high-definition and immersive images, the goggles have the capability to give the inspector control over the movement of the camera. The goggles can operate the pitch and yaw of the drone camera: pitch up and down of 90 degrees below horizontal to 30 degrees above horizontal and yaw left and right of 30 degrees. With this seamless operation of the drone’s camera, inspectors can very easily look at the item of interest.

The drone and FPV goggles are interactive. The goggles can be set to control the camera gimbal, which facilitates the goggle movement to change the camera angle. The camera gimbal is the device that enables the camera to rotate freely in any direction or suspends it so that it will remain level when its support is tipped. The inspector can easily adjust the camera view by simply turning his or her head. For example, when an inspector looks to the left, the drone’s camera rotates with this movement. Then, as the inspector looks to the right, the camera gimbal follows. The drone is stationary, but the view changes via the goggles’ gimbal control and the head movement of the inspector. For a video demonstration, see https://youtu.be/Ahf9akBEtxw.

As the Minnesota team started to test the drone and the goggles, they found that the images were very clear and the first-person view was incredible.

Connie Yew, FHWA team leader for construction management, tried on a pair of FPV goggles for the first time and viewed the video footage from an inspection. The video was shot at the equivalent visual distance of 3 feet (1 meter). Her reaction to the high-definition images speaks for itself. She explained how it was very immersive, so much so that she extended her arm out and said, “I feel like I can reach out and touch the beam.”

Tim Anderson, bridge engineer with the FHWA Minnesota Division, agrees with Yew. “The visual image that the inspector gets when using the goggles with the drone during routine bridge inspection will be a game changer,” he says.

The combination of the drone’s high-definition camera and the high-definition displays in the goggles provides live images that are so immersive and clear that inspectors have the ability to identify defects, even hairline cracks. See a video demonstration with visible hairline cracks at https://youtu.be/ESAfBJTkNEs.

Spreading the Word

After demonstrating that FPV goggles, when paired with a drone, can be useful for bridge inspectors to supplement bridge inspections, the Minnesota Division started a presentation and outreach program. The program began with some regional meetings with county engineers and bridge inspection staff. In 2017, division staff gave the presentation and demonstration during 10 meetings, reaching about a third of the 87 counties in Minnesota. This led to an invitation to present the topic at the Minnesota County Engineers Association winter conference in January 2018, where the presentation was given to more than 150 people, representing all of the counties in the State. As a result of these outreach efforts, four counties will be incorporating drones with FPV goggles as part of their 2018 routine bridge inspections.

The FPV goggles that many people see as merely toys can be easily connected to a consumer drone to create a great tool for bridge inspectors. The availability, cost effectiveness (approximately $2,800 or less), and high-definition quality of the drone, combined with the seamless connection of the video feed to the high-definition displays of the goggles, result in a system that can easily be added to the toolbox of any bridge inspector.

“The simplicity and high-definition capabilities of the drone and FPV goggles make the tools incredibly useful to any bridge inspection staff, large or small, to supplement bridge inspection practices,” says Dave Conkel, MnDOT State aid bridge engineer. Bill Lohr, field operations team leader with the FHWA Minnesota Division, agrees and adds, “We absolutely see FPV goggles as the next step in the evolution of the use of drones.”

Joe Campbell, P.E., is the assistant division bridge/area engineer for the FHWA Minnesota Division. He holds a B.S. in construction engineering from North Dakota State University and an M.S. in civil engineering–structures from the University of Wisconsin-Madison.

For more information, contact Joe Campbell at 651–291–6121 or joe.w.campbell@dot.gov.