Over the last 100 years, the U.S. transportation system has been developed primarily for human operators. For example, roadway signs were designed to be easily seen, read, and interpreted by humans. However, as vehicle automation technologies have evolved, driving responsibilities have steadily shifted from humans to automated driving systems (ADSs).

ADS developers and infrastructure owner operators (IOOs) that manage the roadways want safe and efficient operation of automated vehicles, and, to achieve this goal, significant testing must take place. The Federal Highway Administration’s Collaborative Research Framework for Automated Driving System Developers and Infrastructure Owners and Operators (“the Framework”) facilitates the investigation of interfaces that will advance road-ADS integration.

Purpose of the Framework

Without standards or evaluation procedures for ADS testing and without collaboration between ADS development and infrastructure stakeholders, the widespread integration of this technology may be difficult when incorporating needed road improvements.

The successful operation of ADSs on our roadways heavily depends on the ADS being able to interpret and navigate its surrounding environment. However, with the development of ADS technology still in its infancy, ADS developers and IOOs have many challenges to overcome, such as ADSs navigating complex traffic scenarios and adapting in adverse weather as well as building supportive infrastructure. Given that ADS developers and roadway stakeholders seem to have diverse, but complementary, visions on the testing and evaluation needs, the likelihood of safe and efficient integration is necessarily contingent on productive IOO and industry collaboration.

Consider a typical State department of transportation (DOT) operations scenario where a busy highway is being resurfaced or expanded. Presently, the procedures and standards outlined by a State DOT are designed to maximize the safety of a crew within the work zone while still ensuring continual mobility for drivers. For instance, one direction of travel is shifted from the left lane over the center line into a lane typically used for the opposite direction of travel. A work crew must ensure that enough advanced warning of the lane shift exists and that visible barricades are distributed with proper spacing, while drivers must steer through the exchange. A driver deviating from the marked lane shift could lead to worker injury, equipment damage, and/or a traffic collision.

State DOTs can expect that drivers have enough experience with work zone navigation to understand how to proceed safely. However, what might happen if the driver is altogether removed from the situation? Can anyone be certain that the automated vehicles of tomorrow will be able operate safely through a work zone? The answer lies in the quality and scope of testing performed on ADSs, which is expected to displace human drivers over the course of the next two decades. Both the developers of ADS and State DOT employees have a keen interest in how this testing is administered and evaluated.

Using a technology-independent approach (i.e., it does not matter what sensors the ADS uses), the Framework aims to bring these ADS and IOO stakeholders together through a broad set of processes and examples that advise without prescribing regulations or policy. Instead, the Framework adopts the notion that validation and testing of ADS technologies and various infrastructure features are essential to paving the way for safe integration of ADS-equipped vehicles into the road system. Aside from the sheer scope of the possible testing that may be needed, a common understanding of the capabilities that need to be implemented either in the ADS or roadway domains is required by those participating in the collaborative testing.

Collaborative testing promises to provide resolution to these integration issues, resulting in a safe and efficient integration. Numerous stakeholders have recognized the need to investigate various ADS and roadway scenarios, including specific challenges to be addressed. Overall, these efforts will further advance deployment of ADS-equipped vehicles onto roadways throughout the United States, benefiting both ADS and IOO entities.

Framework Overview

The Framework provides a broad suite of tools, perspectives, and approaches intended to facilitate collaboration between the ADS developers and roadway stakeholders. As demonstrated in the work zone navigation example, both ADS developers and State DOTs have a vested interest in the safe integration of ADS technology.

Thus, the Framework was developed with extensive engagement and input from both ADS and roadway stakeholders, including automotive original equipment manufacturers (OEMs), suppliers, technology companies, and State, Federal, and regional government entities. Their combined perspectives can accomplish a collective understanding of safe integration more rapidly and with fewer errors than the respective individual entities.

The Framework addresses nine overarching themes, which support various aspects of the test phases. The Framework includes contextual examples, real-world lessons learned, and various perspectives (e.g., ADS developers, IOOs, first responders, and fleet operators). All stakeholders are interested in—or will be impacted by—ADS deployments on the road network and have a responsibility to engage in the testing and evaluation of these technologies. The Framework enables collaborative support toward their shared goals.

Themes

Collaboration: Collaboration between ADS and IOO stakeholders is critical for successful testing and evaluation. Stakeholder collaboration enables early detection and resolution of ADS issues related to technical, organizational, and strategic test implementations. Collaboration also allows testing participants from diverse organizations, backgrounds, and skill sets to solve specific road and ADS challenges. Open and frequent interaction between stakeholders lead to improved test outcomes. The quality of the testing is also enhanced because input from various points of view can be collected.

"Using the Framework promotes collaboration between ADS developers and IOOs to achieve a common goal and provides each entity the opportunity to share their perspective. This collaboration results in increased success in testing and deployment."

Nick Hegemier, P.E.
Managing Director—Infrastructure
DriveOhio

For example, Pennsylvania DOT (PennDOT) has already recognized the need to prepare for ADS integration across the State. PennDOT assembled eight partners to change infrastructure to support ADS in work zones. They have been testing a variety of strategies, including using different work zone channelizers (cones, barrels, panels), using smart channelizer devices that broadcast their positioning, and generating high-definition maps.

In addition, the project will develop advanced mapping and communications systems for safe ADS navigation at and around 17 different work zones in urban, suburban, and rural areas. Prior to testing in active work zones, the partnership will conduct validation testing in virtual environments and then at a track at the Pennsylvania State University. By engaging automation development teams and creating a comprehensive plan for testing, PennDOT will better understand how ADSs operate in certain conditions and thus will be better equipped to fulfill its mandate for keeping Pennsylvania drivers safe and informed.

Another example of collaboration to achieve roadway safety goals is the Michigan DOT (MDOT) and 3M™ Connected Roads I–75 test corridor. MDOT collaborated with 3M to deploy a 100-day test of 3M Connected Roads prototype solutions in a 3.3-mile (5.3 kilometers) construction work zone. The collaboration involved 3M, MDOT, and a variety of automotive OEMs and sensor suppliers.

Institutional/Organization Issues: Having organizational experts from both ADS and IOO organizations participate early and throughout the test phases will significantly aid in navigating any challenges. For road/ADS testing and evaluation, the Framework makes safety of all road users the highest priority. As a result, State and local regulatory policies must be developed, which requires that policymakers be well informed on relevant topics and kept up to date on ADS integration developments. The Framework aids in navigating challenges that IOO and ADS representatives encounter when conducting tests, evaluations, and pilots.

Common Ground: This theme refers to creating a common or shared working environment so that ADS and IOO stakeholders fully understand each other, which is critical for tests to be successful. When executing road/ADS tests, all stakeholders will have clearly defined expectations, outcomes, and success criteria. There are three key components of common ground: common goals and benefits, common terminology, and common metrics and measures.

Roles and Responsibilities: An important part of the process of IOO and ADS developer collaborative testing and evaluation is identifying who from the various organizations needs to participate, what roles within the organizations are needed, and when (i.e., which phases) they need to participate. Some participants may only be involved in one phase while others may be essential to all phases.

A New Driver: With ADS, the new driver of tomorrow will be the system. The Framework is a document that provides examples and scenarios to aid the transition to a safe and functional road network that can support human- and ADS-driven vehicles sharing the road.

Success Factors: Many factors influence the success of road/ADS testing and evaluation. The most critical success factors include enhancing ADS operation capabilities, comprehension of ADS, roadway test elements, the collaboration and joint testing process, stakeholder engagement and collaboration, and ongoing communication with the public. The Framework aids in assisting the ADS and IOO participants in defining test success factors within each test phase.

Test Logistics: This theme refers to what, how, and where to test, including the development of test scenarios, methodologies, and environments. Test logistics are tailored to specific scenarios and what road features and characteristics and ADS are being evaluated.

“The Framework provides the tools needed to increase interaction among road/ADS entities to enable joint research and development activities.”

Roxane Y. Mukai, P.E., P.T.O.E.
Operations Engineer
Maryland Transportation Authority

Plans: For a collaborative environment to exist, stakeholders’ participation as part of the plan for test design, data collection, and evaluation can be beneficial. For instance, road/ADS testing and evaluation need to account for roadway adaptations (i.e., changing the roads to enhance ADS integration such as different configurations and lane markings), which can be incorporated through collaboration with IOO.

One example is Waymo’s first-responder engagement plan. The objective of this plan is to provide first responders with the knowledge they need to safely identify, approach, and interact with an ADS-equipped vehicle in an emergency scenario.

Sharing Opportunities: Data are a key issue that require thorough discussions with IOO and ADS stakeholders to avoid challenges (e.g., proprietary data/information). Data are critical to evaluate the outcome of tests and ensure effective road network operations. ADS and IOO stakeholders can share a variety of resources, including skills, expertise, information, and existing data.

An example is Arizona’s Institute of Automated Mobility (IAM), a consortium for collaborative data sharing. Arizona’s IAM consortium leverages existing infrastructure to collect performance data on public roads. The research does not require a lot of data collection from collaborating partners. This practice may encourage greater openness and participation from ADS stakeholders.

Framework Application

The activities for successful collaborative testing and evaluation can be categorized into four phases.

Pretest Phase: The output of the pretest phase is a clearly identified problem statement that is based both on the internal needs of ADS developers and IOOs. A clearly defined problem statement supports collaboration among the test participants to then work on a test definition (which is the next step). Program risks, including operational, technical, data, legal, and financial ones, should be identified during this phase. Overall, this collaborative effort is designed to expand the scope of those impacted by the testing effort to minimize longer term issues and delay.

Test Definition Phase: The objective of the test definition phase is to conduct activities that help define the technical and data facets of a collaborative test program. The completion of the test definition phase produces a plan for testing, data management, and quality control, which facilitates subsequent test execution. Critically, the success factors, both technical and organizational, are defined for the program. Having clearly defined success criteria ensure an overall higher test quality.

Test Execution Phase: In this phase, both technical and data facets of the collaborative road and ADS testing proceed as defined in the plan. The focus is on efficiently collecting performance data. The phase includes operational collaboration, ongoing communications among stakeholders, test monitoring, and test adjustments. After completing this phase, ADS performance data are gathered and reviewed to determine if the system is ready to advance to the next phase. For example, if the performance failure at night is considered critical by the stakeholders, then new tests will likely be scheduled after making updates.

Post Test Phase: In the post test phase, stakeholders aim to conclude the collaborative testing and evaluation activities. They review data insights, store data, discuss lessons learned, and evaluate the processes used (which drives future testing and evaluation). Overall, this collaborative effort directly leads to a higher level of confidence by the stakeholders, and by extension the public, that the ADS-equipped vehicles will be safely integrated among existing drivers.

Conclusion

To date, collaboration between ADS developers and IOOs has not been the norm nor has testing ADS-equipped vehicles with varying transportation infrastructure components. ADS technologies have been developed and tested, traditionally, with little or no discussion with roadway stakeholders. Communication between IOO and ADS stakeholders has focused mainly on gaining approval to test on a road network. Proprietary data concerns have been identified as a roadblock to the willingness of ADS developers to work collaboratively with IOOs. Other challenges addressed by the Framework include:

• Developing a consistent and agreed upon language between entities for both ADS and road features and components.
• Identifying and addressing the permitting process and insurance needs.
• Agreeing on common goals and outcomes for a joint test activity.
• Developing a working understanding of the compatibility of safety from both the ADS developer and IOO perspectives.
• Accepting the need to share roles and responsibilities for the test activity.
• Understanding the need to coordinate on any interactions with the public.

However, the Framework addresses collaboration from multiple perspectives throughout a test’s lifespan. It provides examples of successful collaboration, the benefits of collaboration, and information on how and when to collaborate.

Ultimately, this approach focuses the efforts of both the ADS development teams and IOO stakeholders working toward the common goal of safe and efficient ADS integration.

John Harding is a team leader with FHWA’s Office of Transportation Management, Connected/Automated Vehicles and Emerging Technologies overseeing the integration of next-generation technologies into the U.S. roadway system. John holds a B.S. in civil engineering from the University of Pittsburgh and an M.P.A. from George Mason University.

For more information, see https://ops.fhwa.dot.gov/publications/fhwahop21012/fhwahop21012.pdf or contact John Harding (john.harding@dot.gov), 202-366-5665).