The laboratory assessment process is designed to enhance laboratory performance and quality by providing feedback to laboratory managers and staff regarding their work. It provides a credible, professional, and objective assessment that further improves stakeholder confidence in the value of the work performed and outcomes produced. It also serves as an opportunity for technical experts, customers, and stakeholders to exchange views on a regular basis with Federal Highway Administration’s (FHWA) Turner-Fairbank Highway Research Center (TFHRC) staff. While assessments focus primarily on the conduct of research, they also examine whether the researchers have achieved stated objectives.
The Laboratory Assessment—An Expert/Peer Review
The laboratory assessment is a form of expert/peer review. It is an independent assessment by technical and scientific experts whose knowledge and expertise enable them to make credible and unbiased judgments regarding the conduct of the reviewed research. Assessing TFHRC laboratories through expert/peer reviews ensures that the research performed at the TFHRC meets established quality and performance standards. An independent evaluation, laboratory assessments provide a means to determine if the research activities have high potential value and if they have achieved stated objectives. The laboratory assessment process is continuous, with each laboratory having the benefit of an expert/peer review every 4 years.
The main goals of the TFHRC laboratory assessment process are to:
- Enhance laboratory and related research program quality, performance, and relevance by providing feedback and suggestions to managers for laboratory improvements;
- Provide an opportunity for technical experts to exchange views;
- Provide increased opportunities for FHWA customers and stakeholders to provide input to research and related program activities;
- Provide a credible, professional, and objective assessment that further improves customer and stakeholder confidence in the conduct of research and the outcomes produced by the research.
The expert/peer review panel will be comprised of approximately three to five experts/peers who are external to the agency. These experts/peers will be qualified to perform an independent, competent review of the technical and scientific merit and quality of the research.
Being nominated as an expert/peer reviewer is a distinguished recognition of the individual’s professional accomplishments in his or her field of endeavor. The participation of an expert/peer in the review process is broadly acknowledged within the transportation and scientific community.
Panel members may be drawn from:
- Scientists from similar laboratories
- Scientists in allied or other disciplines performing similar types of work
- Managerial or technical mentors from the private sector, academia, or other public agencies
- Scientists that have conducted successful/productive laboratory assessments (for example, from the National Institute of Standards and Technology or other such groups)
- Recently retired scientists/researchers who would have qualified in one of the above categories before retirement
Panel members (laboratory managers and/or senior scientists) may come from:
- Federal laboratories
- State departments of transportation (DOT)
- Private sector organizations
- Customers or stakeholders in the industry
Conflict of Interest: Panel members must not be under contract to the FHWA laboratory in any way.
Panel Diversity and Continuity: It is advantageous to have a diversity of backgrounds represented on a review panel, because of the varying perspectives each panel member can bring to panel deliberations. Likewise, it is desirable to have a degree of continuity from one review of a laboratory to the next review of the same laboratory. When possible, one or two panel members from the prior review should serve on the current review panel.
General Characteristics of Panel Members
Review panel members must be able to encourage, constructively criticize, and provide positive feedback on what is being accomplished. Panel members must possess:
- Capability of quickly grasping the operations and understanding the research
- Ability of contributing cogently to the discussion and also synthesizing it
- Good communication skills
- Ability to commit time to the review
- Expertise that contributes to the review
All panel members must be motivated to participate and contribute their expertise to the enhancement of the host laboratory’s business.
The TFHRC laboratory assessment criteria are based on the Office of Management and Budget’s three criteria for Federal investment in research. These three criteria include relevance, quality, and performance. The laboratory assessment process at the TFHRC particularly highlights the performance and quality criteria, as shown below.
- Research activities are managed in a manner that produces high-quality, identifiable results, utilizing research procedures and practices, which comply with or exceed accepted standards for performance and reproducibility.
- Research activities are guided by an established set of high-priority, research objectives with performance outputs and milestones that show how the outcomes will be reached.
- Research activities have well-defined metrics that encourage research project performance, and promote broader goals such as implementation of research results, including dissemination of knowledge, applications, or tools; transition of technology to the private sector, if appropriate; and innovation, cooperation, and education.
- For major research projects, appropriate termination points and other decision points are adequately defined.
- Research maximizes quality using clearly stated, defensible methods for contract awards, and Federal managers and contractors are held accountable for cost, schedule, and performance results.
- Quality assessment of the research is conducted through comparative methods, such as best-practices identification, expert/peer reviews, and benchmarking.
- In addition to FHWA reports, research is reported in publications that are reviewed by peers.
- Methods are in place for maintaining the expertise of research personnel and the capabilities of laboratory facilities.
- Quality guidelines for statistical information are based on structured planning and sound statistical methods.
- Research demonstrates objectivity in presentation and substance, and integrity, (i.e., the protection of information from unauthorized access, corruption, or revision).
- Research is based on direction as expressed in the FHWA multiyear research and technology (R&T) program plans.
- The research purpose is clear and addresses a specific interest, problem, or need.
- The research is designed to make a unique contribution to addressing a specific interest, problem, or need, and is not needlessly redundant of other Federal, State, local, or private efforts.
Laboratory Assessment Process
The expert/peer review is accomplished through the collaborative efforts of the expert/peer review panel, the review facilitator, the review manager, other laboratory scientists and managers, and the laboratory’s customers and stakeholders. The expert/peer review generally takes two and a half days. In the case of a group of laboratory reviews, the reviews may take 3 days. The first day generally includes a meeting with FHWA managers, a laboratory tour, and briefings. Time is set aside for the panel to conduct independent fact finding, including interviewing researchers, stakeholders and customers, and synthesizing material learned. The last day includes the preparation of the panel’s report of findings and the closeout session.
The expert/peer review panel obtains information to perform its review from:
- The briefing book sent to review panel members prior to the review
- Presentations and other materials about the administration and operations of the host laboratory and the conduct of its research
- Interviews with host laboratory personnel, customers, and stakeholders
- First-hand observations of laboratory activities
- Panel discussions
The assessment criteria guide the direction of the expert/peer review. The content of the review deals with the administration and operations of the host laboratory and its recently completed research, research in progress, and near-term future activities