Wisconsin DOT contracted with the University of Wisconsin Traffic Operations and Safety (TOPS) Laboratory to investigate multiple project evaluation methods through a research grant for HSIP evaluation support. Initial research efforts included project evaluations based on before and after collision maps using the software Intersection Magic and before-after evaluations using benefit-cost analysis. From the beginning of the research, the intent was to use Empirical Bayes (EB) analysis in the project evaluations, but Wisconsin did not have safety performance functions (SPFs), which are required for the EB method. However, once the State acquired the SafetyAnalyst software, the TOPS Laboratory was able to incorporate the EB method into the project evaluations by using the SPFs contained in SafetyAnalyst. The SPFs in SafetyAnalyst were developed using national data and are intended to be calibrated to local conditions. While it was not possible to calibrate the SPFs to Wisconsin conditions due to lack of data, the TOPS Laboratory uses the SPFs to provide a comparison of performance in Wisconsin to that of the nation.
The TOPS Laboratory developed a process to extract the appropriate crashes (by location, type, and year) from the Wisconsin crash database based on the project location and scheduled start and completion dates for evaluation purposes. HSIP projects are evaluated based on five years of before data and three years of after data. Fatal and injury crashes are the focus of the evaluation, but the analysis also considers target crash types based on the nature of the improvement.
The TOPS Laboratory conducts a benefit-cost analysis based on results of both a simple before-after evaluation and an EB analysis to evaluate the projects from an economic perspective. This provides a simple comparison of the results of the two evaluation methods (In the table shown, “S. No.” refers to the site number for the project evaluated, and the “FOS (financial operating system) ID” is used by Wisconsin DOT as the specific project identifier.) and demonstrates how a simple before-after evaluation can overestimate the safety benefits.
| S. No. | FOS ID | Benefit-Cost Analysis Using Empirical Bayes Estimates | Benefit-Cost Analysis Using Before-After Data |
|---|---|---|---|
| 1 | 10220674 | 1.38 | 2.86 |
| 2 | 11504371 | 31.64 | 44.66 |
| 3 | 12060680 | N/A | N/A |
| 4 | 15300191 | 5.34 | 5.94 |
| 5 | 22001570 | 1.39 | 2.01 |
| 6 | 22401570 | 7.72 | 7.91 |
| 7 | 40500971 | 1.01 | 1.21 |
| 8 | 44790371 | 5.14 | 5.34 |
| 9 | 45401572 | 1.66 | 1.79 |
| 10 | 46851471 | 1.22 | 1.24 |
| 11 | 50600072 | 0.76 | 1.28 |
| 12 | 52520071 | 1.04 | 2.21 |
| 13 | 69960674 | 1.58 | 1.52 |
| 14 | 69991072 | -1.19 | -0.86 |
| 15 | 70300370 | 0.33 | 0.36 |
| 16 | 72200191 | 13.09 | 14.66 |
| 17 | 86100270 | 1.71 | 1.62 |
| 18 | 86810571 | 1.44 | 1.47 |
| 19 | 92000371 | 2.00 | 2.94 |
Key Accomplishments
- Developed a project evaluation process incorporating Empirical Bayes analysis into all HSIP project evaluations.
- Demonstrated the importance of using statistical evaluations to reduce the overestimation of safety benefits due to regression-to-the-mean bias.
Results
Originally, engineers in Wisconsin were reluctant to use EB. However, with the assistance of the TOPS Laboratory, the Wisconsin DOT was able to successfully implement a project evaluation process incorporating EB analysis and to receive buy-in at the regional level. The TOPS Laboratory demonstrated the importance of statistical EB techniques in project evaluations through a comparison benefit-cost analysis using simple before and after results to before and after using EB. The results demonstrate the EB analysis reduces the overestimation of safety benefits due to regression-to-the-mean bias.
Contact
Andrea Bill
Traffic Safety Engineering Research Program Manager
Traffic Operations and Safety (TOPS) Laboratory
University of Wisconsin
608-890-3425
bill@wisc.edu