Problem/Issue

The Kentucky Transportation Cabinet (KYTC) is responsible for performing Highway Safety Improvement Program (HSIP) evaluation, including project tracking and crash modification factor (CMF) development. While KYTC staff administer and provide high-level guidance on the HSIP, they have limited in-house capacity and capability to develop CMFs and delve deeper into specific statistical issues. One statistical issue is related to sample size, where there are small sample sizes of a particular treatment due to high construction costs or unique situations in which it is implemented. KYTC has been challenged with finding statistical analysis strategies to best evaluate the benefits of countermeasures with small samples.

Solution

To address these issues, KYTC turned to their long-standing partnership with the Kentucky Transportation Center (KTC), housed within the University of Kentucky. The Division of Traffic Operations in KYTC’s Central Office administers Kentucky’s HSIP funds. Using these funds, KYTC sets up a contract with KTC to provide technical assistance such as data analysis, evaluation of project effectiveness, evaluation of project completion, HSIP Annual Report assistance, and other technical activities that support HSIP administration in Kentucky. The agreement typically runs for two-year periods, functioning much like an on-call agreement. The relationship between the two organizations works efficiently and has allowed Kentucky to perform HSIP evaluations and experiment with unconventional evaluation methods to address concerns with small sample sizes.

Benefits

The strong relationship between the agencies demonstrated its value when KYTC initiated a study to evaluate the safety effects of three focused safety programs: rumble strips, high friction surface treatments, and cable median barriers. Historically, KYTC and KTC have used the Empirical Bayes (EB) before-after approach for HSIP evaluations to estimate the expected change in crash frequency. In these evaluations, KTC identified small sample sizes as a potential challenge to using the EB approach. In addition to the EB method, KTC suggested the Wilcoxon signed-rank test of proportions¹ to compare the safety performance before and after implementation.

Countermeasures such as rumble strips, high friction surface treatments, and cable median barriers target specific crash types such as run-off-road and head-on crashes. While the EB method would indicate the expected change in crash frequency for a given crash type, the Wilcoxon signed-rank test would indicate if the proportion of target crash types changed after implementation. Specifically, the Wilcoxon signed-rank test of proportions assesses the statistical significance of before-after shifts in target crashes proportionately. In this process, using the proportion of target to total crashes, the target crash types are normalized by total crashes at each site for before and after periods.

KTC used both the EB before-after method and the Wilcoxon signed-rank test to evaluate the safety effects of the three programs. The EB before-after method served as a comparison to the results from the Wilcoxon signed-rank test. A comparison of the results demonstrated a general consistency in results between the Wilcoxon signed-rank test and the EB method, recognizing the Wilcoxon signed-rank test focuses on crash proportions and the EB method focuses on mean crash frequencies. While statistical methods are not a replacement for a large sample of high-quality data, these results suggested the Wilcoxon test can be used as further verification when evaluating countermeasures. The literature suggests the Wilcoxon signed-rank test can support statistical analysis with relatively small sample sizes, particularly when there is a large shift in proportions from the before to the after period and the shift is relatively consistent across sites.² The required sample size increases as the shift in proportions decreases and as the variance of the shift in proportions increases among sites.

KYTC and KTC will continue to use the Wilcoxon signed-rank test as a complementary method to the EB analysis. The two agencies will also continue to work together to explore new methods for project evaluation and build from past experience. For example, as part of the cable median barrier evaluation, KYTC identified an issue with how police officers are coding median crossovers on crash reports. In response, KTC performed a manual review of hundreds of police reports and created an algorithm to determine whether a median crossover actually occurred in a crash. There are follow-up efforts to educate officers on the appropriate criteria for reporting median crossovers.

Contact

Tracy Lovell
Division of Traffic Operations, Kentucky Transportation Cabinet
(502) 782-5534
Tracy.Lovell@ky.gov