Description:
SPFs and CMFs are integral components of the HSM methods for estimating the expected number of crashes for a given site. While SPFs and CMFs are widely available from sources including the HSM, CMF Clearinghouse, AASHTOWare Safety Analyst™, and state-specific reports, they are produced using data from specific locations and times. As such, SPFs and CMFs may not be nationally applicable in their raw form. Users should calibrate available SPFs and CMFs to be compatible and applicable to local conditions.
The Calibrator is a tool to help users assess SPF compatibility and applicability. For various theoretical and practical reasons, SPFs and CMFs may not be directly transferable across jurisdictions or over time within the same jurisdiction. For example, SPFs and CMFs are intended for application under certain conditions, including factors related to how the crash data are coded. If data for a given time and place do not meet the same standard or if the conditions differ from those used to develop the SPF or CMF, then it is necessary to calibrate the SPFs and CMFs.
The Calibrator is a Microsoft Excel-based tool to automate the calibration process. It provides information to users that will allow them to assess the suitability of, or to compare between, alternative SPFs. The tool is applicable to all SPFs or predictions from the combination of SPFs and CMFs as is consistent with the HSM Part C Predictive Method. The tool calculates and provides a single calibration factor or a calibration function for each SPF or SPF and CMF combination. The tool also calibrates the dispersion parameter as a constant, inverse linear function of segment length, or nonlinear function of segment length. Finally, the tool provides various goodness-of-fit measures and an assessment of how the predictions are performing over the range of all variables in the SPF and CMFs applied.
Note the tool does not address the calibration of standalone CMFs. Refer to NCHRP Project 17-63 for further information on this subject.
Tool Type:
Software
Owner Source:
federal
Sponsor/Owner:
- Federal Highway Administration (FHWA)
Published:
Availability:
This software is free of charge at the following link: The Calibrator Tool Version 4.0 (2-5-20)
Related Resources
The Calibrator User Guide: This application guide provides background information on safety performance functions (SPFs) and the need for calibration. It also serves as a companion "how-to" manual for The Calibrator software tool. This is an updated user guide based on enhancements to The Calibrator, including the ability to calibrate SPFs based on calibration functions and dispersion functions.
The Calibrator: Quick Reference Guide: This application guide serves as a refresher for users of The Calibrator. It provides a brief overview of the capabilities as well as a summary of the steps and data requirements for setting up and using the tool.
The Calibrator Sample Dataset: This sample dataset provides crash data, traffic volume data, roadway characteristics, and crash modification factors (CMFs) for more than 500 roadway segments. The format is consistent for use with The Calibrator, which serves as a template to help users setup similar datasets or to practice using the tool.
The Calibrator Version 3.0 Training (7-24-18): This recorded webinar describes opportunities to enhance SPF calibration, including the development of calibration functions, use of dispersion functions, and upcoming changes to some of the SPFs in the Highway Safety Manual. The webinar also provides an overview of The Calibrator, focusing on the recent enhancements included in Version 3.0.
The Calibrator Version 2.0 Training (3-21-16): This recorded webinar provides background information on safety performance functions (SPFs) and the need for calibration. It also provides a demonstration of The Calibrator Version 2.0 and an overview of the companion User Guide.
Other Relevant Tools
Highway Safety Manual (HSM): This application guide provides a structured framework and quantitative analysis methods for safety management. It is the foundation on which The Calibrator was developed. The HSM is one of the two sources of all pre-defined SPFs provided in The Calibrator.
Safety Performance Function Decision Guide: SPF Calibration vs SPF Development: This information guide provides guidance on whether an agency should calibrate existing SPFs or develop jurisdiction-specific SPFs. The guidebook discusses the factors to be considered in the decision-making process.
Safety Performance Function Development Guide: Developing Jurisdiction-Specific SPFs: This application guide is a "how-to" guidebook for agencies that are developing jurisdiction-specific SPFs. The guidebook discusses the statistical issues associated with the development of jurisdiction-specific SPFs and provides a step-by-step procedure that analysts can use to develop jurisdiction-specific SPFs.
User’s Guide to Develop Highway Safety Manual (HSM) Safety Performance Function (SPF) Calibration Factors: This application guide provides guidelines for calibrating the default SPFs in the HSM. It explains why calibration is needed and provides guidance on how to implement the calibration process (data needs, basis for decisions, and the impacts of the decisions).
Frequently Asked Questions
What are the system requirements?
- The tool was developed for Microsoft Excel versions 2007 and later, running on Windows Operating Systems with the SOLVER d-in installed.
- The file was built with macros, so you may see warnings about macros needing to be enabled or ’trusting‘ the document before using it.
- It may be necessary to adjust the Trust Center settings. The Trust Center can be accessed by navigating to (File, Options, and Trust Center). In the Trust Center options, click on Macro Settings and adjust according to your organization’s policy. Microsoft also releases updates that can occasionally break ActiveX/Macros.
What are the data requirements?
The development of calibration factors/functions requires data for observed crashes, exposure (traffic volume), and any other roadway characteristics included in the SPF. To use The Calibrator, the data are required to be in a specified format prior to importing into the tool. Refer to The Calibrator User Guide for further details on the required data format.
What should I do if an error message appears?
First, ensure the Solver add-in is activated in Excel. If not, activate the Solver add-in. If the Solver add-in is already activated, uninstall the Excel add-ins and then reinstall the add-ins. This can help to "reset" the program.
What should I do if The Calibrator returns a dispersion parameter of 1.0 (it doesn’t appear to be changing from 1 when I run The Calibrator)?
This is an error. Sometimes such errors can be resolved by closing Excel and restarting. If it still does not work, try uninstalling the Excel add-ins and then reinstalling the add-ins.
Should I include sites with 0 crashes in the calibration dataset?
Yes. The calibration dataset should include a random sample of sites that represent the facility type for the SPF of interest. Do not remove sites with 0 crashes or this will result in a truncated dataset and the calibrated estimates would overpredict crashes for an average site. In general, you should not consider the crash history when selecting sites.
Should I include animal crashes in the calibration dataset?
It depends. The calibration procedure is intended to account for differences between jurisdictions, including differences in animal crashes. For example, the data used to develop the SPF may include a relatively large or small proportion of animal crashes. If you do not wish to consider animal crashes in your SPF predictions, then you can remove animal crashes from the calibration data. If you wish to include animal crashes in your SPF predictions, then you should include the animal crashes in the calibration data.
Should I remove sites from the calibration dataset that don’t fall within the range of AADT values for which the SPF was developed?
It depends. This is a judgement call and you can use The Calibrator to develop CURE plots for AADT to see if the bias is large within the questionable range of AADT. If it appears to be a concern, then you could divide the data and calibrate to values within the range, values below the range, and values above the range, using The Calibrator to evaluate the performance of each calibrated SPF.
Should I always assume the larger AADT is the major AADT? (In some cases, the minor road has a higher AADT than the major road.)
Yes. To be consistent with the SPFs in the Highway Safety Manual, you should assume the major road is always the one with the higher AADT.
Should I apply the state-specific crash type or severity distribution to the calibrated SPF for total crashes to predict the number of crashes by type or severity?
It depends. If you do not have calibrated SPFs by crash type or severity, then you can apply the local crash type or severity distribution to the calibrated SPF for total crashes to predict crashes by type or severity. In fact, you can use The Calibrator directly for this by calibrating the total crash SPF to specific crash type or severity counts.
The caveat is that you should have sufficient confidence in your local crash type or severity distribution. The minimum suggested sample size to develop a local crash type or severity distribution is 100 crashes. If you are developing a joint distribution by crash type and severity, then the minimum suggested sample size is 200 crashes.
Is any single goodness-of-fit measure more important than another?
No. There is not a dominant measure and there is some subjectivity to the use of goodness-of-fit measures to assess the suitability of an SPF. There is also a lack of research in this area to make definitive statements. The goodness-of-fit measures are more appropriately used to assess the relative quality of one SPF compared to another.
Is the dispersion parameter (k) calibrated during the calibration process?
The Calibrator estimates (i.e., calibrates) the dispersion parameter for the calibrated SPF. Calibrating the dispersion parameter occurs through a maximum likelihood procedure, which estimates the dispersion parameter to maximize the negative binomial likelihood function. The tool can calibrate either a constant dispersion parameter or one that varies by length (for segments).
If I develop a new SPF, how do I calibrate the dispersion parameter?
The statistical package used to develop the new SPF should report the dispersion parameter. If you are using the same data in The Calibrator that was used to develop the new SPF, then the dispersion parameter should be the same.
For SPFs that include CMFs or similar adjustment factors (e.g., the Highway Safety Manual Part C Predictive Method), should I apply all CMFs and adjustment factors when calibrating the SPF?
For calibration, you should apply all CMFs and adjustment factors that you will include when you apply the calibrated SPF. For those CMFs and adjustment factors you choose to not apply for calibration, you can use The Calibrator to assess the potential for biased predictions over the range of those variables (i.e., using CURE plots and/or assessment tables). Table A-2 in Part C, Appendix A lists the required and desirable data elements for calibration. It also lists potentially appropriate assumptions for using default values in place of the desirable data elements.
Is there technical assistance available?
Yes. If you’re tackling a particularly tough job, or just want additional help with The Calibrator (or other safety management activities), you can request technical assistance or training through the FHWA Roadway Safety Data Program (RSDP).
Where can I suggest enhancements to The Calibrator or report errors?
If you have suggestions for enhancements to The Calibrator, you can use this link to provide feedback. If you would like to report an error or request assistance, please email Stuart Thompson at stuart.thompson@dot.gov