Sign Simulator Validated In FHWA Study
"Signsim" is the sign simulator used by the Federal Highway Administration (FHWA) to evaluate a group of traffic signs that were proposed as national standards. Based on the simulator results, FHWA made different recommendations for each of the signs to the National Committee on Uniform Traffic Control Devices in January 1996.
The committee, however, questioned the validity of Signsim and requested that the system be validated in some manner. The validation study was performed at the Turner-Fairbank Highway Research Center (TFHRC) by recording the detection distances of actual scaled signs in TFHRC's Photometric and Visibility Laboratory (PVL) and then by comparing the PVL detection distances and the Signsim detection distances to determine if they were statistically significantly different.
Scope of the Study The main goal of the Signsim study was to provide recommendations on 13 specific sign types for the next revision of the Manual on Uniform Traffic Control Devices (MUTCD). From the original 13 sign types, six were recommended for inclusion in the MUTCD revision.
In the PVL study, four of the six recommended sign types were tested for recognition distances. (See table 1.) Two recommended sign types were not used in the PVL study because they were very similar in design to other tested types.
Fabrication of Signs AutoCad, a computer-aided design and drafting tool, was used to develop the symbol signs. After the signs were drawn using AutoCad, a sign-cutting device was used to manufacture the signing materials, which were then mounted on aluminum panels. The aluminum panels were obtained from the TFHRC machine shop, and the signs were assembled by placing the sign sheeting over the aluminum panels. The size of the PVL signs was calculated using a ratio equal to that used in the Signsim study.
Lighting of Signs Lighting in the PVL was staged to simulate nighttime in an effort to mirror the lighting conditions in the Signsim study. In the previous Signsim study, the lights were off, and participants viewed the signs in total darkness. It should be noted, however, that in the Signsim study, the effect of light on the signs presented was different than the effect of light on signs in a natural environment. Normally, when a car approaches a sign at night, the light from the car's headlights causes the sign to look more illuminated, or brighter, as the driver approaches the sign. However, in Signsim, a projector behind a screen provided the light, and as the sign grew bigger in front of the participant (simulating the approach of a car to a sign), the light was defused. That made the sign looked less bright as it expanded.
In the PVL study, nighttime lighting conditions were used to match the dark lab conditions in the Signsim study. In the PVL, illumination of the sign was not altered as the participant approached the sign. This technique was used to avoid directly opposing the lighting effects that occurred either in a natural environment or in Signsim.
Experimental Design The experimental design was two (age group) by n (number of versions of signs, ranging from two to three) by two (laboratory - Signsim or PVL). Age group and lab were between-participants variables, and number of sign versions was a within-participants variable. Sign recognition distance was the dependent variable.
Recognition distance is the greatest distance between the participant and the sign, based on the visual angle of the sign, at which the subject can correctly identify all the features of that sign. For this study, subjects did not need to interpret the meaning of the sign. For example, for the golf cart-crossing symbol, if the subject said a person in a vehicle, then they would have correctly identified the elements of the sign.
Procedure Prospective participants in this study took a far visual acuity vision test, and an acuity of 20/40 or better was required to participate. Participants were paid $30 for their services. The people disqualified as a result of the vision test received $5.
In the PVL, participants were presented with three practice signs, six distractor signs, and 11 test signs - a total of 20 signs. (See table 1.) The three trial signs were presented first to enable the subject's eyes to adjust to the darkness. Trial, distractor, and test sign presentation were all partially randomized. Distractor signs were used to decrease the apparent frequency of the test signs, which have up to three different versions, and thereby discourage the subject from attempting to identify the critical test signs before clearly recognizing the sign's features.
Participants started 100 feet (about 30 meters) from the sign presented (which was attached to the far wall of the PVL) and walked at their own pace toward the sign. Participants were instructed to stop walking when they could positively identify all the features of the sign verbally to the experimenter. The subject was then instructed to look at the tape measure on the floor, using a flashlight, and read aloud the recognition distance. The subject then returned to the starting point while the experimenter mounted the next sign. After the recognition distances for the 20 signs were recorded by the experimenter, the subject was debriefed.
Results and Discussion The results revealed significant differences between the recognition distances obtained using Signsim and in the PVL for all signs except for the third Dead End sign and the "bike with text" signs. The recognition distances recorded in Signsim are consistently shorter than those recorded in the PVL. (See table 2.)
This phenomenon was pointed out in the previous Signsim study. Philips, Fox, and Peters (1998) stated that recognition distances measured using Signsim were somewhat less (closer) than the values found in other simulators and field studies. They noted that the Signsim recognition distances ranged from about 25 meters (82 feet) to about 100 meters (328 feet); other simulator studies have found distances ranging from 100 to 300 meters (328 to 984 feet). They further state:
"It is possible that these differences were caused by differences in the simulators. The images used in this (Signsim) study were slightly blurry, potentially reducing recognition distances. In addition, the luminance of the light emitted from the slide projector did not change; thus, as the slide image increased from about 1.6 cm (0.625 in) to almost 25.4 cm (10 in), the luminance of the image on the screen decreased. However, during daytime driving on the road, the luminance of a sign would not change as one got closer to that sign." (page 14)
During nighttime driving on the road, the luminance of a sign would increase as the car approached and its headlights shined on the sign.
Therefore, it is reasonable to assume that due to the image quality and the degraded luminance of the signs, recognition distances found in the Signsim are more stringent, although, as shown by table 2 data, they show similar trends as the PVL data. Upon further analyses and comparison of individual sign recognition distance results from each of the labs, we find that out of 12 possible effects, Signsim and the PVL report nine identical effects. (See table 3.)
The three dissimilar effect findings are:
- Signsim found a sign effect of "bike in the road"; PVL did not.
- Signsim found an interaction effect for "bike in the road"; PVL did not.
- Signsim found an age effect for "golf cart" sign; PVL did not.
These findings could be due to the degraded luminance and more blurred images prevalent in Signsim. Perhaps Signsim produces some extra effects because the lighting and fuzziness of the signs make them harder to see. Therefore, weaknesses of the sign designs may be more salient in Signsim than in the PVL. Notice that the two signs that produced different results in Signsim and in the PVL were the "golf cart" and the "bike" signs. In both of these signs, the features may be difficult to identify because the golf cart has a person in the cart and one of the bike signs has text to read in addition to a figure. The other two signs used were "curve right" and "dead end," both of which had fewer and less detailed features to identify.
In addition, a difference between Signsim and the PVL is that participants in the PVL have control over their approach to the sign; whereas in Signsim, using the slide projector, presentation of signs remains constant for all slides and for all participants. Therefore, participants did not have to rely on their dynamic visual acuity in the PVL since they were able to pace their walk towards the sign. (Dynamic vision is a particular function of the cones and rods of the eye that is required when a visual target is in continuous motion.) However, a constant simulated speed of sign approach towards the participant in the Signsim may have required the involvement of dynamic visual acuity for sign recognition.
Though there are some differences between the PVL and Signsim, it may be concluded that the majority of the results compared in this study were similar, if not exactly the same, and that Signsim at TFHRC is a reliable laboratory for producing realistic results. Signsim also poses less risk to subjects because they are not on the road and there is little chance of simulator sickness. In addition, the presentation of signs in Signsim, in terms of speed and lighting, is controllable and consistent. In light of the results found in this study, Signsim certainly gives reliable results with which to drive future studies.
Reference B.H. Philips, J.E. Fox, and R.D. Peters. " Computer-Aided Optimization and Evaluation of Selected Signs," Proceedings of the Transportation Research Board's 77th Annual Meeting, Washington, D.C., 1998.
Dr. Karen Mahach is a human factors engineer at George Mason University in Fairfax, Va. However, she works as an on-site contractor at FHWA's Turner-Fairbank Highway Research Center. She has a doctorate in human factors engineering and seven years of experience in surface transportation. Her experience also includes human-computer interaction and air transportation research for the Federal Aviation Administration.
Dr. Kathryn Wochinger is a research psychologist with Science Applications International Corporation (SAIC). She holds a doctorate in experimental psychology from George Mason University. She works on site at FHWA's Turner-Fairbank Highway Research Center as the SAIC support manager for FHWA's Human Factors Resident Research Program.
Rafael Marshall is a doctoral candidate at George Mason University. He works as an on-site contractor at the Turner-Fairbank Highway Research Center in support of the human factors research program.
Deanne Eppich worked as an on-site contractor at the Turner-Fairbank Highway Research Center while she completed the requirements for a master's degree in human factors engineering from George Mason University. She also has a bachelor's degree in computer science and previously worked as a software engineer for IBM and Lockheed Martin.