Learning From The 2003 Blackout
Massive power outages offer multiple lessons on how to position the transportation system for optimal performance during disasters.
The largest power outage in U.S. history rolled across much of the Northeast from Detroit to New York City on a hot and humid Thursday in August 2003. The massive power outage left a swath 6,000 kilometers (3,700 miles) long–including portions of Michigan, Ohio, Pennsylvania, New Jersey, New York, Connecticut, Vermont, and Canada–in the dark. In New York City, workers poured out of the highrises only to find the streets gridlocked, because traffic signals at all of the city's 11,600 signalized intersections had ceased to operate. The New York subway system ground to a halt, stranding more than 400,000 passengers in tunnels. The city's extensive commuter rail network also closed down, leaving few options for routing stranded customers back to their homes in New Jersey and Connecticut since approximately three-fourths of work trips into Manhattan are made using transit.
Traffic signals and public transit are only part of the transportation facilities that depend on electricity. Other systems include tunnel lights and ventilation; intelligent transportation systems (ITS) equipment such as cameras, loop detectors, variable message signs, and electronic toll collection equipment; and pumps to control flooding in depressed roadways.
In Detroit, MI, the August 14, 2003, power outage hit just at the beginning of rush hour, leading to heavy congestion on miles of freeways. By the next day, a heavy rain had flooded several sections of depressed freeway because the sump pumps used to remove water from these sections had no power, and backup generators were unavailable. Cameras and variable message signs were not operational as well, making it difficult for managers to gather and communicate information to the public--except through the lenses of cameras on news helicopters.
Restoring transportation operations is vital to safety, freight movement, and national security. "Historically, transportation has been viewed as an important support function during disasters," says the Federal Highway Administration's (FHWA) Emergency Transportation Operations Team Leader Vince Pearce. "But the more we look at large-scale situations, the more we see that if transportation doesn't work right, it's too hard for other responders to do their jobs. If we can't get the fire trucks and ambulances to the scene, we can't put the fires out or help the injured. Transportation must work at its absolute best in these kinds of situations, and our objective is to help the transportation community bring their resources to bear at the most important time."
How Did the Blackout Happen?
Despite the heat and humidity on August 14, the network of electric transmission lines in the East was carrying a standard load for that time of year. Shortly after 2:00 p.m., a brush fire caused a transmission line south of Columbus, OH, to go out of service. This outage was followed at 3:05 p.m. by the failure of a transmission line connecting eastern and northern Ohio, and then a second line failed in the same area. As more and more sections of the electrical network disconnected from the grid, the blackouts accelerated.
At 4:10 p.m., the system connecting the region south of the Great Lakes, including the cities of Detroit and Cleveland, across the country to New York and New Jersey experienced a profound failure. Within the space of a single minute, many transmission lines failed throughout the entire area, creating a cascading effect in which lines sequentially overloaded and then failed.
Airports halted operations, and elevators stalled midride. Water systems shut down. The communications network was disrupted, since many cellular and other telephone systems ceased to work. Stranded commuters spent the night in train stations, hotel lobbies, and emergency shelters. Others spent many hours trying to get home by foot or ferry boat.
The first question on many peoples' minds, including speculations transmitted by the press: Was this emergency related to national security? The guiding priority in every emergency is the protection of life, but many transportation staff in the affected areas initially feared a terrorist attack. Reaction and response to a terrorist attack would be different from a nonsecurity-related emergency, making it vital to communicate the causes of failure as quickly as possible. Once the causes of the blackout became clear, agency managers shifted their focus from security to safety and then to mobility.
Emergency Transportation Operations ProgramThe August 14 blackout--massive as it was--was only one of 56 federally declared disasters in 2003. To realize the enormity of this statistic, consider that the Nation averaged more than one catastrophic event per week that year: hurricanes, floods, landslides, wildfires, tornados, avalanches, ice storms, and power outages. The impact on the U.S. transportation system is incalculable. To address the effect of major emergencies on transportation, FHWA's Office of Transportation Operations initiated the Emergency Transportation Operations Program in 2001 to examine how the transportation system is used during those situations. "The program's underlying concept has two legs," says Vince Pearce, the team leader. "First is that every incident affects traffic even if it doesn't occur on the transportation network. Second, the transportation network is the route through which responders get to the scene and how they get the ill and injured out to proper care." Under the Emergency Transportation Operations Program, FHWA initiated a series of case studies on the effects of major emergencies on surface transportation, the actions taken to respond, the rationale for those actions, and the ways that technology can be applied to improve those responses. To date, the reports on six case studies are completed or are nearing completion, four produced by the USDOT's John A. Volpe National Transportation Systems Center in Cambridge, MA, and two by Scientific Applications International Corporation in Washington, DC:
Volpe Center staff also completed a cross-cutting study covering the first four case studies and a comparative analysis of all six studies. FHWA used the findings from the case studies during workshops and exercises to spread the lessons learned and a wake-up call about the need to prepare for the unexpected. The FHWA program is bringing together relevant policies, protocols, procedures, and practices at the national level by working with the U.S. departments of Justice, Homeland Security, and Health and Human Services. And it is helping the State and local transportation communities to do the same in their jurisdictions. The program is a multimodal undertaking since evacuations involve not only highway vehicles–cars, trucks, and buses--but also commuter trains, ferry boats, and even movement of pedestrians. |
The Big Apple at a Standstill
With only a few exceptions, the ITS equipment operated by New York transportation agencies went down. The George Washington Bridge, which receives power from multiple locations, remained operational. The bridges and tunnels run by the Metropolitan Transportation Authority (MTA), which had backup generators, also continued to function.
Because of lessons learned from past emergencies, the agencies responsible for the city's transportation system had response plans in place. Previous major blackouts, preparations for the year 2000 (Y2K), and the events of September 11, 2001, had prepared the region to deal with significant disruptions to its transportation network. But the plans did not anticipate the scope and duration of the August 2003 blackout.
With police, fire, and emergency response personnel focused on freeing people trapped in stiflingly hot elevators and dealing with other life-threatening situations, traffic management took a secondary priority. In many cases, citizens stepped in to direct traffic at major intersections when police were unable to reach their assigned stations. Under emergency operating procedures developed after September 11, many of the tunnels and bridges into Manhattan were immediately closed or access to them was restricted. Each of the region's 13 traffic management centers is linked through an interagency remote video network of more than 400 cameras. Although the network maintained connections with two-thirds of the centers, the system was compromised because most of the cameras in the field failed due to the lack of backup electricity.
Within 5 minutes of the blackout, Con Edison, Inc., personnel had notified NYC Transit managers that the power outage was extensive and potentially long in duration. Evacuation of subway passengers began in the next 10 minutes.
As on September 11, water ferries were overwhelmed with passengers trying to leave Manhattan for New Jersey. Buses quickly became caught in the traffic gridlock.
Most businesses were closed the next day, keeping the traffic volumes light. Power was restored to all of New York City at 9:30 p.m. Friday night, approximately 29 hours after the blackout began. The NYC Transit subway system resumed full service by 6:00 a.m. on Saturday morning. It was not until Sunday night that the New York City Department of Transportation (DOT) completed the task of inspecting all of the city's traffic signals.
Blackout Disrupts Detroit Traffic
The Michigan Intelligent Transportation Systems (MITS) Center, managed by the Michigan DOT, oversees 290 kilometers (180 miles) of freeways in the Detroit area. The Detroit DOT operates approximately 430 buses during peak hours and the suburban transportation agency an additional 300 fixed-route buses and about 100 demand-responsive vehicles. The suburban vehicles are equipped with automatic vehicle locator equipment linked to computer-automated dispatching. The Detroit People Mover, an elevated monorail, operates in the downtown.
The blackout hit Detroit at rush hour, so traffic that normally would have been staggered throughout the evening was concentrated in the period immediately after the beginning of the blackout. By 4:15 p.m., the Detroit-Windsor Tunnel across the U.S.-Canadian border was closed down. The tunnel is served by four separate and independent power feeds. "It was the first time in anybody's memory that all four feeds failed," says Neal Belitsky, executive vice president and general manager of the Detroit & Canada Tunnel Corporation. Belitsky adds that the tunnel operators followed a preplanned protocol and evacuated the tunnel in less than 15 minutes.
Electrical service was restored in some parts of the region as early as Thursday evening and in most areas by Saturday night. The power was not immediately stable when it returned. Rolling outages across the region meant that some equipment had to be reset more than once.
Opportunities for the Electricity SectorThe entire U.S. electricity network is so interconnected that points on the grid are separated by milliseconds. Subsequent to the massive August 14 power outage, the U.S. General Accounting Office (GAO) released a report in November 2003, "Electricity Restructuring: 2003 Blackout Identifies Crisis and Opportunity for the Electricity Sector." Among GAO's recommendations for congressional action are the following:
|
Impacts on Other Cities and States
Transportation impacts on other States like Vermont and Pennsylvania were minimal. In New Jersey, the blackout affected power in five counties in the New York City metropolitan area, but the transportation system was relatively unaffected except for movement of commuters out of New York into New Jersey. In Pennsylvania, emergency operations centers were partially mobilized. But by 1:10 a.m. on Friday, August 15, almost all power in Pennsylvania was restored.
In Cleveland, public officials were unable to use the emergency response center immediately after the start of the blackout due to a lack of backup power. A portable generator arrived about 8 hours later. Following the loss of power, most traffic signals in the city went dark, which produced gridlock at many intersections. Off-duty and auxiliary police officers assisted in directing traffic, and emergency generators were later used to power some of the traffic signals.
Most of the congestion had cleared from the downtown area by 6:30 p.m. on Thursday. In fact, Howard Huebner, district roadway service manager with Ohio DOT's District 12, says, "I've never seen Cleveland empty so quickly." By early evening on Saturday, full service was restored to the Cleveland area.
Although all emergencies share certain similar characteristics, each is unique and provides the transportation community with new insights into how to prepare, how to set priorities, and how to respond. The 2003 blackout has added to the growing base of knowledge. The findings of the New York area and Great Lakes region case studies offer lessons learned in six areas: planning and preparation, operating decisions, agency coordination, the role of advanced technology, technical communications, and system redundancy.
|
Agency |
Key Decisions |
---|---|
Information FOR Motorists (INFORM) |
Implemented emergency management procedures and expanded hours of operation for highway emergency local patrols. |
Joint Transportation Operations Center (NYPD, NYC DOT, NYS DOT) |
NYPD reassigned approximately 2,000 traffic agents to begin directing traffic. |
Metropolitan Transportation Authority (NY MTA) Bridges and Tunnels |
Switched to backup generator power and continued to operate. Reversed lanes in crossings to accommodate buses returning to Manhattan, and suspended outbound tolls. |
New Jersey DOT Traffic Operations-North |
Initially lost power but within an hour was displaying highway advisories on its variable message signs. |
NJ TRANSIT |
Implemented its preestablished plan for communications outages, established a bus "bridge" to replace the Hoboken light rail system, and started a "load and go" operation from the Port Authority Bus Terminal in Manhattan to the Meadowlands Stadium. |
New York City Transit-Bus |
Started a shuttle service from Penn Station to the Long Island Rail Road station at Jamaica, sent buses to major subway stations, and suspended fares. |
New York City Transit-Paratransit |
Continued operations and prioritized patrons needing life-sustaining services. |
New York City Transit-Subway |
Began the evacuation of 400,000 subway passengers at 4:20 p.m. |
Port Authority of New York and New Jersey |
Activated the Emergency Operations Center, closed or restricted access to facilities that had lost power. |
Port Authority Trans-Hudson Corporation |
Identified the location of its 19 en-route trains and then shut down the system and evacuated at 4:30 p.m. |
Transportation Operations Coordinating Committee (TRANSCOMSM) |
Issued facility status bulletins by fax, e-mail, and phone, including a toll-free number |
Planning and Preparation
The first lessons learned from the August 14 blackout relate to planning. To respond effectively to a catastrophic event, transportation agencies need to have a plan of action in place to handle the emergency and the process of restoration once the crisis is over. Preparation includes drafting an emergency response plan and revising it as necessary, stockpiling emergency items, and rehearsing crisis scenarios.
One of the key themes related to emergency response plans is the need to learn from previous events and to incorporate that learning into an agency's response plan. The experiences of Y2K and September 11 encouraged many transportation managers to draft their first-ever emergency response plans. For others, the earlier events led to identifying or rectifying any weaknesses in the emergency plans they already had, efforts that proved invaluable during the 2003 power outage. But some agencies' plans still lacked enough depth to address an emergency as prolonged as the August 14 blackout.
Internal planning within an agency is essential. Emergencies happen without warning and do not always go as planned, so being able to rely on agency staff at all levels to make timely decisions is critical, even if they lack complete knowledge of all the circumstances. Response plans need to make it possible for staff members to know their responsibilities in an emergency and to step into their assigned roles quickly and easily, with a minimum of confusion and wasted time. Pre-determined roles and clear chains of command are essential, as are alternate methods of communication.
Several agencies realized the need to develop plans in the event that communication is lost (NonComm plans). During roll call, employing a NonComm plan, New York City police officers in the traffic division are assigned locations, and they must proceed to those assignments during an emergency without being notified. At the Port Authority of New York and New Jersey, executive managers are notified by e-mail, blast fax, and phone in the event of an emergency. During the 2003 blackout, several managers reported to the emergency operations center without being called because they knew the agency's procedures.
Because transportation agencies are interdependent, planning is most effective if done in concert with all the agencies that make up the transportation network, as well as nontransportation agencies. For example, after the blackout, the Detroit-Windsor Truck Ferry staff began working more closely with U.S. Customs and Border Protection officers to plan more effectively for emergencies that affect the crossing of commercial freight between Canada and the United States.
Emergency response plans need to consider the movement of pedestrians as well as vehicles. People walking through highway tunnels and across bridges can prevent emergency vehicles from entering those facilities. To help solve this problem, planning needs to include the identification and publicizing of transportation hubs where pedestrians can assemble for taking buses out of the city. During the blackout, transit staff transported pedestrians across New York bridges and tunnels by bus. In future emergencies, officials foresee using a public address system to tell pedestrians where to catch transportation out of the city.
Stockpiling appropriate supplies and equipment is the second component of preparation. When stockpiling supplies, officials advise not to neglect the small items. Agency representatives mentioned the need to stock water and high-energy food bars for staff members who have to operate emergency operations centers over an extended period of time. When one Detroit official, Ron Ristau, director of service development at the Suburban Mobility Authority for Regional Transportation (SMART), tried to purchase bottled water for his staff during the blackout, he had to settle for buying "everything in sight that had liquid in it." Other officials highlighted the need for extra flashlights and charged batteries for cell phones, pagers, portable radios, and portable computers.
Key Decisions by Detroit Transportation AgenciesOn and after August 14, eight transportation agencies in the Detroit metropolitan area collaborated with other government agencies to make the following strategic decisions.
|
Prior to August 14, the MTA Bridges and Tunnels agency in New York had equipped its systems with full backup generating capacity. Other agencies, however, had not prepared as well. Subsequent to August 14, many agency representatives therefore began purchasing new sources of backup power.
During the blackout, representatives of some agencies were surprised by what was not covered by their backup systems. Two agencies lost power for the card key systems that govern access to their offices. Some agencies had backup power for their computers but none for the air-conditioning units needed to cool that equipment. Some staff mentioned that their generators were centrally, rather than strategically, located. They had trouble deploying the generators because traffic congestion slowed the workers.
In recent years, NYC DOT has upgraded many traffic signals from incandescent lamps to light-emitting diode (LED) displays, which require less power to operate. As a result of the blackout, the agency also is considering adding battery backup to signals at critical intersections.
The transportation officials who were interviewed for the case studies advise periodically reevaluating the need for backup generators, identifying additional means of obtaining backup support, and testing the backup power under a full load. "We are continually running tabletop emergency exercises with communities throughout the State," says Chief Joseph Bober, head of police at New Jersey Transit Corporation (NJ TRANSIT). "It's important for them to know what equipment and services we have to bring to the table, and at the same time, it is important for us as a statewide transportation agency to know what those communities have to bring to the table."
Planning Actions Taken
|
Finally, to maintain a state of readiness, agencies need to conduct frequent drills–the third component of preparation. "We train on a regular basis," says the Detroit-Windsor Tunnel's Belitsky, "both live exercises and tabletops. Each April we close the tunnel on a Sunday morning, and every year our training scenario becomes a little more complex." A tabletop exercise is a training exercise that simulates actions, reactions, policies, and procedures for a specific type of incident, and Belitsky recently participated in one that involved about nine different agencies.
Chief Bober adds, "I can't emphasize the importance of training enough. For example, the drivers who operate NJ TRANSIT's 2,200 buses are trained under the Transit on Patrol (TOP) program to call in crashes, incidents of road rage, or anything suspicious to their dispatcher. The drivers act as another set of eyes and ears."
NJ TRANSIT also provides Community Emergency Response Team training so that employees can assist during emergency incidents. Although they are not to be confused with first responders, the NJ TRANSIT employees can assist when called upon. In addition, NJ TRANSIT trains volunteer employees to serve as "Hub Teams" that are deployed to major transportation hubs during emergencies, where they provide live communications links between NJ TRANSIT and its customers.
Based on lessons learned during the blackout, staff at one agency now will be training in-house employees for jobs normally performed by a contractor. When a generator failed during the blackout, contractor staff members were not available to fix the problem. In-house staff needs to be able to repair and service equipment during an emergency.
Effective planning should include not only planning for the immediate crisis, but also for the recovery. And after the recovery, response and emergency plans should be reviewed for lessons learned. Staff members at Information FOR Motorists (INFORM) noted that, during the recovery, resetting traffic signal controllers was difficult with a limited staff. After the emergency, the agency's staff members assessed their actions, and they now group intersections according to snowplowing routes. The agency will be making laminated sheets of the routes to guide technicians as they reset the signal controllers.
Chief Bober notes that, after the recovery, NJ TRANSIT Executive Director George Warrington took the "aggressive step of personally directing a Trans-Hudson Emergency Transportation Task Force to coordinate interagency responses. It was something that needed to be done, but no one had ever done it. Now we have plans put together no matter what the emergency involves. We've conducted tabletop exercises to make sure our plans work, and we have both sides of the river talking together."
Most of all, plan for all types of emergencies, officials counsel, as crises can come in many different forms. It is true that emergency responses can have a great deal of overlap; several organizations in Detroit mentioned that their snow emergency plans were useful during the blackout. But the more types of events considered under a plan, the more likely an agency will be prepared for whatever happens.
Operating Decisions
Emergency planning enables staff to make day-of-event decisions operate more smoothly. Even for agencies that decided to shut down completely, a preestablished plan made those choices easier. Because a plan was in place, for example, the Detroit-Windsor Tunnel staff was able to make a swift decision to close the tunnel shortly after losing power.
Nevertheless, planning may not prepare an agency for all contingencies, and the operating decision to close the tunnel had ramifications. Executive Vice President Belitsky notes that Canadian nurses staff many U.S. hospitals, and shutting down the tunnel meant they could not get to work. "We've refined some of our procedures now for getting them across the border," he says. Furthermore, operation centers were opened on both sides of the U.S.-Canadian border, but transportation officials were unaware of how to reach each other. "So now we know the phone numbers and even addresses in case we have to go there physically," says Belitsky.
Operating Actions Taken
|
In New York, tunnel managers made several key decisions throughout the blackout. One was to close some traffic lanes within some tunnels. Because the facilities' ventilation systems require an excessive amount of power, managers previously had decided not to connect them to the backup system. Therefore, the tunnel operators had to reduce the number of cars allowed through at any one time to decrease the pollutants. Some bridge and tunnel operators reversed one lane so that there would be three lanes for traffic leaving Manhattan and one for vehicles entering the area.
Agency managers may have to make a number of other difficult operating decisions, such as how to fill staffing needs, how to best serve customers under the circumstances, and whether to continue operations at all. Officials noted the need for a preestablished plan for operating during a communications outage. At agencies that had a plan, field staff members were able to respond to the changing conditions without direct supervision from headquarters. Education, training, and drills may help agency members make better decisions under stressful conditions.
"During the blackout, NJ TRANSIT shuttled commuters from the Port Authority Bus Terminal in Manhattan to the Meadowlands Complex/Giants Stadium, which served as a temporary staging area for New Jersey commuters, who were then shuttled on their normal bus routes to their homes," says Chief Bober. Managers decided on a load-and-go solution to move commuters out of New York City regardless of their final destination, which helped alleviate congestion and confusion.
"For the most part, everyone was together who had to be together to make intelligent decisions," says the chief. "The decisions weren't being made in a vacuum, but rather jointly and in the best interests of the customer."
After the blackout, NJ TRANSIT built an emergency operations center (EOC) that is outfitted with the latest equipment, plus direct lines to the agencies that NJ TRANSIT deals with during emergencies. "The EOC is a think tank and decision tank where everyone sits who decides how we're going to handle the situation," says Chief Bober.
Transit agency representatives add that it is important to set priorities as quickly as possible. Once officials at NYC Transit realized that the blackout would last more than 30 minutes, they initiated established evacuation policies for the subways. Another piece of advice shared by highway and transit agencies: Develop procedures detailing when to restart your system, given that additional power outages, surges, and spikes may suggest waiting for a "quiet time" without surges before starting up.
Agency Coordination
The response to catastrophic events usually requires participation by Federal, State, regional, and local agencies. Coordination among agencies should be ongoing and should be continually reassessed, particularly after a serious incident.
Internally, transportation agencies need coordination among staff, especially for agencies with many different operating entities. As one agency representative mentioned, knowing your colleagues and how to connect them may prove extremely helpful.
Externally, personnel from a transportation agency need to know the functions and capabilities of other transportation agencies. During the blackout, the staff of bridge and tunnel experts that still had functioning cameras verbally relayed existing conditions to the agencies that could no longer obtain video feeds. In addition, preexisting relationships with private carriers to provide substitute and supplemental transportation services were helpful in providing options during the first hours of the blackout.
Coordinating Actions Taken
|
As an example of what can go wrong, one agency official in Detroit noted that long-standing rivalry with another agency made it almost impossible for the two to collaborate in even a basic way, although their cooperation might have eased congestion during the blackout. In another example of the problems that can occur if multiagency coordination is lacking, officials from multiple agencies were required to give permission for the establishment of an emergency response center, greatly slowing its opening. In other cases, established chains of command among agencies proved to be cumbersome and inefficient, also reducing response time.
Transportation agency staff members need to know the capabilities of nontransportation agencies as well and have strong working relationships with law enforcement, public utilities, public health officials, fire departments, and other emergency responders. Because NYC Transit had a preexisting relationship with the electricity provider, Con Edison, the transit agency was able to obtain accurate information on the extent and possible duration of the blackout, enabling the agency to begin evacuating the subways quickly.
The New York area and Great Lakes region case studies revealed that coordination between agencies during emergencies can exist on two levels, both formally among institutions and informally between individuals. As a formal example, the Transportation Operations Coordinating Committee (TRANSCOM), funded by 18 agencies in the Connecticut, New Jersey, and New York metropolitan area, serves as an information exchange network for incidents affecting traffic and transit conditions in that region. The I–95 Corridor Coalition, which includes representatives from transportation agencies in States from Maine to Florida, provides funding to contract TRANSCOM's services in this role when incidents outside the tristate area have impacts on the corridor. "This is when TRANSCOM puts on its I–95 hat," says Marygrace Parker, operations coordinator for the coalition and a former traffic manager and police officer, "and does the same services outside its region."
Both TRANSCOM and NJ TRANSIT have toll-free numbers that staffs from the various agencies can call so that they can conduct conference calls. In addition, TRANSCOM had previously hired a contractor to provide a faxing service physically located outside the region.
Sometimes agencies even agree to share office space. The Connecticut DOT offers space in its building for FHWA Connecticut Division Office staff in the event that the FHWA facility is rendered inoperable. In a similar effort, since the blackout, some of the major transportation agencies in the New York region are linked by dedicated telephone landlines into one another's offices and into those of their contract carriers.
Informal individual relationships may be even more important. Relationships established and nurtured during normal times can pay enormous dividends during emergencies. The Port Authority of New York and New Jersey was able to borrow large generators mounted on trailers from the Port Authority of Baltimore and of Philadelphia. Staff attributed this action to the fact that a district manager had developed relationships with transportation officials in those cities after September 11. The manager knew whom to call.
"If you know each others' resources and understand each others' needs," says Parker, "you start to develop a comfort level. Even if you have no way to communicate during an emergency like the blackout, the operator in Virginia can still anticipate your needs based on the last 10 times you've worked together and will start using his equipment to assist. In this case, Virginia DOT utilized VMS equipment to post messages regarding transportation impacts in the Northeast region."
Technology Actions Taken
|
Role of Advanced Technology
Over the last decade, ITS technology has played an increasingly significant role in helping managers operate their systems during both normal times and emergencies. Advanced technology is vulnerable to the loss of power, however, at any point along the information chain–from equipment in the field to the control centers.
Agencies in New York City and Detroit reported that the ITS equipment that stopped working included traffic signals and signal controllers, variable message signs, closed circuit television cameras, global positioning systems, and some automatic vehicle location systems. Other technologies also were unreliable, such as cellular telephones, cordless telephones, some pagers, some landline telephone systems (once phone company backup battery power expired), some fuel pump computers, two-way portable radios (due to nonfunctioning repeaters), some point-to-point radios in buses, some computer-aided dispatch systems, and climate control and lights in some traffic information centers.
Steven Fern, manager of electronics and communications at SMART in Detroit, says, "The local phone carrier did fine in keeping analog services up and running, just as they have been doing since my grand-father's day. The digital service didn't die immediately, but did 12 to 14 hours into the event, as the in-circuit devices between the central phone office and customers' offices lost power."
His colleague, SMART Director Ristau, adds, "Phone service was somewhat sketchy on Friday. We did try to keep our small bus service going, especially for kidney dialysis patients. Because we couldn't reach some hospitals by phone, however, we reverted to a ‘pony express' kind of system where we sent drivers out physically to the hospitals to see if they were open and doing dialysis."
An important lesson learned is the need to identify those components of the ITS network that should be capable of operating during an emergency. Allocating capital and operating funds to maintain backup power in those parts of the system is imperative. The root causes of a power loss--storm, technological failure, violence, or sabotage--are irrelevant. All blackouts present the same challenges, and the ITS systems should be designed to function no matter what the specifics of the situation are.
Other lessons learned about advanced technology:
- Intersections: The ability to maintain the signal system is key, unlike with freeways, which will operate reasonably well without power. A response plan for priority signals should be based on an evaluation of intersection safety, sight distance, and traffic volumes. Several agencies are now examining the costs of providing backup power supplies to ITS field equipment along key corridors and intersections. Emergency power for the computers controlling the centralized traffic signals--those run from a traffic management center--should be provided by multiple backup batteries.
- Communications: "Old" technologies, such as landline telephones and battery-operated radios, become very important when a crisis is widespread. Officials recommend maintaining some older technology, which may be less susceptible to power outages, in the traffic management center. As for backup power, even quadruple redundancy is not foolproof.
- Operations: Prioritize where backup generators are deployed. To keep emergency vehicles on the road, backup power is essential at fueling facilities. When an outside contractor performs generator maintenance, in-house staff should retain maintenance capability for emergency situations. Regular generator maintenance should be scheduled as standard operating procedure. The ability to maintain toll collection using backup power allows large volumes of traffic to be handled at toll booths with few customers aware of the blackout until they leave the tollway.
SMART's Fern adds that the first lesson learned was to perform system tests. "But any kind of test where you're simulating a failure is inherently an intrusive thing to do," he says. "It gets in the way of business, but there's no other way to know whether your equipment is going to work. We do a WAN [wide area network] failover test every 4 to 6 weeks. We unplug a fiber, actually pull the plug, making it happen exactly the way it would in the real world. It's like security. You can have as much as you want to pay for, but there will be inconvenience."
Although a large percentage of ITS equipment was not fully functioning within the affected areas, agencies outside the blackout area were able to use ITS technologies to alert motorists. The Maryland DOT, PennDOT, New Jersey DOT, and New Jersey Turnpike, among others, all used their ITS technologies to broadcast alerts using variable message signs, highway advisory radio, and Web-based messages. For example, the Maryland DOT placed messages on its I–95 northbound signs, "Massive power outage in NY--Avoid area--Use alternate routes."
Communications Actions Taken
|
Technical Communications
The ability to communicate, internally and externally, is the most critical technological capability required in an emergency, according to the case study findings. Fast communication is crucial to stem anxiety, transmit instructions, and begin the process of recovery. Reliable communications technology is particularly important for transportation agencies, since many employees may be working in the field. Agency staff should drill specifically for the failure of communications equipment.
During the 2003 blackout, transportation officials learned the importance of low-tech solutions. The plain old telephone system proved to be the most reliable form of communications technology during the blackout, because cell phones, cell phone towers, radio repeaters, and Internet connections failed due to the loss of electrical power and limited backup capabilities. Agencies that had newer central telephone systems experienced more problems than agencies with older systems. To communicate, agency representatives used a combination of facsimile machines, pagers, toll-free numbers and conference call lines, older radio systems, and previously installed dedicated landlines.
Ohio DOT found that agency managers' landlines worked, but many were nevertheless unaware that they were receiving a call because the electronic ringers on their telephones were inoperable. Ohio DOT's Huebner has an old-fashioned bell ringer on one of his telephones at home and heard that one.
He adds, "Our two-way pagers helped us communicate the most. Our public information officers have pagers, and after about 4 hours, the cell phones came back on, so they communicated with the media mostly by cell." Huebner notes that Ohio DOT benefited from the helicopters that the local news channels had in the air, reporting on darkened intersections and traffic jams.
Steven Coleman, public affairs officer with the Port Authority of New York and New Jersey, got the word out within the first hour by typing a summary of traffic activity on pagers for executives to review and then using cell phones to call the Associated Press, which then distributed the information to other media. "Then all of our staff was given a police escort across to New Jersey to the emergency command center," Coleman says, "where we have generators and could do much broader outreach. We staffed that operation throughout Thursday night and then all day Friday and Friday night."
Agency officials need to understand how to provide the media and the public with accurate and timely information. Preestablished relationships with media outlets can help an agency better alert its customers about available options. NJ TRANSIT staff is devising a training program for members of the media to enable them to identify the responsibilities of the various transportation agencies in the region and the facilities that those agencies manage or operate. Part of this training will involve developing a list of agency points of contact for the media plus a list of media contacts for the agencies.
Since the 2003 blackout, the Ohio DOT has developed a new process flow for situations where communications go down. "Managers are to report to work," says Huebner, "and notify the district office through their radios on how many crews they've brought in. And we will notify the media, even if we have to drive there, and the local media have agreed to broadcast public service announcements such as ‘ODOT shift 1 is to report.'"
The Trans-Hudson Emergency Transportation Task Force identified communications technology as the leading problem of the blackout. The task force's findings were that most agencies thought they had more communications redundancy than they did, failed to understand the frailty of their technology, and thought that they had better backup power.
The inability to communicate reliably during the blackout was also the most consistent finding in the Great Lakes region. The more communications options that agency personnel have at their disposal, the better. Although many worked at certain times--fax, e-mail, cell phone, pagers and text messaging, two-way radio--none of the technologies worked all the time.
Recommendations are to establish direct lines of communication with nontransportation agencies and media outlets, establish a NonComm plan, and strengthen communications among transportation agencies by preestablishing a protocol for communication--telephone, fax, or Internet--with particular attention paid to ensuring redundancy in those systems.
Another recommendation is to explore the option of joining the Government Emergency Telecommunications Service (GETS) and Wireless Priority Service (WPS), two federally sponsored priority communications systems that provide preapproved users with priority routing of landline (GETS) and wireless (WPS) calls during emergencies, even during periods of peak demand. Users include Federal, State, and local government agencies, plus private companies and organizations with responsibility for national security or emergency preparedness. Transportation agencies are granted a fourth-level priority within GETS and WPS, as are other public utility agencies. For more information, see http://gets.ncs.gov/ and http://wps.ncs.gov.
System Redundancy and Resiliency
The level of appropriate redundancy–the ability to activate backup systems for expertise, equipment, vehicles, communications, and technology–varies from agency to agency. Redundancy needs to be reevaluated constantly based on the results of emergency response training and experiences during actual emergencies. Some large-scale events, such as the 2003 blackout, may always exceed the amount of available redundancy. Given financial and other constraints, managers must assume the most likely types of potential emergencies when planning for redundancy.
From the experience of the blackout, clearly a source of backup power is the most important investment an agency can make. Most other systems--communications, safety, and security--will operate so long as backup power is available and sufficient. Emergency power must be tested and maintained, and must be connected to the appropriate systems.
Redundancy Actions Taken
|
Communications is another area where redundancy can be important. Sometimes double and even triple forms of communications alternatives are not enough. Several areas of New York City experienced a loss of landline communications because three of one phone company's central offices experienced outages. Cell phones and radio networks failed since repeaters either were not equipped with backup power or had batteries that ran out after approximately 4 hours. Also, call volumes overwhelmed the phone systems. Even the city's 911 system failed due to the heavy call volume.
Many times, the need for backup power is not obvious until an agency tries to function without it and a gap is identified. Air-conditioning consumes a large amount of power to cool a building, so it is often not included in the backup power system. But critical technologies such as computers need to be cooled so that they do not overheat.
After August 14, agencies noted some of the items that were left off of the backup power supply but should have been considered for possible inclusion:
- Electronic keyed door entry systems
- Network-based telephone systems
- Fueling systems for public and private vehicles
- Sump pumps for tunnels or roadway sections that are prone to flooding
- Spare outlets for small appliances (such as battery rechargers)
- Air-conditioning for equipment rooms
- Internet servers hosting e-mail systems
- Radio communications systems
- Building security systems
The 2003 blackout served to expand the definition of redundancy. The blackout highlighted the possible necessity of having backup centers located physically outside of the affected region and "virtual operations centers." By having the capacity to connect a computer into a virtual network, an agency can run its operations from a secondary site or even from the homes of key personnel. Redundant facilities are less likely to be affected by the same events as the primary operations site if they are located far enough away.
At a minimum, planners should consider designing redundancy in several areas: agency personnel, communications, utilities, control centers, and equipment and supplies. They must also be aware of the redundancy in the regional transportation system. Agencies should perform inventories of their own internal assets as well as establish contact with other agencies where additional assets could be requested. In many circumstances, these strategies are cost-effective. But if an event covers a wide geographic area, as the 2003 blackout did, parties that have agreed to assist one another may be unable to do so because they need to address their own set of problems or are spread too thin.
Also, test and maintain backup systems. A number of agencies had invested in generators but found that they malfunctioned, required repairs, or even caught on fire when called upon for extended use during the 2003 blackout. Other agencies reported that they test their generators by switching to backup power and simulating a load on their backup system on a weekly or monthly basis for 12- or 24-hour stints. They also schedule routine preventative maintenance.
Assessing the needs posed by an extended loss of the primary system versus a temporary interruption is crucial. Equipment such as air-conditioning and fuel pumps became important as the blackout continued. One lesson learned in this category is that basic facilities should be provided for staff members who may have to work multiple shifts during a prolonged emergency.
But building redundancy into the system can be expensive and seen as "wasteful spending" in ordinary times. It is always cheaper to have only one of a particular type of infrastructure, but failure of that system can hamper response and recovery efforts significantly.
"One of the things we learned is that backup generators and uninterrupted power supplies are worth the money," says Richard W. Morgan, director of information systems at the Ohio Turnpike Commission. "The sign of that is that the power outage did not inhibit anyone from being able to enter the toll road, and investors didn't lose any money."
Because the Ohio Turnpike's service plazas have backup generators, they were able to continue dispensing fuel. At one point, because most gas stations were unable to pump without electricity, the Toledo radio and television stations informed Detroit motorists that they could obtain gasoline at the turnpike's service plazas. "People could come to the turnpike and get gas," says Morgan, "and they did."
Much of the turnpike's backup equipment has been in place for 20 years. Morgan adds that building in redundancy "as you go along makes it easier to intelligently improvise on the fly for something that hasn't been preplanned."
Wake-Up Call
"At the end of the day," says the I–95 Corridor Coalition's Marygrace Parker, "whether it's a hurricane, a blackout, or a terrorist event, the way you manage traffic incidents is essentially the same. There's no incident that occurs on any scale that doesn't have traffic impacts. And most important is having the institutional and personal relations in place, nurturing and sustaining them."
In other words, institutional coordination--and emergency response plans, effective operating decisions, advanced technology, technical communications, and system redundancy–all are vital components of preparation. Most important to remember is that preparation is never complete. It is an ongoing process. The 2003 blackout is just another in a long series of wake-up calls about the need to prepare for the unexpected.