A straight-line wind tore through Minnesota in June 2013, inflicting the largest power outage in Xcel Energy’s history. Gargantuan trees toppled onto wires, and wood distribution poles snapped into pieces.
Wind had ripped down tree limbs in past weather events, but this storm caused unprecedented damage. Because the ground was so saturated, entire trees were pulled out of the ground, and many of them landed on the utility’s distribution infrastructure.
The storm front spanned nearly 100 miles long and traveled right through the heart of the Twin Cities metropolitan area. The area was hit with three separate events within a 24-hour period, knocking out power to 600,000 customers and taking out 73% of the system.
Mobilizing Crews
Xcel Energy did not wait until the storm hit to begin its emergency-response plan. Instead, the utility started planning as soon as it became aware of a weather-related risk. Typically, the utility begins planning two days in advance of a storm, but the utility is nimble enough to respond on a moment’s notice.
In the event of severe impending weather, Xcel Energy organizes an operations call with key leadership to discuss the steps to prepare for the storm response. For example, the managers must verbally inform all electrical workers of the threat and tell them to prepare their personal affairs as they may be gone for an extended period of time. Also, the utility shares the forecast information so workers can inform their families of the possible work assignment.
During the preplanning phase, Xcel Energy’s managers identify several prestaged crews to be on site when the weather hits. When appropriate, the utility uses its ARCOS Callout to contact crew members automatically and ask them to report to their service center. This system can contact all line workers for immediate notification, and then managers can follow up with phone calls to field employees who do not respond or are on vacation prior to the event.
Enduring Adverse Conditions
With its emergency storm-response plan in place, Xcel Energy was prepared to handle the storm restoration in June 2013. The high winds blew through the city, and no one was able to escape from the wrath of the storm, which took down a main line in the middle of a residential area.
The fire department first barricaded the road until linemen could clear the downed wires. Using their mobile data terminals, the field crews first determined they were in the right spot. Then they used phase sticks and proximity testers to alert them when energized lines and equipment were nearby.
Because the storm came in three different waves within a short time frame, the linemen would make repairs and restore power, only to have the poles and lines ripped down by the strong winds once again.
When the second storm hit, the field crews were in full storm-restoration mode. Then, when the next wave of the storm hit, the linemen were already out on the site.
For example, one crew was just getting ready to throw the fuse into the cutout and restore the tap when the sirens started blaring. As the wind picked up, the linemen had to try to hold on to the pole and ride out the storm. With no safe way to make it down to the ground, they had to stay in position while the winds whipped through the restoration zone. After the winds died down, the linemen had to make the same repairs and, once again, work to restore power.
The linemen worked about 16 hours a day on the storm restoration. The crews first focused on responding to the power outages that affected the most customers. When another storm came through, the field crews had to regroup, restart and reprioritize tasks. As a result, the storm-restoration process was more complicated than in previous storm events.
Because of the sheer magnitude of the storm and the number of outages, Xcel Energy engaged its regional mutual aid contacts in the Midwest after the first wave. However, as the second and third storms hit, the utility had to reach beyond those utilities and contractors to get even more assistance. In the end, Xcel Energy had more than 1,000 field workers handling storm restoration, which is the largest field workforce the utility has had in Minnesota following a storm.
Partnering with Others
When responding to a major storm, safety is paramount for field crews as well as mutual aid utilities and contractors. To prevent injuries and incidents on the restoration site, the utility mandates all of the external workers attend a safety briefing. During this meeting, the safety professionals and management team meet with the crews to review Xcel Energy’s policies and procedures.
To ensure mutual aid workers are up-to-date on current conditions, Xcel Energy assigns each crew a bird dog in the field. These key contacts can provide the mutual aid crews with data from their mobile data terminals. The engineers and managers meet with each crew before linemen go into the field. Then they review their expectations and their contact at Xcel Energy.
Overall, the storm-restoration effort in June 2013 went smoothly, and the workers were well distributed across the restoration zone. Rather than bunching up the workers in one area, the dispatchers sent them to different areas throughout the city so they did not get in each other’s way.
Rather than setting up a tent city, as is often the case with large-scale storm restoration efforts, Xcel Energy instead was able to set up all of its outside crews and contractors in hotels within a half hour of the Twin Cities. The utility then set up two common staging areas near St. Paul, Minnesota, and the suppliers delivered the new poles, hardware, lines and other equipment to these locations.
At the beginning of the work day, crews parked their trucks at these staging areas so they could review any issues for the day before starting work in the field. For example, many customers were running their own generators, and the workers needed to be cognizant of energized fences, trees or branches, also known as widow makers.
Every day, Xcel Energy strived to hold tailgate meetings to identify any challenges or safety hazards. The utility encouraged all of its employees and the mutual aid workers to speak up if they had a safety concern or question. Work practices were altered, when necessary, for working safely in the field.
By pulling together as a team, Xcel Energy and its mutual aid crews and contractors were able to restore power to customers following the most severe storm in the utility’s history. Despite being confronted with three separate weather events, the linemen were able to repair the damage and rebuild the infrastructure to handle future storms in the utility’s service territory.
Bernie Walter ([email protected]) is a construction foreman for Xcel Energy and has been with the utility for 29 years. He started as an apprentice, topped out as a journeyman and then moved into a field foremen position.
Tom Cascalenda (thom[email protected]) is the director of design and construction for Xcel Energy. He has been with the utility for 36 years, working as a groundman and as an apprentice and journeyman lineman before entering management.
Troy Browen ([email protected]) is the director of control center and trouble for Xcel Energy, and has been with the company for 29 years.
Companies mentioned:
ARCOS | www.arcos-inc.com
Xcel Energy | www.xcelenergy.com
Sidebar: Lessons Learned
After experiencing one of the largest outages in its history, Xcel Energy field managers and crews evaluated their response plan and made adjustments for future storm events. Here is what they discovered following their evaluation:
1. Ensure all linemen can receive orders. Twenty Xcel Energy crews from other operating companies were unable to accept outage orders. Xcel Energy’s business systems unit performed an analysis and utility found that NSPMN, the Minnesota jurisdiction business unit, needed to be added to its profile. Going forward, the utility will complete this task while the crews are en route by using the roster and adding the business units to the incoming employees’ profiles.
2. Select employees to help with initial safety orientation for incoming mutual aid and contract crews. Xcel Energy has expanded its list of employees with this skill set. It plans to have a staging site just for this purpose and then send them to the service centers.
3. Have a single point of contact for wire down inspectors. Because no one person was responsible, Xcel Energy duplicated efforts with damage assessment during the storm. A new process has been drafted and training completed with the design workforce.
4. Have more pseudo identification numbers available. When Xcel Energy brought its mutual aid and contract crews into the service centers, it gave them a prepopulated unique ID number so it could dispatch outage jobs to the outside crews. In the June 2013 storm, the utility came close to running out of pseudo ID numbers. The utility requested additional numbers and plans to test functionality with the implementation of a new Service Suite dispatch application.
5. Update crews on outages. When an outage job is dispatched to a crew with a mobile device, the outage map is updated on the website. It then updates as the crew goes en route and again when the crew is on site as the workers activate buttons on the mobile data terminal. When Xcel dispatched outages to a pseudo ID, this was a manual process to capture outage data. As such, the crews were unable to update the outage map because they were not using a mobile device. One possible solution is performing a mass update of all known crews en route or on site.
Sidebar: By the Numbers
Three consecutive and separate severe thunderstorms with high winds, damaging lightning and torrential rains impacted the heavily populated Minneapolis/St. Paul Twin Cities and surrounding communities. Here are some fast facts about Xcel Energy’s restoration:
• Crews replaced 60 miles of distribution line
• 628 of 861 of the utility’s total distribution feeders were affected, which accounted for 73% of the system
• 100 feeders were locked out
• 8,000 trouble cases were reported
• 430 out of 389,000 distribution poles were replaced
• 200 out of 80,000 distribution transformers were replaced
• 11 transmission circuits were locked out for a total of 2% of the utility’s transmission system
• 5 transmission structures were replaced
• 610,000 customers sustained outages
• 60,846 man-hours were spent by the utility and regular contractors
• 79,824 man-hours were logged by visiting contractors and assisting utilities
• 1,260 electrical workers, 285 tree/vegetation workers and 250 other workers came from Minnesota, South Dakota, North Dakota, Wisconsin, Colorado, Iowa, Nebraska, Wyoming, Kansas, Texas, Missouri, Michigan, Indiana, Ohio, West Virginia and Pennsylvania.
• 705,000 calls came in on the utility’s toll-free lines.
