NYSEG project team members LaWanda Ervin and Steve Hope review the ARIES assessment results for a staged storm event with Lockheed Martin systems engineers Tim Douglas and Alex Elhage during initial system testing at Lockheed Martin’s facility in Owego, New York.

Utilities Automate Damage Assessment

July 21, 2016
AVANGRID companies partner in the development of an aerial damage-assessment solution.

Superstorm Sandy — a monster hurricane packing torrential downpours and 80-mph (129-kmph) winds — roared into the northeastern United States on Oct. 29, 2012, disrupting electricity service to hundreds of thousands. That extreme weather event — along with others before and since — have led the utility industry to recognize the need for a new level of response to storms.

Among the utilities developing novel approaches to enhancing storm response are AVANGRID companies New York State Electric & Gas (NYSEG) and Rochester Gas and Electric (RG&E), which serve nearly 3 million electricity and natural gas customers in upstate New York.

While Long Island and New York City bore the brunt of the damage from Sandy’s wind, rain and storm surge, more than 117,000 NYSEG customers and 27,000 RG&E customers were without power. One particularly hard-hit area of the region was NYSEG’s Brewster division, just north of New York City, which sustained extensive distribution system damage in the form of more than 600 broken poles. The preliminary damage assessment in the region required more than six days to complete because of inclement weather and the many impassible roads.

To improve damage assessment in the future, NYSEG and RG&E are working with Lockheed Martin to assess how light detection and ranging (LiDAR) technology deployed in a helicopter may assist utilities in collecting information on damage to electricity delivery systems following major storms.

Raw LiDAR data collected during flyovers will be passed through computer-processing algorithms developed by Lockheed Martin. The automated output from the Automated Rapid Infrastructure Evaluation System (ARIES) will quickly provide NYSEG the locations and details of damage, enabling a more focused response, proper crew resourcing and accurate materials procurement.

NYSEG’s Brewster division suffered more than 600 broken poles as a result of Superstorm Sandy. However, impassible roads made it difficult to realize the full extent of the damage until preliminary damage assessment was completed nearly a week after the storm.

Shrinking the Assessment Time

As a result of Superstorm Sandy, New York State’s Public Service Commission (PSC) developed a “scorecard” to assess utility response to major outage events lasting 72 hours or longer. More than 20% of the scoring criteria are tied directly to a utility’s ability to perform preliminary damage assessment and communicate estimated time of restoration (ETR) information within 24 hours. After significant weather events, damage assessment can require several days using current methods because of widespread impassable roadways and inefficient methods of aggregating information. To minimize outage durations, improve customer satisfaction and achieve the damage-assessment timeline set forth in the PSC scorecard, NYSEG and RG&E realized traditional ground-based assessment methods would not suffice. They saw an opportunity to partner with Lockheed Martin to develop a new automated aerial approach.

The damage-assessment methods used by AVANGRID following Superstorm Sandy aligned with the standard approach used throughout the utility industry today: ground-based crews driving along circuits at speeds of 5 mph to 10 mph (8 kmph to 16 kmph) in hazardous conditions, manually entering damage findings into handheld computing devices. Not only was this process tedious and time-consuming, but the resulting information was often subjective, incomplete and sometimes inaccurate.

In 2013, NYSEG, RG&E and the New York State Energy Research and Development Authority (NYSERDA) began discussions with Lockheed Martin to determine how its existing portfolio of decision-support and data-analytics tools could be applied to the realm of automated rapid damage assessment. Interest quickly focused on the ability to adapt Lockheed Martin’s machine vision and image-processing algorithms to assess utility infrastructure damage from aerial imagery.

Throughout the following year, Lockheed Martin collected representative visual camera and LiDAR imagery of distribution systems within NYSEG’s service area from a piloted helicopter. An initial collaboration between NYSERDA and Lockheed Martin resulted in a successful proof-of-concept demonstration of the application of this technology to damage assessment. Not only would this approach dramatically reduce the magnitude of AVANGRID’s resources required to perform a preliminary damage assessment and, hence, reduce costs, but it also would remove the dependencies associated with ground-based assessments (for example, blocked roads and downed conductors) to enable a complete and accurate assessment of the entire impacted area within a matter of hours, as opposed to days.

Data analytics and damage-assessment algorithms developed by Lockheed Martin automatically extract, classify and assess assets of interest from the LiDAR imagery using a utility’s GIS asset database of pole locations as initial guidance.

A New Approach

By the end of 2015, NYSEG and RG&E partnered with Lockheed Martin to develop an implementable prototype system, known as ARIES, featuring automated damage detection by processing aerial imagery through data analytics, complete with a graphical user interface (GUI) for mission planning, status monitoring and results analysis. In addition to damage assessment, the system’s data analytics are also capable of detecting floodwaters within the same type of aerial imagery.

The solution will be instrumental in a more rapid assessment of damage to AVANGRID’s electric network during significant outage events. Data gathered by the new system will provide the ability to make critical resource decisions earlier in an event.

ARIES consists of a data-collection process performed by a helicopter equipped with a LiDAR sensor, the output from which is fed into data-analytics algorithms to produce fast and accurate post-storm assessment results. Depending on the population and distribution of assets, imagery can be collected over an area up to 500 sq miles (1295 sq km) within 24 hours. This data is iteratively transferred to ground-based systems and personnel for subsequent automated analysis and review.

A key aspect of the solution is the data-analytics software developed by Lockheed Martin to enable automatic and detailed damage assessment based on a single, one-time collection of aerial LiDAR imagery. This would not require a baseline set of imagery collected prior to the storm. The system’s analytics software compares the LiDAR imagery to the GIS database of known utility pole locations to determine whether damage is present. When damage is found, the imagery and corresponding damage-assessment results can be presented to an analyst in a GUI for review and confirmation.

For this reason, an optional high-definition camera can be employed so that pictures are presented with the LiDAR imagery in the GUI to help analysts clearly see the damage. The system supports user-initiated generation of damage-assessment reports with varying levels of detail at any time, ensuring repair efforts can be quantified and prioritized efficiently. An accurate ETR can be released promptly to customers following a major storm event.

To achieve efficient data collection and timely analysis while minimizing the number of personnel deployed for damage assessment during and after a storm, AVANGRID and Lockheed Martin have established a distributed architecture for ARIES that enables an optimal use of resources. This architecture consists of a mobile damage-assessment center that can be deployed to a location near or within the impacted area. Imagery collected from the helicopter is periodically offloaded and processed through the data analytics at the mobile damage-assessment center.

Assessment status, results and data for any assets desired to be reviewed by analysts is transferred to any one of AVANGRID’s home offices by a satellite-based network connection. This approach requires a minimal support team to be deployed with the mobile damage-assessment center and enables greater visibility and real-time communication with the larger assessment team back at the home office.

The ARIES distributed architecture intelligently condenses large volumes of collected imagery into comprehensive assessment statistics and reports that can be wirelessly transferred to any home office location for review.

Future of Damage Assessment

Preceding a major storm event, initial preparations are made to ready the system and notify flight crews of potential deployment depending on the anticipated storm track and intensity. Following the storm, the damage-assessment coordinator identifies the data collection area on a digital map display in the GUI. The selected area is matched against a utility’s available asset data. The system then generates a list of asset locations to be used as flight waypoints based on the locations of selected assets, which can be provided to the helicopter crew.

The helicopter crew reviews the intended flight path based on the waypoints file, inputs the flight path into the sensor mission planning software, flies the selected routes and collects imagery of the selected assets within the data collection area using LiDAR. To cover even larger areas in a short amount of time, the system supports the concurrent use of multiple helicopters for data collection. When the helicopter lands, the storage media used to collect data during the flight is removed and readied for analysis. The helicopter crew can then refuel the helicopter, upload and review the next flight path, install fresh storage media and continue flights as needed to survey routes until all required data is collected. The timeline of assessment and repair associated with the new system shows significant shortening. It is expected that the entire preliminary assessment time frame to be within 24 hours of the first flight.

When data is moved from the storage media into the system, the system tracks which utility assets were flown and begins the automatic assessment of those assets. This automatic assessment occurs through the use of software that detects and assesses electrical distribution and transmission assets along a circuit. As assets are assessed by the algorithms, the GUI displays the assessment status on map displays, while analysts use the GUI to select assets for review and confirmation using a variety of filter selections.

Assessment results are displayed hierarchically on a heat map, showing relative damage severity across a geographic area using various color codes for selected assets. In addition, assessment reports can be generated, viewed and exported in a Microsoft Excel-compatible file format using the GUI.

The ARIES solution drastically shortens the preliminary damage-assessment timeline by performing automated analysis and assessment in parallel with LiDAR data-collection flights.

Future Prospects

Ultimately, the purpose of ARIES is to reduce the duration AVANGRID’s customers are without power by providing a reliable and accurate tool for rapidly assessing damage and developing an ETR. Beyond the initial helicopter-based solution, AVANGRID and Lockheed Martin already are investigating the adaptability of the system to unmanned aerial vehicle (UAV) platforms as U.S. federal regulations evolve.

Because of the platform-agnostic nature of the architecture and data-analytics software, the expectation is for the solution to be easily extensible to UAV-based platforms to meet the evolving needs of the broader utility industry in the coming years.

As final testing and deployment of the ARIES system approaches, AVANGRID grows more encouraged and excited for its success. The synergy of the project team has been key to the development of this new product. It has been a true partnership between Lockheed Martin and AVANGRID in the growth from concept to reality. The first ARIES prototype is scheduled to be deployed within NYSEG by early 2017. 

The assessment results from ARIES are organized into a decision-support tool with graphical user interface that highlights the hardest-hit areas, enabling NYSEG to quickly and accurately assess damage, distribute appropriate resources and resume normal operations as quickly as possible.


The author would like to acknowledge the following members of the ARIES team — Rick Evans, program manager; Bob Strebel, technical lead and chief software architect; Tim Douglas, systems engineering lead; Reed Durand, systems engineer; Dave Murphy, software development lead; Bing Li, chief technologist; and Ben Tongue, business development — and NYSEG’s team — Raquel Mercado, LaWanda Ervin, Steve Hope and Carrie Berard.

Ann LePore has spent years managing technology projects that include the integration of technology and business processes to improve efficiency and effectiveness. As program manager of networks planning and innovation, she is responsible for strategic planning and managing innovation projects. Her experience at Iberdrola USA includes energy control center emergency response system management, IT business relationship management, supporting storm response and participating in the Emergency Management Operating Council. LePore holds a BS degree from the Rochester Institute of Technology and is a member of the Finger Lakes STEM Hub Steering Committee.

Sidebar: Why Wait for a Storm?

One of the fundamental aspects of creating a robust and reliable automated damage-assessment system is being able to evaluate its performance on real-world utility infrastructure damage. To preclude delaying the project in the hopes of taking a damaging hit from a storm in the near future, New York State Electric & Gas (NYSEG) collaborated with Lockheed Martin to create a variety of realistic damage scenarios to transmission and distribution poles and conductors at NYSEG’s training facility in Binghamton, New York, U.S.

Lockheed Martin then flew several light detection and ranging (LiDAR) data-collection missions over the facility, capturing imagery of the staged damage to augment and enhance the development and ultimate performance of the Automated Rapid Infrastructure Evaluation System (ARIES) data analytics.

The effort turned out to be a true win-win. In addition to feeding Lockheed Martin’s development efforts, NYSEG capitalized on the opportunity to train several lineman apprentice classes in creating and repairing damage as part of the exercise. The first of NYSEG’s weeklong classes planned and coordinated the damage staged to existing nonenergized transmission and distribution systems regularly used for training purposes.

Members of NYSEG’s lineman apprentice class hoist a distribution pole and conductors into a treeline to emulate realistic storm damage prior to a LiDAR data-collection flight for ARIES development and test.

The scenarios were designed to be representative of real damage following a storm event, while in the process testing the damage-detection capabilities of ARIES data analytics. In addition to broken and leaning poles, and sagging and downed conductors, the team coated several common types of conductors in a layer of ice to emulate conditions following an ice storm and to test the system’s ability to detect conductors in such conditions.

Jim Fiallo, NYSEG lead advisor–training, and Tim Douglas, Lockheed Martin systems engineer, stage a series of ice-coated conductors just prior to an ARIES LiDAR data collection flight to test the system’s functionality in conditions following an ice storm.

Following construction of the damage scenarios, Lockheed Martin had a three-week window to perform data-collection flights over the area with various LiDAR sensors to investigate both sensor and analytics performance in this environment. In concert, NYSEG and Lockheed Martin developed a mock geographic information system database of the structure and circuit configurations with corresponding pictures of each pole for reference. By comparing the true locations and types of damage with the ARIES assessment output, Lockheed Martin is able to continue tuning the data analytics to recognize and report damaged assets better from realistic imagery.

The scenarios and data resulting from this collaborative effort provided valuable inputs for the development and testing of ARIES that are truly representative of the anticipated behavior and performance of the system following an actual storm event.

A LiDAR collection helicopter flies over one of the damage scenarios staged at NYSEG’s training facility, producing a 3-D “point cloud” representation of the infrastructure that is automatically processed through ARIES data-analytics algorithms to determine an assessment.
NYSEG project team members LaWanda Ervin and Steve Hope strategize streamlining their assessment process using ARIES with Lockheed Martin systems engineer Tim Douglas.

Voice your opinion!

To join the conversation, and become an exclusive member of T&D World, create an account today!