line scout

Valuable New Data Sources for Utilities

Dec. 20, 2018
Utilities can seize the day, if they practice holistic data management

New data sources for power utilities continue to emerge, a trend that is likely to continue in the digital age.

Whether these new data sources can and will be harnessed by utilities for significant value, however, remains an open question. This issue has two components.

First, a utility must practice holistic data management to take advantage of digital data routinely generated by its sensors and systems or external sources to improve its operations and create enterprise value. In doing so, a utility can more easily integrate new data sources as they become available.

Second, the power industry must support efforts to define standards, develop applications and share best practices for new data sources to speed time to value.

If these efforts are made in tandem, value creation by a data-driven utility is a certainty.

Be forewarned: Becoming a data-driven utility through holistic data management practices requires a cultural shift to organization-wide cooperation that will change traditional business processes. Once set in motion, that trend is irreversible.

New data sources

Data is the new enabler of value. Today, nearly every group within a utility organization, on both operations and enterprise sides, should seek to optimize their work and gain competitive advantages through data and its analysis. In an increasingly deregulated marketplace prowled by third parties eager to serve our digitally savvy customers, power utilities must exploit every data source that can yield value.

Today, data is becoming available from previously unexpected sources and in formats, speeds, scale and granularity that challenge our ability to analyze it for actionable insights. Three examples of emerging data sources underscore this point: social media, unmanned aerial vehicles (UAVs, aka “drones”), and robotics. Yet myriad data sources already available within our organizations also confirm the efficacy of holistic data management as the basis for managing these new sources.

Let’s look at both new and existing data sources to understand the urgency of adopting holistic data management, which is the key to data-driven value creation.

Social media

Prior to smart meters and advanced metering infrastructure, utilities relied on customer phone calls to report and locate outages. Today, land line phones are vanishing, especially among Millennials, the next-generation of utility customers. Mobile phone-bearing Millennials are more likely to be tweeting on the move, able to provide up-to-the-minute descriptions and/ or photos of outages wherever they are.

A utility can use a customer’s Twitter handle as the linkage to their customer information and property address and encourage customers to send outage information using social media (i.e., tweets). If these customers are additionally incentivized to activate their phone’s geo-tagging feature, their social media output can provide GPS coordinates if they are not tweeting from home that can speed repairs and service restoration. Emerging software applications can analyze and link social media reports to an outage management system and distribution management system, triggering a fault detection isolation and restoration (FDIR) application swiftly enough to improve SAIDI and SAIFI reliability indices. (SAIDI = System Average Interruption Duration Index; SAIFI = System Average Interruption Frequency Index.)

The greater the number of participating customers, the more accurate the data informing a utility’s response. Both parties benefit. As customers play a role in their own service restoration, they are engaged with their utility. Research reveals that engaged customers register greater satisfaction with their utility. The utility benefits from swifter service restoration and a level of customer engagement that can lead to other customer-utility collaborations.   

UAVs, robotics

Unmanned aerial vehicles or UAVs are one component of unmanned aerial systems (UASs) that rely on ground-based pilots and a communications link between the two. “Robotics” refers to machines equipped with sensors, processing and communications capabilities that can replace humans for difficult, dangerous tasks. Inspecting assets, assessing storm damage, non-destructive testing, maintenance of live lines and other tasks can be accomplished with UAVs and robotics.

Current use cases include performing these tasks on high-voltage transmission lines hundreds of feet above the ground as well as underwater at turbine intakes in hydroelectric facilities.

UAVs, for instance, carry a variety of sensors, including traditional photography, LiDar, PhoDar and infrared. LiDar stands for Light Detection and Ranging, which is a surveying method that uses a laser and its reflected pulses to determine, say, topographical features on the ground. PhoDar, or Photogrammetric Detection and Ranging, creates a three-dimensional “point cloud” that assesses various types of terrain including vegetation. Infrared sensors detect heat at wavelengths invisible to the human eye.

Together, these sensors provide reams of potentially valuable data, if a utility is prepared to recover, store, and process it for actionable insights. UAV data can inform asset management, geographic information systems (GIS), outage management systems (OMS), and storm damage assessments.

Success in this endeavor depends on whether a utility is already practicing holistic data management. Yet the industry as a whole must require vendors who provide the various technologies involved to adhere to standards and industry-friendly use cases that result in best practices for value creation. As in past technology advances in the power industry, however, this sensible step is not a given.

In an ad-hoc survey of colleagues in late 2017, I discovered that these amazing machines and their data outputs remain a few steps ahead of the power industry’s current ability to efficiently route, store and analyze that data for valuable insights.

Consider the hours of aerial photography generated by a UAV. It is impractical and an unwise use of resources to have a human reviewing that data. Instead, we need automated imagery analysis based on standards and best practices that will speed time to value. A coordinated power industry effort in this regard will avoid proprietary solutions, support interoperability and, thus, enable more utilities to cost-effectively adopt these new technologies. Otherwise, the tail wags the dog – not a model for progress in the 21st century.   

Becoming a data-driven utility

These eye-catching examples of new data sources simply underscore the value of becoming a data-driven utility and the supporting practice of holistic data management. A brief review of the fundamental concepts involved may be useful here.

Grid operators have long familiarity with operational data from generation units, transmission and some distribution systems that provides the real-time status, performance, and loading of power system equipment – the fundamental information used by system operators to monitor and control the power system. Examples include circuit breaker status (open/closed), line current (in amperes) and bus voltages.

With the advent of digital intelligent electronic devices (IEDs), protection and control equipment also record non-operational data, of less importance to system operators, but the basis for value creation by enterprise groups such as engineering and maintenance.5 Examples include digital fault recorder records, circuit breaker contact wear indicators and dissolved gas/moisture content in transformer oil.

In terms of value, consider the importance of having current data on the health of a substation transformer. Failure to note diagnostic warning signs in a timely manner could result in a costly transformer failure, even a life-threatening explosion, which could take a substation out of service, affecting thousands of customers.

Three basic steps, and one doozy

A utility must first build a strong information and communication technology (ICT) foundation based on standards and open architecture, which integrates and ensures interoperability and backwards and forwards compatibility between devices, networks and databases. This requires cooperation between information technology (IT) and operations technology (OT) personnel, which represents a shift in traditional utility culture.

The next step is to identify and catalogue all sensors and systems and the data they produce, and the end users in the enterprise who can create value from that data. Every IED produces multiple data-producing points known as “data maps.” This leads to an inventory of IEDs and their data maps to help enterprise users understand the types of available data and who needs it. Careful documentation is required, because an IED’s “attributes” reflect that different vendors’ IEDs capture and render different non-operational data in different ways. For instance, data may be generated at a predetermined sampling rate, when a preset threshold is exceeded, or event-driven.

Matching data sources with end-users creates a “data requirements matrix,” which guides the network architecture that sends non-operational data across the corporate firewall into a data repository or warehouse that allows authorized end-users access on-demand.

The ultimate result of all these steps – holistic data management – is to make any and all operational and non-operational data available on-demand by operations and enterprise users who can use it to create value.6

Coming full circle

The value of establishing holistic data management for existing and, indeed, new data sources should be obvious. It is supported by the use of standards for interoperability and cost efficiencies, open architecture for simplicity and the shortest route from data capture to end use and value creation. As new data sources emerge, they can be integrated into a holistic data management approach. Without such an approach, a utility risks its ability to proactively meet marketplace challenges and, therefore, compromises its competitiveness.


1  McDonald, John D., “Embracing Holistic Data Management Prepares Utilities for the 21st Century: how power utilities can best manage data in the future, and be transformed,” TechCon North America 2018

2  McDonald, John D., “Drones and Robots: A Data Management Opportunity,” T&D World (29 November 2017)

3  Arghandeh, Reza and Zhou, Yuxun, Big Data Application in Power Systems, Elsevier, 2018; see Chapter 1, McDonald, John D., “A Holistic Approach to Becoming a Data-Driven Utility.”

4  McDonald, John D., “Integrated system, social media, improve grid reliability, customer satisfaction,” Electric Light & Power (Dec. 1, 2012)

5  McDonald, John D., “Extracting Value from Data: Distribution Automation and IEDs,” Electricity Today (May 2013)

6  McDonald, John D., “Managing Big Data: Challenges & Winning Strategies,” Electric Energy T&D (May/June 2014)

John McDonald is a T&D World Executive Insights Board member.

About the Author

John D. McDonald | SmartGrid Business Development Leader

John D. McDonald, P.E., is Smart Grid Business Development Leader for GE’s Grid Solutions business.  John has 45 years of experience in the electric utility industry. John joined GE on December 3, 2007 as General Manager, Marketing for GE Energy’s Transmission and Distribution business. In 2010 John accepted the new role of Director, Technical Strategy and Policy Development for GE Digital Energy. In January 2016 John assumed his present role with the integration of Alstom Grid and GE Digital Energy to form GE Grid Solutions.

John was elected to the Board of Governors of the IEEE-SA (Standards Association), focusing on long term IEEE Smart Grid standards strategy. John was the Chair of the Smart Grid Interoperability Panel (SGIP) Governing Board for 2010-2015 (end of 1Q) coordinating Smart Grid standards development in the US and global harmonization of the standards. John is a member of the NIST Smart Grid Advisory Committee and Chair of its Technical Subcommittee.

John is Past President of the IEEE Power & Energy Society (PES), Finance Committee Chair of the Smart Energy Consumer Collaborative (SECC) Board, the VP for Technical Activities for the US National Committee (USNC) of CIGRE, and the Past Chair of the IEEE PES Substations Committee. He was on the IEEE Board of Directors as the IEEE Division VII Director. John is a member of the Advisory Committee for the annual DistribuTECH Conference, on the Board of Directors and Executive Committee of the GridWise Alliance and Finance Chair, Vice Chair of the Texas A&M University Smart Grid Center Advisory Board, and member of the Purdue University Strategic Research Advisory Council. John received the 2009 Outstanding Electrical and Computer Engineer Award from Purdue University.

John teaches a Smart Grid course at the Georgia Institute of Technology, a Smart Grid course for GE, and substation automation, distribution SCADA and communications courses for various IEEE PES local chapters as an IEEE PES Distinguished Lecturer (since 1999). John has published 100 papers and articles in the areas of SCADA, SCADA/EMS, SCADA/DMS and communications, and is a registered Professional Engineer (Electrical) in California, Pennsylvania and Georgia.

John received his B.S.E.E. and M.S.E.E. (Power Engineering) degrees from Purdue University, and an M.B.A. (Finance) degree from the University of California-Berkeley. John is a member of Eta Kappa Nu (Electrical Engineering Honorary) and Tau Beta Pi (Engineering Honorary), a Life Fellow of IEEE (member for 48 years), and was awarded the IEEE Millennium Medal in 2000, the IEEE PES Excellence in Power Distribution Engineering Award in 2002, the IEEE PES Substations Committee Distinguished Service Award in 2003, the IEEE PES Meritorious Service Award in 2015, the 2015 CIGRE Distinguished Member Award and the 2015 CIGRE USNC Attwood Associate Award.

John has co-authored five books and has one US Patent: Automating a Distribution Cooperative from A to Z: A Primer on Employing Technology (National Rural Electric Cooperative Association – 1999); Electric Power Substations Engineering (Third Edition) (CRC Press – 2012); Power System SCADA and Smart Grids (CRC Press – 2015); Big Data Application in Power Systems (Elsevier - 2017); Smart Grids: Advanced Technologies and Solutions (Second Edition) (CRC Press – 2018); and US Patent (9,853,448) on Systems and Methods for Coordinating Electrical Network Optimization (December 26, 2017).

Voice your opinion!

To join the conversation, and become an exclusive member of T&D World, create an account today!