CN Utility Consulting IEEE reliability metrics

The Measure of UVM

May 16, 2016
Overemphasis on IEEE reliability metrics diminishes public safety and fire protection efforts.

A decade ago, experts were encouraging utility executives to realign their vegetation management programs with measurable reliability benefits. It was suggested this realignment would result in lower costs and bring utility vegetation management (UVM) out of the industrial age and into the technological realities of modern business management. The most significant and clearest measure of reliability improvement has been with extra-high-voltage transmission, because the internationally mandated FAC-003 standard spurred utilities to achieve specific results. FAC-003 affected transmission lines, which account for about 3% of the total miles of overhead power lines in the U.S., but what about the other 97% of miles of overhead T&D electric lines in the U.S.?

Although the bright light of business models, including economies of scale and process management, has been shined into the voids of UVM performance, there is no consensus that the industry has given birth to the golden age of utility arboriculture. Young arborists, some freshly trained on UVM, are still put to the daily task of working around lethal conductors engulfed in vegetation. Utility arborists, scolded by the public for being butchers, continue to wonder why the industry does things the way it does them. Every year, a few go to work and never return. Many more tree workers lose their lives or are injured because they made contact with electrical energy, often not knowing the wires were there.

It is worth reviewing what the industry does for reliability with regard to vegetation management and the assumptions behind it, if only to reduce the risk to human life.

Reliability-Centered UVM

Studies conducted to evaluate how UVM has conformed to reliability data revealed several interesting findings:

• Reliability-centered maintenance (RCM) applied to UVM may shift focus away from other objectives, such as safety, fires, environment and the customer.

• When UVM programs are prioritized to improve reliability metrics, long-term UVM workloads are likely to increase.

• An increase in tree-related outages can occur at the same time reliability is improving because reliability metrics are relative (see chart 1).

The need to apply computer-enabled technology to UVM has coincided with the adoption of IEEE 1366. The interruption cost estimate calculator designed by the U.S. Department of Energy reinforces the nexus between data-driven UVM and a burgeoning growth of reliability data. It would be imprudent to discourage actions that prevent outages that would affect large electric loads. However, it also is important to recognize the priorities of electric system reliability are not the only motivation for performing UVM.

The consequences of applying electric system reliability metrics to UVM should be fully vetted. Is it acceptable that RCM encourages UVM policies that increase the frequency of outages? Improvements for relative reliability and absolute reliability are two separate outcomes, both of which require adequate planning and resources to achieve.

Chart 1. SAIFI improves while UVM outages increase.

Case Study of Reliability Metrics

The following is a comparison study between the system average interruption frequency index (SAIFI), the system average interruption duration index (SAIDI) and tree-related outages per pole mile between three companies (X, Y and Z). 

The UVM programs for these utilities represent a RCM program (utility X), a compliance-based program (utility Y) and a utility (utility Z) that had favorable reliability performance but was dissatisfied with its UVM program. All three companies were excellent performers based on SAIDI and SAIFI (see charts 2 and 3). Utility Z initiated the study with CN Utility Consulting by asking the question, “How is it possible we achieve best-in-class reliability when we know the UVM program is not meeting best management practices?”

By comparing the metrics from the three companies within a field of other companies, CN Utility Consulting found several revealing facts that demonstrated the limitations of using reliability metrics as a measure for the standard of care for UVM.

Utility X almost doubled the number of tree-related outages on its distribution system in five years (2008–2012) and still showed significant improvements in SAIFI (31%) for the same years (see chart 1). By focusing UVM efforts on high-customer-density feeder lines while deferring maintenance on single-phase lines with lower customer density, X was able to report almost best-in-class metrics in 2012. An increase in SAIFI in 2011 could have been an anomalous year given the return to dropping SAIFI values in 2012. Subsequent years, 2013 and 2014, suggest something else is happening. Potentially, there is an upper limit of tree-related outages at which point SAIFI also will increase.

Another interesting aspect of the reliability metrics occurs when SAIFI increases faster than SAIDI: The customer average interruption duration index (CAIDI) actually improves in spite of overall worsening of the other metrics. This is another example of how reliability metrics might be misleading. A utility could show exemplary reliability — in comparison to industry averages — by only focusing on non-major events, even while vegetation-related SAIDI, SAIFI and outages are increasing.

Outstanding reliability is a product of interpreting statistics and focusing on the bigger picture of utility reliability, which includes other failure modes such as equipment failures and generic categories such as non-major weather events. If asset improvements are made — such as replacing equipment, adding isolating fuses, monitoring with sensors and making equipment more resistant to weather, vegetation and animals — then reliability will improve and vegetation appears to be under better control. In reality, it is the asset that is improved not the vegetation management.

Chart 2. Utilities X, Y and Z have different UVM strategies but similar SAIFI.
Chart 3. Company Y has slightly higher SAIDI but program is focused on bigger picture.

In a comparison of reliability indices for the three utilities of interest (X, Y and Z), they all show excellent metrics when compared with the UVM industry in general. If the charts only included peer utilities (with similar key characteristics), all three utilities are best in class among their comparators. These two charts also show X and Z, the utilities with an RCM program, compare well with Y, the utility with a compliance-based program. Of note, a majority of the other utilities in the first quartile have cycle-mandated programs (regulatory-based programs).

As previously noted, utility X is able to achieve high-reliability performance as measured by SAIFI while the absolute number of outages has increased 143% over a nine-year period. Utility Z knew there was something misleading about its best-in-class tree-related reliability metrics, because the increases in the percent of trees in contact with power lines and the increases in the percent of reactive maintenance were not represented by these measurements.

What was unclear to Utility Z was the relative value of SAIFI and SAIDI. In other words, IEEE metrics are customer-density dependent and UVM is tree-density dependent. When comparing utilities X, Y and Z with peers using an absolute rate (outages per mile), a different picture emerges (see chart 4). Utility Y, which is focused on regulatory mandates, achieves similar SAIFI/SAIDI but a much better outage prevention performance than the RCM utility, X, while meeting other objectives as well. These objectives include public and worker safety, fire risk reduction and environmental quality, which results in best-in-class reliability measured as an absolute rate.

Chart 4. Company X prioritizes for SAIDI/SAIFI improvements, Y prioritizes by clearance distances, Z now knows the difference.

UVM Objectives

Reliability is a major objective, especially as the industry enters into an era when climate changes are a significant threat to utility operations. However, effective vegetation management is not a direct response to conditions that negatively impact reliability. The reliability metrics that currently guide vegetation management do not measure or recognize the full extent of the UVM workload or forestry conditions. It is not the incremental tree growth that impacts reliability but rather the accumulative growth that leads to branch and tree failure. Nonetheless, it is still growth UVM must control to prevent interruptions. This growth component must be managed in a cost-effective way, and UVM must be performed to satisfy other objectives besides reliability.

The use of reliability metrics as a measurement of UVM efficacy is pressuring utilities to practice reliability-measured UVM rather than sustainable UVM that adheres to principles of forestry and urban arboriculture. All vegetation near power lines must be managed at some point, regardless of its impact on reliability. Delaying maintenance to serve improvements to reliability metrics compromises all of the objectives and creates a greater long-term risk. This research was performed to demonstrate the shortcomings of RCM and to offer some alternatives to steer UVM in the direction of a more balanced approach that includes multiple objectives.

William Porter is a director of consulting services at CN Utility Consulting, a company that offers UVM consulting and field services. Porter has more than 20 years of experience in the UVM industry and has led consulting projects and benchmark studies throughout the U.S. and Canada. He has focused his research on legal, program development and comparative studies.

Sidebar: Beyond the Reliability-Centered Model

The following discussion demonstrates the theory, logic and empirical evidence the industry should be supporting utility vegetation management (UVM) beyond the current reliability-centered maintenance (RCM).Theory In theory, UVM programs operate on a set of objectives established to support a mission and vision for a utility. These objectives are collectively the driver and benchmark on which the program is measured and implemented in a continuous spiral of improvements. Objectives provide the theoretical framework behind UVM. Without such a framework, UVM would be reactions to system failures and liability. The cost alone of reactive UVM has driven vegetation management programs to become more preventive. To be preventive, one must have a plan based on a theory of objectives, strategies and results.LogicOnce the theory behind UVM is established, then logical steps are taken to achieve the framework of objectives. Theory may be the lodestar, but logic is the navigation system and it has to adjust to stay on course. For example, a UVM program must show a return on investment, but what logic gets a utility there? UVM has sought cost-effective ways to manage high risk, but to maximize a return on investment, it has bypassed managing smaller risks and made compromises and adjustments. This strategy has enabled UVM departments to show success by using IEEE reliability metrics. Managers can report success if the system average interruption duration index (SAIDI) and system average interruption frequency index (SAIFI) are reduced. For upper management, this provides a reduced cost burden achieved through reliability improvements. Unfortunately, the marriage of UVM risk and reliability metrics also has meant performance for other objectives and risks could be reduced, ignored or transferred away from UVM departments. Worker safety liability has been shifted to contractors. Reliability performance shows UVM is performing what is reasonable and shields utilities from public liabilities. State regulatory compliance validates this idea by requiring reliability metrics. Other objectives like fire prevention, environmental quality and customer service have become weak objectives.Regardless of relative ranking, each objective should have key performance indicators. Strategies and tactics should be formulated to support measureable achievements and improvements. Some objectives, such as reliability and safety, may be more important, but a program ought to be balanced. A single objective such as reliability should not be the only characteristic of quality service. For example, UVM, from a customer perspective, is an inconvenient intrusion on private property where vegetation is negatively altered to complete an electric system correction. Asset corrections protect equipment and ensure reliability, which is misconstrued as the primary — if not the only — element of customer service. Why is UVM not considered a customer service activity in which customers perceive they are receiving a benefit to their property or vegetation and to their community? Perhaps utilities have come to accept adversarial relationships with rate-paying customers.Empirical EvidenceWhile logic is necessary to navigate, utilities need empirical evidence — an analysis of readings from their instrument cluster — to support and measure the performance of the industry’s logic. The following four sets of data are readings from the current era of UVM, and they suggest a need to change the industry’s theoretic lodestar or reevaluate the logic used to navigate UVM programs:Reliability is a UVM problem, but RCM is an asset management strategy. RCM, from an asset perspective, is designed to address the majority of failure modes that are likely to occur during normal operating conditions. RCM is integral to the life cycle of assets, which includes constructing, maintaining, extending and replacing. Since abnormal conditions are usually an event such as weather, IEEE devised a statistical method to exclude certain outages in reliability performance metrics. The IEEE 1366-2012 standard defines a major event as one "that exceeds reasonable design and/or operational limits of the electric power system. A major event includes at least one major event day (MED)."A T-MED is a calculated threshold for excluding outages in reliability metrics. It provides a limit beyond which a utility should not be expected to perform within expected reliability parameters. Berkeley National Laboratory (BNL) has performed cumulative statistical research on utility reliability data collected over the past 13 years. Some specific variables are associated with measureable improvement in SAIDI and SAIFI. T&D expenditures, percent of underground, hot weather and increases in customer density were found to correlate with lower SAIDI and SAIFI. Other variables such as wind are associated with increases in SAIDI and SAIFI. Although the BNL research does not differentiate wind-caused outages between tree and equipment failures, CN Utility Consulting benchmark surveys have found at least 43% of outages during MEDs are tree related. BNL studied wind-event data, including and excluding MED. If MEDs are included, a 10% increase in average annual wind speed is correlated with a 75% increase in SAIDI. In contrast, if MEDs are excluded, a 10% increase in average annual wind speed is correlated with only a 2% increase in SAIDI. This data analysis is significant to UVM since the majority of tree-related outages are caused by wind and other loading events. Without actionable knowledge of the life cycle of trees and their failure modes, and without a reliability system designed around the effective management of rights-of-way land and nearby trees, RCM is not likely to address tree-related reliability nearly as well as it can guide asset management.Reliability performance is not measured the same from one utility to the next. The CN Utility Consulting benchmark surveys found 88% of responding utilities track tree-related SAIFI, SAIDI and CAIDI, and 74% of utilities are using this information to make planning and resourcing decisions for the UVM department. Additionally, 59% of utilities have not strictly used the IEEE 1366 guidelines for separating MEDs from non-MEDs. In fact, the 1366-2012 revisions left the door open for utilities to adopt their own specific definition of a catastrophic storm. Some utilities may have to comply with a state commission definition. Differences in MED definitions and thresholds may effectively increase or decrease the number of events that become MED and subsequently influence the threshold for determining MED. When one major event is excluded, the average used to determine the T-MED also is lowered, which causes additional events to be classified as major. Consequently, the measurement of reliability performance is inconsistent and possibly misleading, because SAIFI and SAIDI numbers vary depending on whether outage events are included or excluded.Reliability is worsening, according to SAIDI. Despite the reported improvements to non-MED SAIDI, the most current research into reliability data indicates distribution system SAIDI measurements are worsening by 10% annually over the 13-year period of study. In recent years, major storm events have become a critical reliability issue, and tree-related outages are the chief contributor. A comparison of the ratios of tree-related to system SAIFI revealed non-MED tree-related SAIFI was, on average, 24% of system non-MED SAIFI. In contrast, MED tree-related SAIFI was 43% of the system MED SAIFI. This difference underscores the significance of the SAIFI amplitude vegetation causes during MEDs compared to other distribution system failure modes. The risk factor for MEDs is greater for UVM than it is for equipment failure in the absence of tree damage. The Larsen study also found a 10% decrease in precipitation is correlated with a 3% increase in SAIFI. It has been thought drought conditions may contribute to tree-related outages.
Above-average wind speed and duration of interruptions (SAIDI) (Larsen, 2015)

Voice your opinion!

To join the conversation, and become an exclusive member of T&D World, create an account today!