Madison Gas and Electric (MGE) is similar to most utilities in the area of information systems and applications development and support. Its technical support organizational structure in this area, which includes separate departments for information technology (IT), operations technology (OT) and operations support (OS), has evolved over 30 years. Although the departments are separate, MGE’s current implementation of a distribution management system/outage management system (DMS/OMS), which started in 2011 as part of the Department of Energy’s Smart Grid Investment Grant, was a catalyst that incented the three departments to work closely together.
Technical Support Structure
MGE’s IT department supports enterprise technology applications, including a customer information system (CIS) and enterprise resource planning, as well as newly implemented systems like advanced metering infrastructure (AMI) and interactive voice response (IVR). IT also is responsible for managing infrastructure and operation functions, such as telecommunications, mainframe and server administration, database administration, network administration and security.
OT is responsible for managing real-time transmission and distribution applications, such as supervisory control and data acquisition (SCADA), the related telecommunications network it uses and DMS/OMS currently being implemented.
OS is responsible for the development and support of distribution operations and engineering applications, including a geographic information system (GIS) and its Oracle database, a work management system, a maintenance management system, transformer load management (TLM), planning functions and mobile field systems.
While OT is responsible for the overall DMS/OMS system because of its tight integration with SCADA, the new OMS, which is fully integrated with the DMS, uses and generates information that is relatively broad in scope. This requires interfaces with customer information, IVR outage calls, and AMI outage notification and restoration messages.
For this reason, IT was tasked with supporting database and application servers, security services, and the interfaces with IVR and AMI. Furthermore, the advanced DMS applications, such as on-line operator power flow and fault location, isolation and service restoration (FLISR), depend on an accurate and timely updated transmission and distribution network model.
As a result, OS was tasked with supporting the development of the DMS/OMS network model, model validation and end-user support because of its experience with the GIS, network modeling and network analysis. OS had been responsible for supporting the legacy in-house-built outage management application. With the new OMS being a component of the integrated SCADA/DMS
system, that support is now shared with OT.
OS faced a challenge when it needed database support and wanted to align its staff with the organization’s core competencies. IT also faced the challenge of wanting to offer more support to OT and OS to more efficiently and effectively use internal resources. The DMS and OMS implementation served to magnify the challenges facing the OS department and made aligning resources and competencies even more critical.
OS approached IT with the proposition of transitioning the database support and some application support to IT, and IT responded positively to the proposal. After IT and OS agreed to proceed, this led to the initiation of the OS-IT database and application-migration project. The objectives of the project were to strengthen corporate alignment and synergies (to avoid silos), improve portfolio planning and management, leverage skilled resources, maintain information security, support business continuity and risk management, and manage and maintain system performance and support levels provided by OS. The project involved two steps: identifying and prioritizing applications to migrate.
The first step of the project was to identify which applications to migrate to IT. An inventory found more than 50 applications in the OS portfolio. The applications ranged from GIS, VB 6, ASP, .NET, SQL and batch files to Excel. The applications were grouped based on business value and technical requirements. The goal was to begin migrating applications that had low to medium business value and technical requirements to validate if the OS business requirements could be met. In addition, enterprise-like applications would be migrated because they would fit better in IT than OS. For example, this group contained applications that support a larger user base within the corporation versus a specific group within the operations area.
The second step of the project was to migrate applications in groups of low risk to high risk and by the migration effort. For example, the number of interfaces played a key role in the migration effort along with interfaces with similar applications. Applications with lower risk and lower migration effort were migrated first, while those with higher risk and more migration effort were scheduled later in the project. The decision was made to do an evaluation after the first group of applications were migrated to validate the business objectives were met. A go–no go decision would be made to either continue with the project or make adjustments to the plan.
The migration of each group of applications consisted of designing the end-state database architecture before beginning. Prior to migration, an application evaluation would be made to consider three functions:
- Converting the application from VB 6, ASP and Excel to .NET
- Redesigning to avoid any duplication of data only if the effort was low
- Minimal redesignof the table structure unless the application was being rewritten.
Any new functionality would need to be approved by a steering committee.
IT-OS Governance Model
To sustain business benefits and organizational and cultural change, a governance model was developed for the project and ongoing success after the project. The governance model was a joint effort between IT and OS. It was developed prior to the commencement of the migration project to begin convergence between IT and OS.
The purpose of the governance model was to provide two-way communications between IT and OS. It also was a way for IT to communicate the services it offers.
The roles and responsibilities of the governance model identified who or which area was responsible for different parts of the project and long-term roles:
- Steering committee was responsible for governance oversight of the project.
- Portfolio manager was responsible for application lifecycle from the business perspective.
- Resource owner was responsible for the accuracy and validity of the electronic information asset.
- Information security was responsible for ensuring corporate compliance with internal security policies.
- IT enterprise systems was responsible for the IT applications development and support.
- IT services was responsible for the help desk and infrastructure support.
The applications development and support area of the governance model identified what the planning process was for new systems, upgrades and maintenance. As new projects are identified, they are reviewed and prioritized on a quarterly basis. The selected projects are then scheduled and resources are assigned between IT and OS. Project execution includes the project management methodology, system development life cycle (SDLC) and development standards used for all new projects. It also identifies who will lead the change management, review any software contracts, and own the support and budget for any maintenance.
As part of this area, it was found OS may want to be part of the project prioritization process along with using the project management methodology, SDLC and development standards. IT agreed to share and train OS on the methodology and standards and to involve OS in the development or review of new and existing standards because of the similarities in their respective areas.
The infrastructure maintenance and support area of the governance model is responsible for managing hardware and network. This included system response time, database performance, system availability, support response, system monitoring, backup and recovery, maintenance, upgrades and business recovery programs. These will be established with service-level agreements between IT and OS, or at least by communicating the business expectations.
The security area of the governance model is responsible for managing MGE’s cyber security plan, which includes the security incident process, user access, configuration, reporting and monitoring, and segregation of duties. It is important that OS understands the corporate cyber security plan and attempts to adhere to the policies wherever possible. In cases where cyber security processes are not followed, a risk assessment will be completed and documented.
The incident management process is managed by MGE’s IT help desk. This is the central support area for all IT-related problems, requests and questions. OS decided to route all incident calls through the help desk and leverage this support area.
The change management area of the governance model manages and documents all changes to the corporate computing environment and follows the Sarbanes-Oxley requirements. Initially, OS did not have procedures or processes in this area. However, through better understanding of the existing IT procedures, OS understood the importance of leveraging the IT procedures to standardize and ensure corporate compliance.
A project like this may seem simple in nature. MGE has identified some key lessons learned that should be considered by any utility thinking about this type of convergence initiative:
- Strategic direction. Determine what makes sense for the entire organization rather than a single department. The IT, OT and OS departments should consider partnering together whenever possible. It is recommended to determine the roles and responsibilities of each technical area at the beginning of any new project.
- Integration. The smart grid is driving the need to integrate IT, OT and OS systems. With the tsunami of data, it is forcing the IT, OT and OS areas to work together to implement data warehouses and data analytic/business analytic tools to improve the value of information and help organizations to make key business decisions.
- Takes time. It takes baby steps to build trust and confidence. This type of project should be implemented in stages. At the end of each stage, review the lessons learned and make adjustments where needed. Validate the benefits and service levels.
- Time is money. Answer the following questions: Is this the right time? What is the return on investment (ROI)? How can resources and skill sets be optimized? It is recommended doing a cost-benefit analysis. Even if the ROI is not favorable, consider implementing a governance model to open the lines of communication between the technical areas. It will save time and money in the future.
- Resource planning. Many utilities are faced with an aging workforce. Consider optimizing resources and skill sets between IT, OT and OS. For example, consider having OT support the real-time applications and leverage IT for infrastructure support or enterprise applications. Leverage each area’s strengths and look for opportunities to consolidate or train other areas on best practices. For example, train OT and OS on IT’s SDLC, programming standards and project management methods. OT and OS can help IT to be more successful with integrating enterprise systems with OT and OS.
- Optimized work flows.Timely and accurate as-built (and as-operated) asset models are critical for DMS and OMS to improve system reliability and outage restoration time. It is important the departments responsible for these work processes understand the big picture and the impacts to other applications and work processes.
In summary, if a utility is considering consolidating some technical support functions, determine which functions and applications make sense to migrate to IT and the best approach to migrate them, whether in groups or all at once. Begin with the end in sight, but start small to ensure the business objectives are met.
A project like this can take time to begin. In any event, consider developing a governance model between IT, OT and OS even before an initiative like this begins. It will open the lines of communication between the technical support functional areas to begin achieving synergies enterprisewide. There is no one organization structure that fits all utilities. In any case, it requires IT to think more like OT and OS as well as OT and OS to think more like IT.
Tim Statz ([email protected]) is the manager of operations services at Madison Gas and Electric Co. He is responsible for GIS, the electric and gas distribution operations models, the utility’s reliability efforts, asset management, support for the software applications used by the operating areas, and underground gas and electric facilities location. Statz has more than 38 years of utility experience and holds a bachelor’s degree in computer science.
Jared Bushek ([email protected]m) is the director of information technology at Madison Gas and Electric Co. He is responsible for applications such as the customer information system, enterprise resource planning, advanced metering systems, and the management of IT infrastructure and operations functions. Bushek holds a MBA degree and bachelor’s degree in industrial engineering.
Lisa Hahn ([email protected]) is a principal advisor at Quanta Technology and the project manager for Madison Gas and Electric Co.’s Smart Grid Investment Grant distribution management system implementation. Hahn has more than 21 years of utility experience implementing strategic initiatives, including AMI, DMS and OMS systems. Prior to Quanta, she worked at Alliant Energy. She holds a MBA degree, a bachelor’s degree in computer science and a project management professional certification.
Madison Gas and Electric Co.| www.mge.com