By Holden Mann

For Associated Electric Cooperative Inc. (AECI), the results of a 2016 audit for compliance with reliability standard FAC-008-3 came as a shock.

The utility had assumed it was complying with the standard, which governs the determination of transmission facility ratings; instead, auditors discovered that the software it used to calculate ratings was missing a number of key pieces of information, resulting in widespread inaccuracies in its facility ratings.

“We came into it [thinking] we were living in happy-camper-ville, and came to find out we were not there,” James Vermillion, senior transmission planning engineer with AECI, told SERC Reliability’s Spring Compliance Seminar this week in Charlotte, N.C. “[We] wound up with a very concerned attitude about it [and] did develop a mitigation plan to correct the situation. So … over the course of a year, we went out … to inspect and field-verify 25% of the system each quarter.”

Physically inspecting the facilities was only the beginning of AECI’s mitigation efforts; more important was the work the company spent over the next three years overhauling its rating methodology and associated computer systems. Among other things, AECI installed a new software layer to ensure that any changes in relevant data are flagged so that engineers can ensure the central database has been updated accordingly, along with updating its procedures to ensure all personnel know how to use the new system and that its reliability coordinator has full access as well.

The changes, amounting to 13 new internal controls coordinating the efforts of 11 companies and departments, paid off in 2019 when SERC re-audited AECI in 2019 and gave the company a “clean bill of health.” AECI has continued to implement changes since its second audit. Work recently began on expanding the company’s data collection to include 7,000 miles of 69-kV lines.

LUS Earns Praise with Comprehensive Approach

Vermillion was one of several presenters at the SERC gathering who stressed the importance of internal controls in preventing violations of NERC reliability standards, or in detecting and eliminating ongoing infractions. A very different audit experience was recounted by Andrew Ledoux, an electrical engineer at Lafayette Utility System (LUS) in Louisiana, which joined SERC in 2018 after the dissolution of the SPP Regional Entity. (See FERC Approves Dissolution of SPP RE.)

Although LUS’ geographical footprint is relatively small, primarily covering only the city of Lafayette, the utility is registered as a balancing authority; transmission owner, operator and planner; generator owner and operator; and distribution provider. As a result, the entity is subject to enforcement of 77 NERC standards, and the compliance burden can be higher than expected, as Ledoux found out when he began to review the utility’s systems ahead of its first audit by SERC for violations of critical infrastructure protection (CIP) standards.

“About a year and a half ago, I didn’t know what an internal control was,” Ledoux said. “So it took a little bit of time to research … and what we found [is] that all this is is just a reminder that you need to do something for compliance.”

Ledoux and his team built up the utility’s expertise quickly, with a complete set of CIP internal controls in place by March of last year. The system included multiple gatekeepers — individuals responsible for monitoring open tickets and ensuring they are completed before the due date — and periodic reviews of policies and procedures to evaluate their real-world impacts. When the CIP audit team inspected the company in November 2019, auditors found no potential violations and no areas of concern.

Striking Control System Balance

One of Ledoux’s biggest struggles was with resisting the urge to go overboard with LUS’ control schemes. The company had to find the right mix of preventive measures, to ensure that violations never arose in the first place; detective controls, to find infractions once they occurred; and corrective actions, to stop ongoing issues and repair any damage that has occurred. To make the most of limited manpower and resources, the company decided to base the plan around preventative controls, complemented by a small amount of detective measures.

“Part of the problem I was having was getting too bogged down with … all these controls,” Ledoux said. “Just focus on preventative, and you can always expand later, if you want to go that route.”

Justin Kelly, senior CIP auditor at SERC, provided a different perspective. Kelly emphasized that while 100% compliance is always ideal, utilities cannot realistically expect to hit this target all the time — and do not, in fact, do so, judging by the number of infractions his team has identified. He suggested that when utilities assume that violations will occur despite their best efforts, they can pursue a more comprehensive mitigation strategy.

“I agree with Andrew, to a point, that preventative is the first way to go, because there’s a lot of things that you can catch through getting yourself a nice base of preventative,” Kelly said. “But really the next step to maturity is [through] detective [controls], because once you do the detective [work], you’re going to catch the things that you never thought would happen.”