FERC accepted a settlement between New York State Electric and Gas (NYSEG) and the Northeast Power Coordinating Council in which the utility admitted to violating NERC’s requirements for maintaining backup control centers.

The settlement, which carries no monetary penalty, was filed by NERC in its monthly spreadsheet Notice of Penalty on July 31; it was the only settlement in the spreadsheet and the only NOP filed that month (NP24-10). In a filing issued Aug. 30, FERC said it would not further review the settlement. Commissioner Judy Chang did not participate in the decision.

The settlement stemmed from a violation of EOP-008-2 (Loss of control center functionality), approved by FERC in 2018 in order to “ensure continued reliable operations of the [electric grid] in the event that a control center becomes inoperable.” NPCC discovered the noncompliance during an audit in 2020.

According to the settlement, NPCC found that NYSEG’s backup and primary control centers used a shared communication path with a single point of failure. This contravened requirement R6 of the standard, which mandates that reliability coordinators, balancing authorities and transmission operators ensure their primary and backup control centers maintain separate functionalities.

NPCC reported that seven communications lines terminated in a single room common to both the primary and backup control centers. In the event of a “catastrophic event” at the primary control center, the utility would lose its connection with about 150 remote terminal units (RTUs), 62 of which provide data from its substations. This represents a loss of data from more than half of its 121 grid-connected RTUs.

Further investigation revealed that NYSEG had discovered the issue during a prior audit in 2017 and labeled it an area of concern. The utility first sought to address the problem with its telecommunications vendor, but the vendor delayed implementation of the proposed solution for more than a year before telling NYSEG in 2019 that it “could no longer support the solution as designed.”

NYSEG then pursued a permanent solution, which was “in the planning stages” when NPCC conducted its 2020 audit. But the regional entity said the utility did not assign the task the necessary priority or management oversight, and thus the violation lasted longer than it would have with proper prioritization. Along with EOP-008-2, NPCC also found that NYSEG had violated the standard’s predecessor, EOP-008-1, which was in effect when NYSEG registered as a transmission operator and was required to comply with it.

NPCC assessed the violation as a moderate risk to grid reliability. It pointed out that the shared point of failure would have reduced NYSEG’s visibility into its system and compromised its ability to work remotely if the primary control center became inoperable. The RE said a catastrophic event compromising the primary center “would likely be a long-duration event,” exacerbating the risk.

At the same time, the RE acknowledged that the risk of such a catastrophic event affecting the primary control center is low. It also pointed out that even if NYSEG lost its ability to monitor the system, NYISO and neighboring TOPs and BAs could still monitor their respective systems, ensuring some visibility into the grid’s health.

NPCC determined that no monetary penalty would be required in light of NYSEG’s cooperation in the enforcement process, lack of prior relevant noncompliance and agreement to settle the matter rather than calling for a hearing. However, the RE did feel it necessary to elevate the matter to the spreadsheet NOP because of the length of the noncompliance and the fact that it became aware of the issue through a compliance audit rather than the utility reporting the problem itself.

To mitigate the problem, NYSEG removed the single point of failure by migrating the communications lines. It also created a new NERC compliance tool to monitor compliance projects and make sure schedules are maintained properly, trained relevant personnel on the tool, and updated its project management procedures to specify that leadership must review the project management plan when changes to a project’s schedule are needed.