By Rich Heidorn Jr.
A joint FERC-NERC review of nine unnamed utilities’ system restoration and recovery plans found them “for the most part … thorough and highly detailed” but also identified room for improvement and called for additional studies.
“The reviewed plans require identification and testing of black start resources, identification of primary and alternate cranking paths, and periodic training and drilling on the restoration process under a variety of outage scenarios,” the report said. “Likewise, the joint staff review team found that participants had extensive cyber security incident response and recovery plans for critical cyber assets covering the majority of the response and recovery stages.”
Staff from FERC, NERC and Regional Entities gathered information from “a representative sample of nine registered entities with significant bulk power grid responsibilities, including some entities that are registered with NERC in multiple functions.” The report emphasized that the staff review “was not a compliance or enforcement initiative.”
The report identified several opportunities for improving readiness through measures including improving the clarity of some NERC reliability standards.
It also took note of best practices used by some participants that went beyond NERC requirements, such as the inclusion of illustrations and step-by-step procedures in restoration plans and conducting drills that involve the actual transfer of control center operations to an alternate site. “The actual evacuation and verification of functionality of recovery resources can reveal unknown issues or problems through use of the alternate site’s cyber assets,” it said.
Recommended changes included:
- Clarification on when system changes will trigger a requirement to update restoration plans. “In considering these measures, the kinds of events that may warrant an update to the system restoration plan should be identified, taking into account the length of time the system is affected, as well as the overall objective of ensuring that restoration plans are generally flexible enough so that system modifications can be addressed without continuous updates.”
- Exercises and drills testing the transition from transmission operator island control to balancing authority area control error and automatic generation control.
- Cyber security incident response plans and recovery plans for critical cyber assets should designate accountability at the cyber asset level (e.g., energy management system (EMS) servers, remote terminal unit concentrators, network routers).
- More detail on the types of cyber security events that should trigger a response and reports. “While the team recognizes that [Critical Infrastructure Protection] version 5 will require responsible entities to have processes to identify cyber security incidents, consideration should be given as to whether any additional clarification or improvements are needed once some experience is gained with CIP version 5,” which takes effect for some assets on April 1.
- Expanding the use of cyber security technical expertise and advanced technical tools.
- Reducing the risk of recovery plan “inventory assumptions.” It said “entities may assume that hardware from external sources or other third-party vendor support needed for recovery of critical cyber assets will be available, without necessarily having measures to ensure availability. Likewise, entities may not consider interdependent or common-mode failure scenarios, which can create the need to recover multiple critical cyber assets concurrently from the same vendors.”
Among the studies recommended were:
- Assessing system restoration steps that may be difficult if operators lose supervisory control and data acquisition computer systems, inter-control center communications protocol or EMS functions.
- Identifying factors to be considered for replacing black start resources, including locational diversity and dual-fuel capability.
- Determining the benefits of including existing or future voltage source converter DC lines in restoration plans.
