By Holden Mann

Last year’s GridEx V security exercise provided the electricity industry with a number of lessons about crisis management that are proving timely in the current COVID-19 pandemic, according to NERC officials.

“One of the reasons we do exercises like this is so we are prepared to deal with significant challenges, and we are certainly in one now,” said NERC Senior Vice President and CEO of the E-ISAC Manny Cancel at a media briefing on NERC’s after-action report on the exercise. “And certainly, some of the lessons we learned from GridEx and the procedures that we practiced [have] prepared us for dealing with COVID-19, specifically the business continuity procedures that we and our entities have in place.”

NERC conducted GridEx V, the fifth of its biennial exercises, on Nov. 13-14, 2020, drawing more than 7,000 participants from more than 526 organizations across the industry and government — up from 6,000 participants and 450 organizations for GridEx IV. Organizations were asked to respond to a wide array of threat vectors representing what one PJM official called a “true doomsday scenario.” (See GridEx V Throws New Tech Curveball.)

Distributed Play Borrows from Life

The exercise had two major components. First was a distributed play model involving players from across the electricity industry, as well as government officials and representatives from interdependent industries such as natural gas, water, finance and telecommunications. The two-day exercise presented participants with a range of simulated challenges including social media hacks, vehicle fires at regional facilities, intruders in headquarter buildings and infections by malware.

In creating the distributed play scenario, planners drew heavily on lessons learned in previous exercises as well as from real-world crises. For example, the malware that attacked participants was patterned after the 2016 CrashOverride cyber-attack against Ukrainian utilities, a choice inspired by the malware’s targeting of industrial control systems (ICS) intended to cause chaos in the electric grid.

“When the exercise planners … were putting this together, the ICS component of CrashOverride in 2016 provided a lot of great training value to the industry,” said Matthew Duncan, senior manager of resilience and policy coordination for E-ISAC. “We had very good feedback from the utilities, seeing this malware that they could [respond to] in an exercise environment and really test their defense capabilities against it.”

Executive Tabletop Brings Needed Focus

The distributed play model was accompanied by an executive tabletop session with participation by leaders of electric, natural gas and telecommunications industries, along with senior government officials. While this has been a feature of previous exercises, this year the executive tabletop featured, for the first time, a separate scenario from the distributed play. In addition, unlike in previous events, the scenario was focused on a regional rather than a national threat, which organizers hoped would create more useful response data.

“In previous GridExes having the entire nation and continent under attack made it difficult to get down to the technical detail necessary [to model an effective response],” Duncan said. “By picking the Northeast, New York State and Southern Ontario, we were able to get sufficient detail to play this out, and it was very helpful.”

Recommendations from the executive tabletop included the following:

• Ensure grid emergency response and restoration plans describe coordination with federal and state or provincial authorities in the event of a national security emergency;
• Incorporate natural gas providers and pipeline operators into restoration planning and drills;
• Enhance coordination with communications providers to support restoration and recovery and work to ensure the 6-GHz spectrum communications band remains open to utilities in emergencies;
• Build consensus with the Department of Energy on procedures and requirements for issuing grid security emergency orders;
• Identify key supply chain elements and ensure inventory can be shared in a crisis;
• Expand participation in the Electricity Subsector Coordinating Council (ESCC) cyber mutual assistance (CMA) program; and
• Strengthen industry and government coordination between the United States and Canada.

Supply Chain Participation Falls Short

NERC set seven objectives for GridEx. The organization reported that six of these were fully achieved: practice incident response plans, expand local and regional response, engage interdependent industries, improve communication, engage senior leadership and gather lessons learned.

The seventh objective, increase supply chain participation, was only partially achieved. NERC had hoped to expand engagement with the vendor supply chain in GridEx V after calling out utility operators in GridEx IV for failing to recognize the importance of maintaining vendor support. (See Ukraine Attacks, ‘Fake News’ Color NERC GridEx IV Drill.) However, only three major electric industry supply chain vendors officially registered for the most recent exercise. While more vendors may have participated unofficially, NERC said that organizations must work harder to include these critical industry players in their response plans.

Organizations are already planning for GridEx VI, scheduled for Nov. 16-17, 2021. MISO has begun selecting internal committees to design scenarios and lead simulations for the next exercise. (See MISO Preps for GridEx VI.) Duncan said future GridEx planning will continue to draw from current events such as the COVID-19 pandemic to ensure that the industry is as ready as possible.

“We say that GridEx is designed to overwhelm even the most prepared utilities, and we use realistic scenarios to keep making the industry better,” Duncan said. “There’s no shortage of threats out there, but the only way we’re going to get better at mitigating and defending against those threats is [by] practicing against those threats, and that’s what GridEx is all about.”