By Holden Mann

A simulated social media hack was among the surprises lobbed at participants in GridEx V, the latest entry in NERC’s series of exercises testing industry preparedness for cyber and physical attacks.

More than 425 organizations across industry and government participated in the two-day exercise, which began on Wednesday with a distributed play model representing a wide array of threat vectors that Steve McElwee, PJM’s chief information security officer, called a “true doomsday scenario.”

Along with utility companies and regulators, the drill included representatives from farther-flung sectors, such as natural gas, electrical equipment manufacturing, telecommunications and even finance, in an attempt to game out the broader social impacts of an attack on the shared electrical grid.

Stacking the Deck

“One of the important design parameters that we use when we develop GridEx is we essentially break the system,” NERC CEO Jim Robb said in a media briefing Thursday. “That’s how the electricity industry learns: We break things, and then we figure out how to fix them and prevent the breakage from happening next time. So, it’s purposefully an overwhelming act of violence.”

This year’s challenges included the takeover of one utility’s Twitter account by malicious hackers that then used it to spread disinformation to the public and other participants, which one player described as the major “curveball” of the scenario. Additional threats included technological incursions such as the use of rogue USB devices and ransomware, which — along with physical attacks such as intruders in headquarters buildings and vehicle fires at regional facilities — put essential infrastructure out of commission. Utilities were tested both on their ability to handle the initial attacks and their capacity to ride out the damage and get their systems back online.

The distributed play exercise was joined in its second day by a similarly comprehensive but more targeted scenario in Thursday’s executive tabletop session, which presented an attack on the northeastern part of the North American grid. Test designers decided on this scenario, the first region-specific exercise in the history of GridEx, in hopes of gaining deeper insights than were available in previous years. The northeastern setting gave participants the opportunity to explore characteristics of the region such as U.S.-Canada relations, the interdependence of the electric and natural gas sectors, and the impact of a prolonged outage on financial players in New York City.

“There are very few cyber-only or physical-only incidents, and as our world grows more interconnected and our infrastructure grows more interdependent with other systems and functions, we must look at our risks [from] both a physical and cyber perspective,” said Brian Harrell, assistant director of the Cybersecurity and Infrastructure Security Agency at the Department of Homeland Security. “The scenario is real, it’s relevant, and it focuses on industry and government partnerships and how we [can] collectively get better.”

Fighting Back

As Harrell suggested, risk is not the sole focus of GridEx. The scenario also provides a sandbox for the public and private sector to test mitigation tools without danger to the general public. This year’s scenario was no different, with participants aiming to address vulnerabilities identified in previous GridEx iterations.

One focus for industry players in this year’s scenario was to actively engage with the vendor supply chain. Vulnerabilities often center on specific equipment, yet in the public report following GridEx IV, NERC called out utility operators for failing to engage with vendors to the degree they did with other utilities, government, and law enforcement. (See Ukraine Attacks, ‘Fake News’ Color NERC GridEx IV Drill.) The criticism spurred greater efforts in this year’s exercise, though participants acknowledged that considerable work is still needed.

“The supply chain issue is extraordinarily complex and hard to think about over time, because the threat vectors change continuously and … a good device today may be exposed tomorrow,” said Southern Co. CEO Tom Fanning, co-chair of the Electricity Subsector Coordinating Council. “So, it isn’t [enough to] have certified equipment in our supply chain. … We must have a process of cyber hygiene and collaboration over time.”

On the public side, GridEx V provided a chance to test out the responsibilities granted to the Department of Energy since the last exercise under the FAST Act, amended in 2018 to designate the department as the lead agency on cybersecurity for the energy sector. The change gave broad new authority to DOE to coordinate with state and local governments, in addition to utilities, and GridEx provided an opportunity to test the practical limits of these powers prior to a real emergency.

“What we don’t want … is to be in an actual situation where we’re figuring out the right policies and how we share that information, and what type of information [to share], so that we can have the situational awareness to advise the president,” said Karen Evans, assistant secretary in DOE’s Office of Cybersecurity, Energy Security and Emergency Response.

Ongoing Development

The GridEx exercises have expanded considerably since the first iteration in 2011, which involved just 75 industry and government organizations across the U.S. and Canada. Unlike that scenario, which was inspired by the Stuxnet attack in Iran and focused exclusively on cybersecurity, GridEx now aims to include the widest possible range of participants so that every aspect of the system can be tested.

This has led to criticism that the scenarios presented are unrealistic, with participants in previous years comparing the prepared situations to a “disaster movie” rather than helpful practice for recovery. NERC acknowledged these issues but said they overlook the true goal of the exercise.

“The grid is designed with a tremendous amount of redundancy, it operates in real time, and the loss of even a major power station in many cases is not a catastrophic consequence because the industry is prepared for that and designs around it,” Robb said.

“That makes a scenario [such as the one] we’ve laid out implausible but still worth testing,” he added, citing the potential to uncover unsuspected vulnerabilities and suggest new avenues of cooperation.

NERC will release its report on GridEx V by March 2020.

RTOs Take Part

RTO officials also gave their take on the exercise Thursday.

Keri Glitch, MISO’s vice president and chief information security officer, said the scenarios included “network breaches caused by an internal source, a potential intruder in the headquarters building, as well as a vehicle fire near a regional facility.”

“Our employees and industry partners collaborated well and learned a lot from the drill,” Glitch said.

About 120 CAISO employees took part in the exercise, along with representatives from federal, state and local agencies and 39 RC West participants, IT Enterprise Support and Campus Operations Director Matt Turner said.

“We assessed how employees reacted and communicated the scenario injects, which included a plan to return to normal operations. During the simulation, we injected additional issues, such as making key personnel unavailable, to evaluate the depth we have on the team and their ability to adapt to the situation,” Turner said. “Our exercise is designed to push the limits, as far as we could, to identify areas for improvement.”

SPP said more than 200 staffers took part, after more than a year of preparation by the RTO’s leadership team. “SPP’s incident coordination team led IT, operations and other staff in response to simulated threats to system reliability, communications channels and cyber assets, all in the interest of strengthening defenses, enhancing resilience and refining emergency response procedures,” spokesman Derek Wingfield said. “In the weeks leading up to the go-live of our Western reliability coordination service, GridEx also gave us the opportunity to test our preparedness alongside some of our new customers in the Western Interconnection.”

NYISO, ISO-NE and ERCOT also confirmed their participation. “Physical and cybersecurity measures are a constant practice of vigilance and focus of attention,” NYISO CEO Rich Dewey said.

“Past GridEx exercises have proven to be valuable training opportunities for many departments within ISO New England, and we look forward to practicing and improving our response capabilities,” RTO spokesman Matthew Kakley said.

While PJM regularly conducts simulator drills with its transmission owners and other critical players, GridEx allows the RTO to test its operations under extreme conditions, McElwee said. “It’s far beyond any situation we’ve experienced.”

Amanda Durish Cook, Tom Kleckner, Michael Kuser, Hudson Sangree and Christen Smith contributed to this article.