Supply chain rules from NERC and the federal government are increasing costs and procurement cycles for utilities and technology vendors, cybersecurity experts said yesterday.

The recent cyber breach of SolarWinds’ Orion product, which gave Russian hackers access to multiple federal agencies, “really is a wakeup call,” Tom McDonnell, power generation and energy industry leader at Rockwell Automation, said during a webinar sponsored by POWERGEN International. “That vendor-regulated [entity] relationship has to be a lot tighter than before.”

But McDonnell had a plea to his fellow panelists, who were from American Electric Power and NERC. “The one thing we ask is, don’t overcomplicate things for vendors. … Clear communication and common sense are really critical.”

He said he feared the electric industry will face the kind of overkill found in some Food and Drug Administration regulations. “The joke that we would always make in that space is you create 8 pounds of paper for 1 pound of drug.”

Jeffrey Sweet, director of security assessments for AEP, said the utility’s costs and workload have increased as a result of supply chain requirements from NERC standards, presidential executive order 13920 and Section 889 of the National Defense Authorization Act.

“It’s increasing the need for us to assess our vendors and the [security] of our products and services,” Sweet said. “Because of the increased assessment time, it takes longer for us to get through the purchasing process.”

Sweet said the SolarWinds breach could affect utilities. “It very possibly can, based on what I understand and what the investigations have turned out so far. … The code base for SolarWinds, certain versions, was in fact compromised. … Some entities have claimed that they have actually seen callouts going from their SolarWinds to some command-and-control centers. So please, definitely check your environment and make sure you don’t have those versions of SolarWinds.”

Howard Gugel, NERC’s vice president of standards and engineering, discussed the organization’s supply chain work to date and several issues it will confront in the future, including gaining an understanding of interactions between the bulk electric system and behind-the-meter generation and other distributed energy resources, referring to them as “the great unknown.”

He also said system planners must eliminate siloed thinking. “We’ve planned the system just thinking about physical assets, and then the IT issues would be left to the IT folks. I think as we go into the future, we’re going to have to get those two groups talking much more together and ensuring that as we plan the system, we think about the cyber impacts on IT; and then as we begin to roll out the connectivity of things in the future, that they link back into the planners and make sure that there’s a good handshake that occurs there.”

Gugel also cited issues over virtualization and cloud computing. “We’re beginning to tackle that right now with our cyber standards team looking at those issues. How do you implement that? How do you practically roll that out in the field?”

Sharing Assessments

Sweet noted the need for continuous monitoring of vendors.

“Just because everything was good when you first assessed them doesn’t mean it stays good for the rest of the term of that contract,” he said. “Many of our contracts may be three or five years or even longer. … Things change. How are they conducting the business? Who’s influencing their business? Have they moved operations overseas, or is there another company that’s purchased their operations? … Even if the ownership doesn’t change, things change within a company. And so, the policies and standards that … they had in place may have changed, and now they may not be as effective as they once were.”

McDonnell said Rockwell, a multinational manufacturer and technology and solutions company, is “constantly changing where we manufacture things. … We’ve got to have that relationship with the vendor that is a very open and transparent relationship that you have to revisit on a timely basis.”

AEP joined with Fortress Security in 2019 to create the Asset to Vendor Network to reduce the costs of assessing vendors. The network now also includes Southern Co., Hitachi ABB and NiSource. (See CIP Compliance: Don’t ‘Boil the Ocean.’)

“We’ve matured our program, and now we’re trying to help the rest of the industry by providing them a lower cost of getting that assessment data, including the foreign ownership control and investment entities; the provenance reports and stuff of that nature,” Sweet said. “We’re trying to get that out there so that even a small utility can afford to have that.”

Impact on Competition?

Moderator Scott Affelt, president of XMPLR Energy, asked whether the supply chain rules could reduce competition by forcing some vendors out of business.

“If the vendor is actually doing what we’re asking them to do and shows us they’re doing it, then it won’t have an impact,” Sweet said. “But if the vendor refuses to comply with the standards or meet the requirements of the standards, then they’re probably going to get put to the side, at least by those who are regulated.”

Gugel said if some vendors exit the business, others will likely rush to fill the vacuum.

As for the costs of compliance? “Bearing an appropriate amount of cost for an appropriate reduction of risk is probably a good thing,” Gugel said. “As a consumer, I would expect that.”