Despite the growing threat of cyberattacks, utilities in many countries lack consistent regulatory frameworks for hardening their systems against malicious actors, according to a new report from the National Association of Regulatory Utility Commissioners and the U.S. Agency for International Development.

The groups produced Evaluating the Prudency of Cybersecurity Investments: Guidelines for Energy Regulators as a resource to help energy regulators craft strategies for determining whether utilities’ cybersecurity preparations are cost-effective and worthwhile. The guidelines are particularly intended for Europe and Asia, but the organizations hope they will be useful for any regulator that has not created their own frameworks yet and are unsure of where to begin.

“Start with a plan, and don’t waste time trying to make it perfect; this will also help to get the operators thinking strategically,” the report says. “The U.S. experience of many state regulators shows that starting to do something, even when it seems to be a drop in the ocean, provides expertise, feedback and engagement that are precious for shaping continuously improved strategies.”

Evaluating Effectiveness a Major Challenge

A crucial step in the development process, and the one that may be most challenging for regulators just starting their task, is deciding how to evaluate the effectiveness of a particular cybersecurity standard. It may be impossible to tell if one standard works to keep the grid safe because the metrics to determine success have not been developed yet. Another standard may seem to be working fine until an attacker exposes a vulnerability in a utility’s security protocols that the regulator never thought to test.

The report acknowledges that the four-step checklist for establishing reliability metrics in general is well understood: First, researchers and experts identify a list of useful indicators. Regulators then impose the calculation of these metrics on regulated companies, which report their value on a regular basis. These values are used to assess whether future investments will be effective for delivering the desired results.

But applying this process to create cybersecurity metrics is difficult, as “research concerning cybersecurity performance indicators is ongoing, while established practices are nearly nonexistent.” While metrics have been developed — the report singles out the Electric Power Research Institute’s Cyber Security Metrics for the Electric Sector as “one of the most advanced studies in the field” — these are at a very early stage of maturity.

Even if a metric seems to show improvement after a particular measure is implemented, the two events may not necessarily be connected; the positive change could simply be because of increased awareness of cyber risk on the part of a utility’s employees.

No One-Size-Fits-All Approach

In addition to an incomplete body of research, regulators must contend with the wide array of threatening actors, which have varying levels of sophistication as well as widely differing goals. Because each country faces a different set of adversaries, it may be hard to apply the work done in one market to other geographies, the guidelines say.

While the report draws on previous work by regulators — such as NERC’s Critical Infrastructure Protection (CIP) standards, as well as others from the U.S. and other countries — it does not present “unique turnkey solutions” for adoption and warns against copying successful strategies too closely. An effective regulatory framework must be tailored to address the circumstances of each individual market, based on the best information available and cognizant of the practical limitations facing regulated entities.

Suggested questions to shape regulators’ development are:

• What are my objectives, and where should I start?
• What strengths, weaknesses, opportunities and threats exist for utilities in my country from a cyber perspective?
• Are there governing laws or administrative rules that limit or expand my influence in this area?
• What mutual aid agreements are in place, if any, between my country and its neighbors?
• Do I have enough skilled personnel in-house to address cybersecurity cost identification and benchmarking?

The report says ensuring cybersecurity preparedness can be accomplished only by those most familiar with a particular market and with the most at stake in the event of a successful attack. While much can be gained from sharing information on threats and defense measures, there is no single approach that can be used on a wide scale, it says.

It is “likely [a gold standard] will never appear because the design of a regulatory approach is not a technical task, but it is truly connected to a country’s values, vision and legal environment,” the report says. “Regulators must get started immediately and learn lessons along the way because experience will answer more questions than a 1,000-page book that would become outdated in six months’ time.”