By Tom Kleckner

In an industry where grid operators often engage in bickering and litigation over their borders, ERCOT and SPP have proven neighbors can also collaborate for the common good.

The grid operators’ in-house developers have worked together to produce version 2.0 of the Macomber Map, a visualization tool for control rooms. Projected on large overhead monitors, the map provides a geographical depiction of the system, including customizable views of power flows, constraints and other core system data that feed into the map and are then pieced together.

“As we collaborate on the software, we’re supporting each other,” said Mike Legatt, ERCOT’s principal “human factors engineer.” Legatt says the Macomber Map improves operators’ decision-making by increasing their situational awareness and simplifying the complexity wrought by integrating renewable resources, changes in market design and faster information flows.

“Our experience … is a great illustration of a new way of thinking, both about the relationships between technology and grid reliability and between independent grid operators like ERCOT and SPP,” said Cody Parker, SPP’s supervisor of operations support. “The adoption rate of users has been tremendous. It’s been clear that the … user-friendliness of the tool, the performance and responsiveness of the tool … those aspects have won over all the operators.”

Seeing the ‘Big Picture’

The system is named in honor of its creator, ERCOT’s Gary Macomber, a human factors engineer who died in August 2008, a few weeks before his first map became a production-level tool.

“Gary was really an incredible guy. He could see the big picture,” Legatt said. “This was the one [project] he was really excited about, because it’s pulling together disparate data from people who gather it. By doing all that work for the operators, it gives them more time and more mental bandwidth to be solving these problems.”

“It’s opened up a whole new visualization framework without having to go through the physical process of drawing things pixel by pixel,” said SPP senior engineer Jeff Parker (no relation to Cody). “We’ve been able to use modern graphics-rendering capabilities built into it, instead of the old style of manually drawing everything.”

Legatt was behind the code for the map’s first version. However, he’s quick to say it’s the control room operators who developed the map.

“It’s all the users — operators, engineers and other groups within our organization — that have really defined how the tool grew and developed into what it is today,” he said.

In 2013, ERCOT released the code to the open source community for adoption by other users, such as governments, utilities and emergency responders who could benefit from improved situational awareness. The code has also been picked up by power-flow engineering students.

That led to version 2.0, which has an improved graphical user interface, among other upgrades.

Legatt credits SPP’s developers and operators for the new features. SPP staff tailored the tool to aggregate and analyze historical and real-time data from their energy management, markets, weather and other systems. The grid operators said SPP’s improvements enable operators to run what-if scenarios to monitor and mitigate congestion and outages.

“They’ve done some incredible work,” Legatt said of SPP. “They are showing things we were already showing, but in a better way. The graphical rendering is much nicer and much faster. It works better in remote-access situations … all thanks to Cody and his team.”

Human Factors Engineering

Cody Parker and his team became interested in Legatt’s work on human performance improvement through mutual industry contacts. Human factors engineering, also known as cognitive ergonomics or user-centered design, combines psychology and engineering.

Parker spent several months talking with Legatt, who has two master’s degrees and a doctorate in clinical health and neuropsychology. He recently defended his thesis in pursuit of a second doctorate from the University of Texas at Austin in energy systems engineering.

Once SPP’s Integrated Marketplace went live in 2014, Parker and his team were able to devote their full energy to adapting the map for their use.

“Working with Mike Legatt has been an absolute pleasure. It’s the key to our success and the ongoing coding,” Jeff Parker said. “He brings a whole new layer of ideas and questions … as we implement a requirement.”

After Legatt joined ERCOT in 2006, Macomber asked him to sit with ERCOT’s operators so he could understand their needs. “It became even more clear why we needed this tool,” Legatt said.

ERCOT began using the map’s first release in its training exercises in 2009, allowing its operators and those of market participants to visualize system restoration in black-start exercises and other scenarios.

Woody Rickerson, the ISO’s vice president of grid planning and operations, says he has seen performance continue to improve in training exercises. “That tells us those operators are even more prepared for success in real-life situations,” he said.

‘Eye-Opening’ Visit

Cody Parker visited ERCOT during one of the black-start exercises. Seeing the training was “very eye-opening,” he said.

Parker and his team took the code back to SPP and, after six months of development, integrated the tool with its other systems. The RTO has been using the map in its operations since March and recently began using it to train all of their operators. Parker’s team is now working on making the tool available to SPP’s external training department.

“We were able to take the users’ feedback and quickly apply it,” Parker said. “More often than not, that enhanced development was available for the [next] round of trainers. That was one of the huge advantages of going with the open-source solution.”

“The nice thing about using a product we have the source code for is we can go in and make things the way we need to make them work. Using a vendor’s tool, we just couldn’t go in and make any changes we wanted,” said SPP’s Tim Van Prooyen, a senior operations programmer and analyst.

One of the tool’s other advantages is that its updated rendering makes it easier to run on virtual machines — programs that emulate dedicated hardware.

“Before, you needed a physical machine on each person’s desk, but now you can [use] virtual systems without dedicated video graphics cards. Any user can pretty much use it on any machine, remote and on-site,” Parker said.

“The only thing we ask is if they find out ways to make it better … to let us know about it,” Legatt said. “One of the benefits of open sourcing the map and building these kinds of collaborations is that different parties need different things. So every enhancement they make that flows back to the open-source community benefits everybody else, moving us all to a better place.”

“This one project has opened the doors in multiple aspects,” Jeff Parker said. “It’s been much more of an exchange of ideas beyond the Macomber Map.”

Asked what version 3.0 might look like, Legatt promises it will be exciting. “The best thing is this collaboration with SPP. We have proven to ourselves this philosophy works.”

The Macomber Map’s source code and more information can be found here. Legatt said he and others can provide additional information and training support to help others customize the map to serve their own needs.