Vistra rapid scaling mobile

Rapid scaling

When a solution has proven value at a pilot site and is approved for scaling, a team of software and machine learning engineers immediately takes over to refactor, modularize, and containerize the code. That way there is a single software ‘core’ package for each deployment that can be updated and improved.

A product owner manages the overall process and takes ownership for use and adoption. Over time, the team developed seven solutions archetypes, which provided consistent approaches, logic, assumptions, and algorithmic elements as a basis for each new application. This gave each new solution a big head start when development began.

Turn each successful proof of concept (or MVP) into a product

AI models developed

400+

Select a rapid scaling step

Vistra implemented an MLOps approach to create a “factory” that standardized the deployment and maintenance of more than 400 AI models. This enabled the team to bring live data from each of Vistra’s power units into a single database; use Gitlab software to manage version control for code; containerize the code so it could be easily deployed to any environment; set up a scheduler (using AirFlow) to make sure recommendations and actions were delivered on time; create dashboards to monitor model performance and usage; and manage the continuous improvement of each model to make sure the plants were sustaining the value they captured.

Create machine learning operations (MLOps) infrastructure

What is MLOps?

Machine Learning Operations (MLOps) is the set of practices and infrastructure to manage the production and deployment of analytics solutions or products. Improvements in AI tooling and technologies have dramatically transformed AI workflows, expediting the AI application life cycle and enabling consistent and reliable scaling of AI across business domains. This framework can underpin a company AI “factory” to achieve scale.

percent of each product is reusable

50-70

At the Moss Landing Power Plant in California, the unit that faces the ocean had to deal with more moisture and salt than the one that faces inland, so the degradation profile for each filter is different. So dedicated customization teams made up of data scientists, engineers, operators, and power generation experts worked with each power plant to tailor the solution to the unique conditions of that particular plant.

Because each product is designed to be modular and reusable, 50 to 70 percent of each is ready to be used from day one. But customization is always needed for each plant’s characteristics. Take the maintenance solution we developed to understand the best time to replace inlet filters at gas turbines (each filter costs $150K, and a plant needs to be shut down for two days to replace one).

Customize and adapt for “last-mile” implementation

Building on Vistra’s culture of continuous improvement, McKinsey worked to help Vistra to get the most from AI. That included a program for operators to explain the models and how they were developed, as well as how to use the tool itself. As Rachit Gupta, Vistra Energy’s vice president for generation and wholesale applications, said: “People had to know what the model was doing and learn to trust that it was right. Once they saw that the models were generating recommendations that made sense and helping lower heat rate, they were ready to start using them more.”

Build capabilities

—Rachit Gupta, Vistra VP for generation and wholesale

Once they saw that the models were generating recommendations that made sense and helping lower heat rate, they were ready to start using them more.

“

There’s a simple screen that not only shows you what needs to be optimized but how much it’s worth. That really registers with people.”

“

From the beginning of the solution development process, McKinsey designers worked with operators to understand their day-to-day activities. What soon became clear was that plant operators had constraints on their time and had to manage dozens of inputs that were tracked on an array of screens in the control room. Adding to that workload would overwhelm operators and reduce the solutions’ effectiveness. The tools had to make operators’ lives easier.

Design for the operator experience

For this reason, the screen that displayed the AI solutions and recommendations were integrated into PowerSuite, an interface that operators already used so they didn’t need to monitor another screen. The displays themselves were designed to be easy to read. A solution displays a green signal if the plant is running optimally for the given conditions, and red when it isn’t. A brief recommended action accompanies any red display, with the value attached to implementing that recommendation.

Turn each successful proof of concept (or MVP) into a product

Create machine learning operations (MLOps) infrastructure

Customize and adapt for “last-mile” implementation

Build capabilities

Design for the operator experience

Select a rapid scaling step

Teams also incorporated multiple approaches to reduce risk by building functional limits (e.g. maximum throttle pressure or heat levels) into the code and putting all code through biweekly peer reviews and multi-week testing. McKinsey Risk Dynamics experts worked with the team to test assumptions, review code, and ensure that all risk best practices were reflected in the models.

What is MLOps?

Additional training for the core tech team covered how to build and maintain models, resolve issues, and deeper machine learning and analytics skills to understand how models work in detail. This happened by working alongside McKinsey AI experts. Vistra has also partnered with the University of Texas to offer a range of analytics courses for employees.

—Lloyd Hughes, Vistra operations manager

There’s a simple screen that not only shows you what needs to be optimized but how much it’s worth. That really registers with people.