Steven Low, professor of computer science and electrical engineering at Caltech, says we are on the cusp of a historic transformation—a restructuring of the energy system similar to the reimagining and revamping that the communication and computer networks experienced over the last two decades, making them layered, with distributed and interconnected intelligence everywhere.

The power network of the future—aka the smart grid—will have to be much more dynamic and responsive than the current electric grid, handling tremendous loads while incorporating intermittent energy production from renewable resources such as wind and solar, all while ensuring that when you or I flip a switch at home or work, the power still comes on without fail.

The smart grid will also be much more distributed than the current network, which controls a relatively small number of generators to provide power to millions of passive endpoints—the computers, machines, buildings, and more that simply consume energy. In the future, thanks to inexpensive sensors and computers, many of those endpoints will become active and intelligent loads like smart devices, or distributed generators such as solar panels and wind turbines. These endpoints will be able to generate, sense, communicate, compute, and respond.

Given these trends, Low says, it is only reasonable to conclude that in the coming decades, the electrical system is likely to become "the largest and most complex cyberphysical system ever seen." And that presents both a risk and an opportunity. On the one hand, if the larger, more active system is not controlled correctly, blackouts could be much more frequent. On the other hand, if properly managed, it could greatly improve efficiency, security, robustness, and sustainability.

At Caltech, Low and an interdisciplinary group of engineers, economists, mathematicians, and computer scientists pulled together by the Resnick Sustainability Institute, along with partners like Southern California Edison and the Department of Energy, are working to develop the devices, systems, theories, and algorithms to help guide this historic transformation and make sure that it is properly managed.

In 2012, the Resnick Sustainability Institute issued a report titled Grid 2020: Towards a Policy of Renewable and Distributed Energy Resources, which focused on some of the major engineering, economic, and policy issues of the smart grid. That report led to a discussion series and working sessions that in turn led to the publication in 2014 of another report called More Than Smart: A Framework to Make the Distribution Grid More Open, Efficient and Resilient.

"One thing that makes the smart grid problem particularly appealing for us is that you can't solve it just as an engineer, just as a computer scientist, just as a control theorist, or just as an economist," says Adam Wierman, professor of computer science and Executive Officer for the Computing and Mathematical Sciences Department. "You actually have to bring to bear tools from all of these areas to solve the problem."

For example, he says, consider the problem of determining how much power various parts of the grid should generate at a particular time. This requires generating an amount of power that matches or closely approximates the amount of electricity demanded by customers. Currently this involves predicting electricity demand a day in advance, updating that prediction several hours before it is needed, and then figuring out how much nuclear power, natural gas, or coal will be produced to meet the demand. That determination is made through markets. In California, the California Independent System Operator runs a day-ahead electricity market in which utility companies and power plants buy and sell power generation for the following day. Then any small errors in the prediction are fixed at the last minute by engineers in a control office, with markets completely out of the picture.

"So you have a balance between the robustness and certainty provided by engineered control and the efficiency provided by markets and economic control," says Wierman. "But when renewable energy comes onto the table, all of a sudden the predictions of energy production are much less accurate, so the interaction between the markets and the engineering is up in the air, and no one knows how to handle this well." This, he says, is the type of problem the Caltech team, with its interdisciplinary approach, is uniquely equipped to address.

Indeed, the Caltech smart grid team is working on projects on the engineering side, projects on the markets side, and projects at the interface.

On the engineering side, a major project has revolved around a complex mathematical problem called optimal power flow that underlies many questions dealing with power system operations and planning. "Optimal power flow can tell you when things should be on or conserving energy, how to stabilize the voltage in the network as solar or wind generation fluctuates, or how to set your thermostat so that you maintain comfort in your building while stabilizing the voltage on the grid," explains Mani Chandy, the Simon Ramo Professor of Computer Science, Emeritus. "The problem has been around for 50 years but is extremely difficult to solve."

Chandy worked with Low; John Doyle, the Jean-Lou Chameau Professor of Control and Dynamical Systems, Electrical Engineering, and Bioengineering; and a number of Caltech students to devise a clever way to solve the problem, allowing them, for the first time, to compute a solution and then check whether that solution is globally optimal.

"We said, let's relax the constraints and optimize the cost over a bigger set that we can design to be solvable," explains Low. For example, if a customer is consuming electricity at a single location, the problem might ask how much electricity that individual is actually consuming; a relaxation would say that that person is consuming no more than a certain amount—it is a way of adding flexibility to a problem with tight constraints. "Almost magically, it turns out that if I design my physical set in a clever way, the solution for this larger simple set turns out to be the same as it would be for the original set."

The new approach produces a feasible solution for almost all distribution systems—the low-voltage networks that take power from larger substations and ultimately deliver it to the houses, buildings, street lights, and so on in a region. "That's important because many of the innovations in the energy sector in the coming decade will happen on distribution systems," says Low.

Another Caltech project attempts to predict how many home and business owners are likely to adopt rooftop solar panels over the next 5, 10, 20, or 30 years. In Southern California, the number of solar installations has increased steadily for several years. For planning purposes, utility companies need to anticipate whether that growth will continue and at what pace. For example, Low says, if the network is eventually going to comprise 15 or 20 percent renewables, then the current grid is robust enough. "But if we are going to have 50 or 80 percent renewables," he says, "then the grid will need huge changes in terms of both engineering and market design."

Working with Chandy, graduate students Desmond Cai and Anish Agarwal (BS '13, MS '15) developed a new model for predicting how many homes and businesses will install rooftop solar panels. The model has proven highly accurate. Researchers believe that whether or not people "go solar" depends largely on two factors: how much money they will save and their confidence in the new technology. The Caltech model, completed in 2012, indicates that the amount of money that people can save by installing rooftop solar has a huge influence on whether they will adopt the technology. Based on their research, the team has also developed a web-based tool that predicts how many people will install solar panels using a utility company's data. Southern California Edison's planning department is actively using the tool.

On the markets side, Caltech researchers are doing theoretical work looking at the smart grid and the network of markets it will produce. Electricity markets can be both complicated and interesting to study because unlike a traditional market—a single place where people go to buy and sell something—the electricity "market" actually consists of many networked marketplaces interacting in complicated ways.

One potential problem with this system and the introduction of more renewables, Wierman says, is that it opens the door for firms to manipulate prices by turning off generators. Whereas the operational status of a normal generator can be monitored, with solar and wind power, it is nearly impossible to verify how much power should have been produced because it is difficult to know whether it was windy or sunny at a certain time. "For example, you can significantly impact prices by pushing—or not pushing—solar energy from your solar farm," Wierman says. "There are huge opportunities for strongly manipulating market structure and prices in these environments. We are beginning to look at how to redesign markets so that this isn't as powerful or as dangerous."

An area of smart grid research where the Caltech team takes full advantage of its multidisciplinary nature is at the interface of engineering and markets. One example is a concept known as demand response, in which a mismatch between energy supply and demand can be addressed from the demand side (that is, by involving consumers), rather than from the power-generation side.

As an example of demand response, some utilities have started programs where participants, who have smart thermostats installed in their homes in exchange for some monetary reward, allow the company to turn off their air conditioners for a short period of time when it is necessary to reduce the demand on the grid. In that way, household air conditioners become "shock absorbers" for the system.

"But the economist says wait a minute, that's really inefficient. You might be turning the AC off for people who desperately want it on and leaving it on for people who couldn't care less," says John Ledyard, the Allen and Lenabelle Davis Professor of Economics and Social Sciences. A counter proposal is called Prices to Devices, where the utility sends price signals to devices, like thermostats, in homes and offices, and customers decide if they want to pay for power at those prices. Ledyard says while that is efficient rationing in equilibrium, it introduces a delay between the consumer and the utility, creating an instability in the dynamics of the system.

The Caltech team has devised an intermediate proposal that removes the delay in the system. Rather than sending a price and having consumers react to it, their program has consumers enter their sensitivity to various prices ahead of time, right on their smart devices. This can be done with a single number. Then those devices deliver that information to the algorithm that operates the network. For example, a consumer might program his or her smart thermostat, to effectively say, "If a kilowatt of power costs $1 and the temperature outside is 90 degrees, I want you to keep the air conditioner on; if the price is $5 and the temperature outside is 80 degrees, go ahead and turn it off."

"The consumer's response is handled by the algorithm, so there's no lag," says Ledyard.

Currently, the Caltech smart grid team is working closely with Southern California Edison to set up a pilot test in Orange County involving several thousand households. The homes will be equipped with various distributed energy resources including rooftop solar panels, electric vehicles, smart thermostats for air conditioners, and pool pumps. The team's new approach to the optimal power flow problem and demand response will be tested to see whether it can keep stable a miniature version of the future smart grid.

Such experiments are crucial for preparing for the major changes to the electrical system that are certainly coming down the road, Low says. "The stakes are high. In the face of this historic transformation, we need to do all that we can to minimize the risk and make sure that we realize the full potential."