AI and Its Growing Energy Demand

By: Zico Kolter

AI models, especially the large models powering systems like ChatGPT, have exploded in capabilities and usage over the past years. In response, there have been many commitments from large companies to spend tens or hundreds of billions of dollars to build data centers that can power these AI models.

Why it matters: These data centers, in turn, need to be powered by large amounts of electricity: it’s not uncommon for newly proposed data centers to consume over 1 Gigawatt (GW) of electrical power. By comparison, the entire Pittsburgh area uses an average of between 1-2 GW of electrical power over the course of a day.

The big questions: But why do AI models use this much electricity? What is driving this rapid increase in demand? And perhaps most subjectively, will the resulting increases in AI capabilities and availability be “worth” the cost in power?

Catch up quick: Doing any kind of computation on a computer requires power: to use your laptop, you need to plug it into an outlet, and if you’re doing a particularly intensive task, you’ll likely even notice it heating up from that power consumption. AI models work the same way but on a much larger scale.

The AI models that power systems like ChatGPT work by first transforming the text you type into numbers, then multiplying and adding these numbers many trillions of times, very quickly, to eventually produce a written response, an image, or a video. The costs associated with data centers correspond to: 1) the cost of the building/facilities themselves; 2) the cost of the computer chips that run the computation; and finally 3) the electrical power used to run the chips and run the cooling systems to prevent them from overheating.

The details: AI’s energy demands have grown rapidly for two main reasons. 

• First, because of a phenomenon called the “scaling laws” of AI models. For several years, researchers have recognized that if you increase the number of computations used by the AI models: say going from 100 billion to 1 trillion to 10 trillion operations to produce an output, then the performance of the resulting model will improve by corresponding amount. Due to the desire to create ever-more-capable models, companies want to create models that use more and more computations, and hence more and more energy.
• The second reason is due to our increasing use of AI. In the past 2.5 years, AI has grown from a set of niche use cases, to a tool that many of us use every day. Thus, the net effect of larger models and massive growth is a rapidly increasing demand for electrical power, even in light of other improvements of efficiency.

Worth noting: We don’t always need more electrical power to run the AI models: the efficiency of computer chips are also improving exponentially as are the efficiency of the underlying algorithmic approaches themselves, which can offset much of the increase in power.

The big picture: All of this finally leads to the natural question: is this increasing demand for power “worth” its cost? The answer, naturally, depends on our individual perception of the value of AI systems. If you are inherently skeptical about the value provided by AI systems, then you may feel that the benefits do not outweigh the energy costs. But if (and I now have to acknowledge that I place myself in this camp) you believe that AI has the potential to substantially transform our world for the better, then the energy cost may seem to be well worth it.

AI technology, if deployed responsibly, has the potential to drastically increase productivity, to enable us to create software in a faster and more robust manner, to advance science, and to ultimately benefit the human condition. Most revolutions that ultimately have increased the quality of life for the majority of humanity — the industrial revolution, modern transportation, and the introduction of computers — all came with associated increases in energy cost, and AI will likely be no different.

What's next: As work across Carnegie Mellon shows, AI has the potential to drastically improve our energy consumption as well, assisting in developing more efficient techniques for grid operation, building better materials for batteries, and potentially even truly revolutionizing energy through accelerating the development of technologies like nuclear fusion. These are all big bets, to be clear, and advancing science is never a sure thing, but AI at its best can be a unique enabler of so many beneficial downstream technologies.

As we move AI technology forward, and build the needed infrastructure to power this revolution, it is incumbent upon all of us to ensure that the positive impacts of AI are worth the substantial energy cost. Owning to work at Carnegie Mellon and elsewhere, we are well-positioned to meet this challenge.