Penn professor on gen AI’s rapacious use of energy: ‘One of the defining challenges of my career’

“The hardware costs and infrastructure costs are really daunting, they’re growing at such high rates,” said Lee, an engineering professor at the University of Pennsylvania. “This is probably one of the defining challenges I’ve seen in my career so far.”  

Lee’s research looks at sustainable computing and how to maximize efficiency with power constraints. He’s now turned his focus to the rise of generative AI’s impact on efficiency. 

The computational demands of gen AI include both training and deployment. Companies tend to be concerned about the financial costs of training AI models, he said, but there are also deployment costs. Creating a single image can use as much energy as is needed for a full smartphone charge, researchers at Carnegie Mellon University showed last year. 

These kinds of energy costs will only grow as AI becomes more integrated into everyone’s lives, Lee noted.

“The difficulty is that when you build these data centers and pack them full of computer hardware, you’re gonna need power to run all that computer hardware,” Lee told Technical.ly. His concern is the stress it puts on the power grid and the increase in carbon intensity. 

Benjamin Lee. (Courtesy University of Pennsylvania)

As they look to increase capacity to experiment, big tech companies like Microsoft, Amazon and Google are building data centers and buying graphics processing units — aka GPUs, the parallel-processing circuits built for images that are now the main chips used for training large language models. Internet data centers’ growing need for electricity has led to suggestions for coal power expansion, despite the potential environmental impacts. 

The global technology industry produced 2%-3% of the world’s carbon emissions in 2021, according to the UN Environment Programme. There’s been investment in renewable energy, utility scale batteries and nuclear power to offset the amount of power data centers are using. 

Electricity usage rates are increasing faster than rates of renewable energy installations, Lee noted, despite the fact that they’re being built faster and faster — worldwide installations jumped 50% from 2022 to 2023. 

“You need to be able to understand where your energy is going,” Lee said. “What kind of workloads, what kind of hardware is consuming all that energy, and then you can figure out how to target and reduce those energy costs.”

To improve sustainability in computation, the industry needs to continue to invest in renewable energy and think about energy storage, Lee said. Data centers also need to be more flexible about when they are operating, opting to do more computing at times of day when there is more energy available from wind or the sun, for example. Some have suggested AI could be deployed to lower buildings’ energy consumption.

Based on his research, Lee believes he has a general idea of what AI training energy consumption looks like, but going forward there remains uncertainty about exactly how intensely companies and the general public will use AI models. 

“It’s really about the rate of adoption,” Lee said. “And how broadly that adoption happens.”