Building Public Trust: Developing a Framework for Measuring and Reporting the Impacts of AI

By: Emma Strubell and Tamara Kneese

AI is changing our lives — from our health care to the way we work. But as AI becomes more capable and integrated into the U.S. economy, its growing demand for resources, such as energy, water, land, and raw materials, is driving significant economic and environmental costs.

Why it matters: Resource consumption associated with AI infrastructures is expanding quickly, and this has negative impacts, including asthma from air pollution associated with diesel backup generators, noise pollution, light pollution, excessive water and land use, and financial impacts to ratepayers. Without full visibility by policymakers into the range of AI’s impacts and public participation to minimize these risks, we risk losing the public’s trust and slowing the advancement of AI.

The big issue: To create effective, sustainable resource policies, policymakers need to be able to access clear measurements of AI’s computing-related, application and system-level impacts and their negative outcomes. While there have been regulatory and technical attempts to develop scientific consensus and international standards around the measurement of AI’s environmental impacts, a more holistic picture needs to include impacts from AI’s application, such as applying AI to oil or gas drilling, and AI’s system-level impacts, broader social and economic impacts such as health impacts on local communities due to increased air pollution. These are challenging to measure and predict.

The way forward: Effective policy recommendations require more standardized measurement practices, including corporate transparency and innovation around technical ways to collect and report data, and engagement from multiple stakeholders to meet the needs of states, local government offices and communities. There is also a need to balance the potential costs and benefits of AI data centers and related energy infrastructures, to reduce state and local opposition.

Carnegie Mellon and Data & Society researchers have developed a plan of action with a series of recommendations to address these issues:

1. Develop a database of AI uses and framework for reporting AI’s immediate applications in order to understand the drivers of environmental impacts.
2. National Institute of Standards and Technology (NIST) should create an independent consortium to develop a system-level evaluation framework for AI’s environmental impacts, while embedding robust public participation in every stage of the work.
3. Mandate regular measurement and reporting on relevant metrics by data center operators.
4. Incorporate measurements of social cost into AI energy and infrastructure forecasting and planning.
5. Transparently use federal, state, and local incentive programs to reward data-center projects that deliver concrete community benefits.  

The bottom line: Data centers have an undeniable impact on energy infrastructures and the communities living close to them. This impact will continue to grow alongside AI infrastructure investment, which is expected to skyrocket. It is possible to shape a future where AI infrastructure can be developed sustainably, and in a way that responds to the needs of local communities. But more work is needed to collect the necessary data to inform government decision-making. The US needs to adopt a framework for holistically evaluating the potential costs and benefits of AI data centers and shaping AI infrastructure buildout based on those tradeoffs. That framework includes establishing standards for measuring and reporting AI’s impacts, eliciting public participation from impacted communities, and putting gathered data into action to enable sustainable AI development.

Go deeper: Read the full memo published by the Federation of American Scientists.