Delphi Group, University of Maryland, Meta Honored for COVID-19 Survey

The team of Carnegie Mellon University’s Delphi Group, the University of Maryland Social Data Science Center, and Meta is the recipient of two awards for its work on the COVID-19 Trends and Impact Survey.

The American Association of Public Opinion Research (AAPOR) awarded the researchers the Policy Impact Award and the Warren J. Mitofsky Innovators Award. It is the first time a single project received two awards in the same year from AAPOR, the professional society for survey scientists.

"We are honored to be recognized for this work along with our many collaborators who have made it possible, and we're grateful to the millions of survey respondents who provided crucial data for understanding the pandemic," said Alex Reinhart, an assistant teaching professor in Carnegie Mellon’s Dept. of Statistics & Data Science, and a member of the Delphi Group.

Responding to requests from the U.S. Centers for Disease Control and Prevention and public health researchers globally, the Delphi Group and the University of Maryland Social Data Science Center created the COVID-19 Trends and Impact Surveys (CTIS), in partnership with Meta (formerly Facebook).

Utilizing Facebook as a sampling frame to reach tens of thousands of individuals daily initially in the US and later globally, data from randomly selected users of the platform were used to forecast cases and healthcare needs to allow improved and rapid planning response. Data were subsequently also used to monitor health behaviors, vaccination hesitancy, mental health, and the like. Numerous methodological, legal, and technical challenges were successfully navigated to develop this approach and share such detailed data, providing unprecedented access to partners outside the social media organization.

The COVID-19 Trends and Impact Survey (CTIS) is the largest public health survey to date. Carnegie Mellon conducts the survey nationally, and the University of Maryland runs it internationally. Each day, a random sample of Facebook users are asked to answer questions about COVID symptoms, risks, mitigating behaviors, mental health, testing, vaccination and more. 

"There was no template for this survey. We've been responding to a public health crisis in real time," said Robin Mejia, an assistant research professor in Carnegie Mellon’s Dept. of Statistics & Data Science, and a member of the Delphi Group. "It's been amazing to work with this team as we have updated the survey, built the data infrastructure and provided the information to those who need it."

The Policy Impact Award recognizes outstanding research that improves policy decisions, practice and discourse. The award committee applauded the efforts of the CTIS in tracking trends in COVID-related topics and the impact that the survey results have had on government policies around the world. The committee also recognized the ability of the CTIS to provide geographic granularity that made the results of the survey useful at nearly all levels of government.

The Warren J. Mitofsky Innovators Award recognizes innovations in the field such as new theories, ideas, applications, methodologies or technologies. The award committee honored the CTIS for its novel use of social media; best survey practices at a global scale; enterprise speed to collect timely, critical data through a public-private partnership with transparent methodology; and immediate and broad API access to the public and microdata access for researchers worldwide.

The survey started in April 2020 and has collected 106 million responses worldwide. Such detailed data has never been available during a public health emergency. The survey helped public health officials understand how to save lives and safely reopen public life, and informed researchers on the social, economic and health effects of the pandemic.

At Carnegie Mellon, Delphi is led by Ryan Tibshirani, of the departments of Statistics & Data Science, and of Machine Learning, and Roni Rosenfeld, of the department of Machine Learning.  Its mission is to “develop the theory and practice of epidemiological forecasting, with a long-term vision of making this technology as universally accepted and useful as weather forecasting is today."

Delphi involves researchers in both the Statistics and Machine Learning departments, and collaborators at many other universities. During COVID, Delphi built the COVIDcast website with many different kinds of data about COVID, and worked on forecasting and other research. The surveys that received the awards were part of Delphi's original work to get data that could be used for forecasting, but then grew dramatically to involve questions useful for public health more generally. There are now over 70 universities and nonprofits with access to the survey data to do research and make policy decisions.

“This project is a great example of a collaboration between disciplines and a partnership between several institutions, public and private,” said Frauke Kreuter, a professor in the Joint Program in Survey Methodology at the University of Maryland. “All partners shared our aspiration for methodological rigor and worked furiously to create quick access to information.”

Ultimately, our goal was to support health partners to offer informed public health decisions," said Curtiss Cobb, vice president of research, demography and survey science at Meta. "We are proud that this innovative partnership has proven effective in advancing public health insights during a global pandemic and look forward to exploring how we can further support health partners."

Read more about the AAPOR awards at the association's website