This research team, based in the Human-Computer Interaction Institute, will receive funding for their continued research on designing automation policies for workers in the hospitality industry. Over the grant period, the team will pilot their framework by developing policy to regulate automation and ensure worker health in the hospitality sector in cooperation with the United States Occupational Safety and Health Administration (OSHA).

This project will receive funding to develop a web-based tool to address the shortcomings of current practices in community asset mapping, which will be piloted in multiple municipalities in the region. This project is directly aimed at improving communities, particularly those in some distress, to seed their futures by enabling them to compete for government funding and private investments.

This team will develop an integrated research effort to characterize job opportunities in a regional transition to clean energy, with a focus on mapping the skill sets of today’s underemployed and vulnerable populations to employment opportunities in a decarbonizing regional economy. Focusing on ten counties in Southwestern Pennsylvania, they will perform this mapping under scenarios that assume growth in different activities in a clean energy economy.

Professors Li and Branstetter will continue to investigate new transportation options to drive low-income citizens to greater success. This project will expand on the researchers’ previous work on increasing low-income access to transportation options, investigating new modes such as public transportation, bike sharing, scooters, and short-term car rentals.

This research team posit that as public sectors seek efficiencies through the increased use of algorithmic systems, developers are becoming default policymakers, with citizens having little opportunity for input, and sometimes no opportunity for redress or remedy. The team proposes an actionable framework to guide the discovery, analysis, and mitigation of existing or potential policy gaps across the life cycle of public algorithmic systems to inform their future development.

This project aims to develop an understanding of how decision-makers work with AI tools to support content moderation in real-world contexts. Scientific and design knowledge remains scarce on how the strengths of human and AI judgment can be effectively combined in practice while mitigating the drawbacks of each, and the team plans to develop new design guidelines and policy recommendations that can support more effective AI-augmented decision-making in these settings.

Technology has become fundamental to trading and allocation of capital in equity markets. This has led to a proliferation of exchanges and trading platforms, and the execution of trades in small pieces. Combined with advances in high frequency trading, this has raised new questions about what constitutes a fair price for retail traders in equity markets. This research project will examine what possible regulatory reforms can limit the adverse consequences experienced by retail investors due to these technological advances.

The Pennsylvania Commission on Sentencing (PCS) seeks to revise the impact of the Prior Record Score to reduce the score’s overall impact on recommended criminal sentences. Under the present system, for any given offense, a high Prior Record Score could quadruple sentence length, which might adversely affect minority groups. This project seeks to support PCS through a comprehensive analysis of data on sentencing decisions as the Commission seeks to devise more equitable sentencing processes.

Climate goals under the Biden Administration include a 50 percent reduction in greenhouse gas (GHG) pollution from 2005 levels in 2030, and 100 percent carbon pollution-free electricity by 2035. Achieving these objectives requires major infrastructure investments as well as substantial technology and policy developments—and transportation electrification sits at the intersection of these changes. This proposal focuses on addressing the policy needs of electrifying the U.S. transportation sector and intends to develop a policy analysis framework to measure potential impacts of various federal investment scenarios to find efficiencies and maximize the carbon offset potential of new electric grid investments.

This project is a partnership with hospitality union UNITE HERE to understand how automation affects workers’ safety at work, their own autonomy, and the quality of client-facing service. The objective is to co-develop policy with workers, OSHA staff, and union representatives, while also exploring opportunities to create new safe and sustainable jobs.

This project investigates the influence of digital design on the quality of consumers' decision making while retail investing. The use of various design elements, gamification, social cues, and retail exuberance on the frequency, efficiency, and risk-taking of retail traders will be analyzed. Ultimately, this research aims to inform understanding of asset pricing, retail trading behavior, and the design of effective consumer protection policy.

Through better election transparency, this research aims to support the restoration of trust in the democratic process, while maintaining accuracy in determining election outcomes. This project has previously developed rigorous statistical tools to continuously monitor election audits and potentially end them early, as soon as results can be verified with high confidence that the announced outcome is correct. Despite having created this more statistically efficient risk-limiting audit, further work will be done to decentralize the mechanism so it can be applied to elections collecting ballots across various locations. Continued work will strive for large-scale acceptance, standardization, and deployment of risk-limiting audits in government.

The Chronicle of Higher Education recently observed "American higher education has become a powerful means for perpetuating class divisions across generations." This research aims to analyze the use of high-quality digital Calculus and Statistics courseware in unprivileged school districts and their effect on high school students' achievement and interest in pursuing college-level studies in STEM fields. The goal is to analyze how this courseware can help to close the gap in access to higher education to produce meaningful results with policy impact.

This project will develop estimates of the monetary value of improved workplace safety, determining the benefits from such policies. This project is more generally relevant to evaluate the ethical standards of jobs. With the advent of new technologies and the increased flexibility of work (e.g., Gig economy jobs), workers often must give up important amenities and benefits that are not priced into the wage. Although the hourly wage of these new jobs might be comparable to that of more traditional jobs, the total value of the former jobs might be lower when considering non-wage benefits. Our novel method for pricing non-wage amenities will allow us to compare the value of new tech-driven jobs to traditional jobs characterized by substantially more non-wage benefits.

2020 Funded Projects

In 2020, we funded twelve projects that brought together Carnegie Mellon University’s world-class researchers in collaboration other academic institutions and local and national practitioners, applying cutting edge AI and machine learning techniques and proven social science-based approaches to address our most critical and timely societal challenges. Projects helped respond to the pandemic, address long-standing racial inequities and ensure the security and transparency of U.S. elections.

Developing technological solutions to combat COVID-19

The Impact of the COVID-19 Shutdown on the Most Vulnerable Households: New Data to Identify the Greatest Need (Lee Branstetter and Beibei Li)

In a matter of months, the COVID-19 pandemic and economic shutdown has severely jeopardized already marginalized and at-risk populations in ways unlike other economic recessions. To address the economic downturn, limited resources must be correctly allocated in a rapid and effective manner. Using large-scale GPS data in combination with data records maintained by Allegheny County’s Department of Health and Human Services, this project will equip policymakers and service providers with the tools to respond to the urgent medical, social and economic needs of this unprecedented crisis. For example, this project aims to characterize the loss of both “formal” and “informal” income, going beyond traditional metrics to examine the effect of the shutdown on the gig economy. Additionally, the insights gathered from this project will help to bolster contact tracing efforts and evaluate the impact of the pandemic on income loss, mental health, housing instability, and other social factors that are difficult to identify and measure with existing data resources.

Designing Better Autonomous-Transit Systems for Enhanced Workforce Resilience (Corey Harper and Destenie Nock)

From wildfires to the current pandemic, there is an increasing demand to identify new systems that meet critical transport needs without putting drivers at risk. Autonomous vehicles could meet such a need while bolstering a more environmentally friendly transportation infrastructure. During the current COVID-19 crisis, transportation systems sit at the crux of quarantine management and workforce protection, as essential workers that rely on public transportation and transit workers are at high risk of exposure to the virus. Autonomous vehicles could support contactless delivery, emergency response and evacuation, and essential worker commuting. This study will consider the workforce implications and climate impact of integrating autonomous vehicles into transit systems, as well as develop practical tools to inform policy regarding autonomous vehicle adoption and infrastructure development.

Co-Developing Automation Policy for the Post-COVID Hospitality Industry (Jodi Forlizzi, Sarah Fox. and Chinmay Kulkarni)

The COVID-19 pandemic has had a devastating impact on the hospitality industry, causing this major sector to effectively grind to a halt. In April, a staggering 98% of members of UNITE HERE, the largest hospitality workers union in North America, had been furloughed or laid off. While automation is a promising avenue to revitalize this sector and complement human labor, there are reasonable concerns about displacement or a lack of employee input into its integration. In partnership with UNITE HERE, we are co-designing automation policy  and visioning technology that addresses the needs and concerns of workers in the hospitality industry, ensures that work in this sector is safe and fulfilling, and supports policymaking and collective action. 

An Open-Source Decision Tool to Identify and Support Responses to Emergent Constraints in the Medical Supply Chain (Erica R.H. Fuchs, Valerie J. Karplus, M. Granger Morgan, and Sandra DeVincent Wolf)

In the early stages of the COVID-19 pandemic, a supply chain breakdown resulted in a shortage of the elastic for ear loops, preventing the production of at least 9 million additional medical masks. This is just one example of the myriad ways in which the pandemic has emphasized the significant global interdependencies within the health and manufacturing sectors, as well as hampered these sectors’ ability to meet the increased demand for critical PPE. While identifying and responding to these weaknesses in the medical supply chain is now particularly important; the supply chain issues highlighted by COVID-19 are a long-term problem. In order to overcome existing and future bottlenecks and capacity constraints, we are working with Catalyst Connection, the Manufacturing Extension Partnership of Southwestern Pennsylvania, to develop an open-source decision tool that uses publicly available data to inform domestic and international manufacturing responses to resource demands and guide innovation to meet supply constraints.

PaCE: Developing a Pandemic Consumption Expenditure Index (Laurence Ales, Rebecca Lessem, Christopher I. Telmer, and Ariel Zetlin-Jones)

OpenTable restaurant reservations rapidly collapsed two weeks prior to Pennsylvania Governor Tom Wolf’s Shelter-In-Place order. Movie theaters in Georgia remain empty in spite of the state government’s decision to re-open them on April 27. These are just two examples suggesting that consumer behavior—not government mandate—will be the key driver of the economy’s transition from lock-down back to economic health. Through the development of a Pandemic Consumption Expenditure Index (PaCE), we are examining consumer behaviors in real-time in order to minimize the economic impact of the COVID-19 pandemic and bolster critical decision-making for policymakers, businesses and workers through each phase of lockdown and reopening. Beyond COVID-19, PaCE will help policy makers better understand how other pandemic-like shocks to the economy are impacting consumer behavior at a level far more granular and relevant than existing data sources.

The Role Of Co-Experience And Technology In Mitigating Isolation From Social Distancing (Erin Carbone, Laura Dabbish, Simon DeDeo, and George Loewenstein)

The prolonged isolation caused by COVID-19 has translated into depression and other mental health issues for nearly half of American adults. Though COVID-19 has drastically exacerbated the problem, social isolation was already a long-term problem threatening the elderly, mentally ill and other vulnerable populations, and will continue to be when COVID-19 recedes into the past. While technology will never be a perfect substitute for in-person interaction, it can help mitigate the effects of social isolation. This project considers the impact of virtual interactions on loneliness, social connectedness, and mental wellbeing through a psychological and behavior lens, to identify and test the viability of developing sufficient substitutes for in-person interaction by leveraging the design features inherent to video chat, phone calls, text messaging, and other communication channels. The specific focus of the project is on the importance of the simultaneity of the experience – i.e., knowledge that the individuals are experiencing the same thing at the same time – as well as mutual awareness of that simultaneity.

Equity and Fairness in AI

Fast and Fair Hiring via Segmented Evaluations

On average, a given job listing in the United States will attract applications from 118 candidates. As widespread job losses in certain sectors coupled with surges in labor demand in others have drawn attention to several major flaws in the hiring process, this project considers a novel human resources approach to counteract long-standing biases and inefficiencies within the hiring process. Traditionally, an individual hiring manager or a small group of hiring managers review a complete application to decide whether or not to hire an individual. However, there is significant evidence that the process is not only inefficient, but often biased against people of color, immigrants, and other marginalized demographic groups. To address these issues, the research team has developed a novel blended human-AI approach where applications are evaluated in a segmented manner. Throughout the year, the team will test and study the new approach to measure improvements in equity and efficiency.

Understanding Historical Biases in EEG Data and Neuroscientific Studies Because of Bias in Data Acquisition Systems (Pulkit Grover, Shawn K. Kelly, Ashwati Krishnan, and Christina M. Patterson)

Electroencephalography (EEG) systems have been a common medical practice to identify brain conditions for nearly one hundred years, yet only recently a team of researchers at Carnegie Mellon University determined that common EEG machines are less effective for patients with coarse and curly hair, which is common in individuals of African descent. This significant gap has shed light on the sampling bias inherent to past EEG data, which informs clinical decision-making regarding stroke, epilepsy, and other neural conditions. Through a comparative analysis of neuroscientific data gathered using traditional EEG systems and “gold-cup” electrodes developed to work with coarse and curly hair, this project aims to quantify statistical differences between these two methods and explore algorithmic techniques for correcting bias in past EEG data, as well as incentivize clinicians and neuroscientists to switch to newer, fairer EEG systems.

An Integrated Framework for Studying and Regulating Human-AI Hybrid Decision-Making Systems (David Danks and Zachary Chase Lipton)

From generating risk scores that inform lending decisions to helping to narrow a pool of job applicants, algorithms are increasingly used as a predictive tool to improve the choices made by human decision-makers. However, little is known about the ways in which humans interpret, use, and trust AI-supported tools. Through a series of behavioral experiments, this project aims to better understand how AI-supported tools actually impact human judgement and evaluation patterns. These experiments will include examining how participants react when an algorithm supports or refutes their initial judgment, how access to additional information impacts decision-making, and how the benefits of correct judgments and costs of incorrect decisions affect participants’ behavior.

Laying the Technological Groundwork for Child Welfare Decision Support Systems through Advanced NLP (Alexandra Chouldechova, David Steier, and Yulia Tsvetkov)

Child welfare agencies across the country are continually looking for ways to use their data to better support families and improve decisions at every stage of their processes.  Recent efforts to develop machine learning tools for child welfare, such as those in Allegheny County, PA and Douglas County, CO, have primarily focused on structured administrative data.  In this project we’re partnering with the Allegheny County Department of Human Services on an innovative project that will leverage unstructured free-text data to support case management, supervision, and quality improvement efforts.  We are developing advanced natural language processing technologies capable of using high volumes of both structured and unstructured data to assist service providers, caseworkers, supervisors, and other DHS staff.

Making Explainable Machine Learning Work for Public Policy Problems (Kasun Amarasinghe, Rayid Ghani, Kit Rodolfa, and Ameet Talwalkar)

As applications of machine learning and AI are rapidly expanding into new areas of public life, it is important to consider whether public sector decision makers adequately understand the outcomes of the algorithms in a way that leads to optimal outcomes for society. In order to address this gap, we are exploring the applicability and effectiveness of existing approaches for explainable machine learning in public policy contexts. This project will examine the impact of improved explainability on policy outcomes related to increasing high school graduation rates, preventing adverse police interactions with the public, supporting mental health interventions to break the cycle of incarceration, and reducing long-term unemployment. Based on these results, we aim to generate a set of guidelines for governments, non-profits, and policymakers who are procuring machine learning systems, as well as for the researchers and practitioners who might be developing systems for use in public sector decision-making.

Improving the transparency, accuracy and effectiveness of U.S. Elections

Developing and deploying risk-limiting audits with continuous monitoring (Aaditya Ramdas and Michael I. Shamos)

According to a recent Gallup poll, 59% of Americans say they are not confident in the honesty of U.S. elections. Public trust in elections is integral to democracy. At the same time, manual election recounts when voting results are called into question cost time and money, and may not always be possible due to lack of paper trails. Even the process of deciding to perform a recount is generally ad hoc and triggered by post-election concerns. This project aims to use rigorous statistical tools to continuously monitor the audit and potentially end it early, as soon as it can be verified with high confidence that the announced result is correct. The increased efficiency of auditing will lower the time and money involved without sacrificing legitimacy, and encourage more states to normalize the creation of paper trails as well as sound post-election audit processes. Through better election transparency, we aim to support the restoration of trust in the democratic process, while maintaining accuracy in determining election outcomes.

AI for Good

Evaluating People’s Perceptions of Fairness in Machine Learning (Jason Hong, Nina Balcan, Ariel Procaccia, Hong Shen)

One critical question facing today’s ML/AI practitioners and public policy makers is: in a given AI system, which fairness notion is most acceptable to the people affected by the system? This project aims to develop better ways of presenting the results of machine learning models to lay people. This will be investigated using crowdsourcing (e.g., Mturk) to measure and aggregate people's perceptions of fairness in a scalable manner.

Improving Breast Cancer Diagnosis w/Interpretable Multimodal Machine Learning (Zachary Lipton, Adam Perer)

This project’s goal aims to increase the effectiveness of breast cancer screenings by using AI to enhance current processes for continually training radiologists using clinical data. This system will help doctors to discover their strengths, weaknesses, biases, and blind spots, providing feedback that can help them to improve their readings, as well as identifying cases that could be discussed in department reviews to better drive departmental education.

Counterfactual Risk Assessment for Improved Decision Support in
Child Welfare Services (Alexandra Chouldechova, Edward Kennedy)

While access to linked administrative data is increasingly available, it is difficult for child welfare workers to make systematic use of historical information about all the children and adults on a single referral call. Predictive analytics offers a way forward. By building risk assessment models using routinely collected administrative data, this project aims to better identify cases that are the most likely to result in adverse child welfare outcomes. These risk estimates could then be supplied to call workers in real time to help them prioritize cases for investigation and the offering of services.

Uncovering the Source of Machine Bias (Yan Huang, Param Vir Singh, Duyu Chen)

This project addresses machine biases by examining the potential sources of these biases. In particular, it seeks a deeper understanding of how human bias generate different separations for different groups, which then translate into different separations in machine-assisted decision-making, in cases where the data-collection process is endogenous. In the next phrase of the project, researchers will test their proposed method with real-world hiring data. They will collaborate with organizations that aim to automate their hiring and job performance evaluation practice.

Future of Work

Gigs, Risks, and Skills (Erina Ytsma and Geoff Parker)

This project studies the value of job security to low-skilled workers, the demand for skills training by blue-collar workers and employers, and the role that online labor market platforms can play in skills training and income risk mitigation. The first objective of the project is to estimate the demand for income risk mitigation for blue collar workers, and to assess the feasibility and profitability of online labor market platforms providing some level of income insurance. The second objective of the project is to understand the demand for general skills training by both the supply- and demand-side of the low-skilled labor market.

Entrepreneurship and the Platform Economy (Matthew Denes)

This project investigates whether platform-based jobs can incentivize and support entrepreneurship by providing nascent business owners with an additional source of income. In addition to characterizing the size and scope of the platform economy, this research group is evaluating the impact of gig employment on both income volatility and entrepreneurial activity. These insights could inform the development of policy to support entrepreneurship and protect gig workers.

Societal Futures

Better Videos for better Education (Michael Smith and Pedro Fereirra)

The project’s aim is to help create a center of excellence at CMU studying how technological change in the education market will impact students, educators, and society as a whole, and to use rigorous academic research to positively influence these outcomes. The researchers’ immediate goal is to analyze the impact of technological change in the context of video instruction. To the best of their knowledge, there are no large-scale systematic studies analyzing whether and how students learn with video.

Diversity and Inclusion in Open Source Software Development (Laura Dabbish, Jim Herbsleb)

This projects proposes empirical research on diversity and inclusion in open collaboration software projects to inform the design of socio-technical interventions to enhance participation of women in open source. The researchers’ intended broader impact is to extend the design of these environments and to engage in a dialog with the technical community entrusted with shepherding open source software. The goal of this work is to support design and management of open collaboration organizations so that these systems support effective career growth for females and inclusive participation by a diverse population.