PROJECT OMEN

The goal of PROJECT OMEN is to establish a scalable training environment for helping people to understand how they can be influenced and are influencing others in social media.  Based on a train as you work philosophy, PROJECT OMEN, will help people understand how people are impacted by disinformation, hate speech, and extremism online through immersive play.  The CMU IDeaS center is part of the large multi-institution group working on PROJECT OMEN.  IDeaS has several lines of effort in PROJECT OMEN reflecting the breadth of research, training, and outreach in the center.

Critical Thinking in the Information Ecosystem
Researchers: Dr. David Danks, Dr. Mara Harrell
Collaborators: Office of Naval Research, Cognitive Performance Group 

Critical thinking, reasoning, and information evaluation all play key roles in our ability to successfully navigate the complex information ecosystem. This project is developing a set of critical thinking education & training modules to improve people’s ability to recognize narratives, understand the role of evidence, learn from varieties of content, and design responses or interventions to increase their own social cybersecurity. These modules will be usable by people from a range of backgrounds, and for a range of purposes.

Recognizing and Countering Influence Campaigns and Online Hostility
Researchers: Dr. Kathleen M. Carley, PH.D. Students: Catherine King, Christine Lepird
Collaborators: Office of Naval Research, Intelligent Automation Inc., SoarTech, Air Force Research Lab 

Influence campaigns in the new information environment are conducted by altering either or both who is talking to whom (the social structure) and who is talking about what (the narrative).  This project is developing an immersive gameplay environment complete with a set of training materials,  educational content, data and lesson plans for helping people to identify, characterize, and counter (if needed) online hostile activity and actors.  Particular attention is payed to helping the trainee learn to: 1) recognize bot, troll, and state sponsored accounts; 2) recognize hate speech and online recruitment attempts; 3) understand how inauthentic actors help in the spread of disinformation and hate speech, and in recruiting to extremist groups; 4) recognize information maneuvers; 5) improve the effectiveness of their response to online hostility.

Media Manipulation

Media can be manipulated in many ways from the creation of deep fake videos making it look like some is saying or doing something they are not, to simpler changes that alter what issues are trending and so are prioritized to the reader. In some cases, media manipulation may promote social good, in others it is used to destroy reputations, spread hate speech, attack at risk minorities, or recruit people to extremist causes.  The goal of the media manipulation research is to understand how we can help individuals and communities be more resilient in the face of such manipulated media, and to identify the principles and guidelines for separating acceptable from unacceptable manipulation.  IDeaS has several lines of effort in media manipulation.

Ethics & Policy of Manipulated Media
Researchers: Dr. David Danks, PH.D. Student: Jack Parker
Collaborators: Accenture Labs, Carnegie Endowment for International Peace 

Manipulated media--particularly heavily edited or synthetic images and videos—present some of the most significant challenges to our social cybersecurity. Pictures and movies can be particularly compelling, both psychologically and emotionally, and so it can be hard to protect ourselves against false or misleading narratives and presentations. This project is examining ethical and policy issues around manipulated media, especially in the context of social media platforms: What ought they do about manipulated media? What can they do given existing policies and regulations? How might those policies change, or new ones develop, that enable more ethical responses to manipulated media?

Content Moderation & Inauthentic Actors
Researchers: Dr. Kathleen M. Carley, PH.D. Students: Joshua Uyheng, Daniele Bellutta
Collaborators: Accenture Labs, Office of Naval Research
Recent graduates: Dr. David M. Beskow 

This project has two foci: understanding the role of inauthentic actors and assessing alternative ways of countering their activity.  For the first foci, the goal is to understand how inauthentic actors are used to manipulate which media is being consumed, and what content is prioritized. In authentic actors include bots, trolls and cyborgs.  This project explores the signs that these actors are operating in isolation or in a coordinated fashion, the extent to which they are successful in spreading disinformation and hate speech, and the strategies they use for spreading such information.  For the second foci, the goal is to understand whether actor suspension or content moderations strategies are effective at stopping manipulation by these actors. This project considers what policy and remediation strategies are needed to counter online hostility, given that the technologies used for identifying inauthentic actors and identifying harmful messaging such as disinformation, conspiracy claims and hate speech, do not operate error free.

Automated Early Warning System for Cyber-Intrusion Detection

Cyber-attacks are a critical problem for society, and malware is used by attackers in a large class of cyber-attacks. The volume of malware produced is overwhelming, with hundreds of thousands of new samples discovered every day. The vast majority of malware samples are actually variants of existing malware, produced by transforming or obfuscating an existing sample in such a way that it can evade detection by end-point security products and other defenses.

The focus of this project is on automated detection of cyber intrusion within end-user computing devices, meaning any dedicated computing device from a smart sensing node, to a smart phone, all the way up to a dedicated laptop or desktop computer. A long-term trend over the past 30 years has been a steady increase in the number of independent end-user computing devices that are associated with an individual Human User. This trend is a natural consequence of the steady increase in capability and decrease in cost of portable consumer electronics.

End-user computing devices are under increasingly sophisticated attacks from state and non-state adversaries and the increasing number of computing devices per Human User further increases the threat cross-section for such attacks. This project proposes to add a security co-processor that uses Machine Learning techniques to identify behaviors that are substantially different from the learned normal behavior of the human user. The research will extend to exploring the ways to model the human user, ways for the security processor to interact with the human to improve security, and ways for the security processors on an local organization's devices can collaborate in order to improve the local organization's security.

Associates: Professor L. Richard Carley, Carnegie Mellon University, ECE; Professor Kathleen M. Carley, Carnegie Mellon University, ISR; Professor Diana Marculescu, University of Texas Austin, ECE; Office of Naval Research / DoD

Projects on Polarization

Project #1

Like playing mental chess, comprehending real-life political arguments presented in prose places unrealistic demands on people’s limited cognitive resources (e.g., working memory). So, instead of reasoning about an argument as stated, most people reason about their own overly simplified representations – summaries of an author’s argument created on the fly. The psychological processes that generate these simplified representations are highly susceptible to automatic and effort-minimizing biases such as confirmation bias.

This suggests a promising explanation for why people are so often biased when evaluating arguments that oppose their own views. Moreover, it points to concrete methods for reducing political polarization. In a preliminary study, Professor Simon Cullen (CMU) and Vidushi Sharma (Princeton) found that presenting an argument visually, using color, line, and shape to display its logical structure dramatically reduces confirmation bias compared to presenting the exact same argument in identical prose. Recently, they replicated the original findings, while also discovering target arguments where visual presentation did not reduce confirmation bias. What makes the difference? In current work, Cullen, Professor Daniel Oppenheimer (CMU, SDS/Psychology), Nick Byrd (CMU, Intelligence Community Postdoctoral Fellow), and Sharma investigate if the bias-reducing power of visual presentation emerges only when participants evaluate counter-attitudinal arguments that appeal to their own values. 

To test this hypothesis, we developed arguments for paradigmatically liberal political views that have the unusual feature of appealing to paradigmatically conservative values (e.g., a pro-choice argument that appeals to the value of law and order), and vice versa (e.g., a pro-life argument that appeals to the interests of marginalized groups). We also developed matched arguments that lack this feature (e.g., a pro-choice argument that appeals to gender equality, a pro-life argument that appeals to the values of small government). We predict that the depolarizing power of visual argument presentation is real, but that it only emerges when people evaluate arguments that ultimately “speak their moral language.” That is, to be persuasive, political and moral arguments must not only communicate their logical structures clearly; these structures must also ‘bottom out’ in values their intended audiences actually share.

Associates: Professor Simon Cullen, Carnegie Mellon University, Philosophy; Professor Daniel Oppenheimer Carnegie Mellon University, SDS/Psychology; Nick Byrd, Carnegie Mellong University, Intelligence Community Postdoctoral Fellow; and Vidushi Sharma, Princeton