Leveraging Human Psychology to Thwart Cyber Attacks
CMU researchers use cognitive AI to get into the mind of the person behind a cyberattack to build stronger defenses for future networks
By Stacy Kish
According to the Statista website, 5.5 billion malware (malicious software) incursions were detected around the world in 2022. Most cyber defense systems are structured around machine learning, a form of artificial intelligence (AI) that uses data and algorithms to do faster and complex information processing that humans have difficulty doing. It is often used to prevent unwelcome infiltration into a computer network or deflect engaging emails. This approach is effective at the surface level, but it doesn’t address the person engineering the attack.
Partnerships in innovation
Cleotilde Gonzalez, research professor in the Department of Social and Decision Sciences at Carnegie Mellon University, aims to address cybersecurity using cognitive modeling, a form of AI directed at using algorithms to imitate humans and to understand the psychology of the cyber adversary.
Gonzalez’s team has partnered with Peraton Labs, an applied research organization that addresses cybersecurity, electronic warfare, mobility, analytics and networking for government and commercial customers worldwide. Recently, Peraton Labs was one of five teams to receive an award from the Intelligence Advanced Research Projects Activity (IARPA), the research and development arm of the Office of the Director of National Intelligence
“Currently most cyber defenses in the world assume that there is a level of rationality of the attacker,” said Gonzalez. “Right now, none of the programs in existence have given much importance to the psychology of attackers. By bringing in human characteristics, the solutions we provide for cyber defenses will be more effective.”
Addressing the human element
As a leading researcher on cyber psychology research, Gonzalez plays a key role in this larger project. Gonzalez and her team are merging research on human decision biases with cognitive modeling, an AI approach founded more than five decades ago by Herb Simon and John R. Anderson, two CMU faculty members who have taught in the Department of Psychology at Dietrich College of Humanities and Social Sciences.
“We are presenting a process in which attackers’ decision making is transparent, not a black box,” said Gonzalez. “Our process operates as an open cognitive box, allowing us to explain why a person makes a certain choice.”
Gonzalez and her team will use cognitive models to emulate the behavior of the cyber antagonist. Throughout a cyberattack, the behavior of the person orchestrating the attack changes. Gonzalez’s team is using cognitive AI to replicate these behaviors at the cognitive level. By understanding their adversary at a psychological level, it may be possible to develop more effective network defenses.
Leveraging psychological insights for enhanced cyber defense
“We should be able to trace the information that the attacker leaves in a computer system — the breadcrumbs — to be able to determine whether they are falling into certain cognitive biases,” said Gonzalez. “This information has been known in the psychology literature for centuries, but we have never used it as a weapon of defense.”
The program will unfold in several phases. Throughout the process, the CMU team will work alongside colleagues at the University of Texas, El Paso, and the University of Washington, who are also partnering with Peraton Labs on the ReSCIND project.
During Phase I, the three university teams will conduct a series of surveys to identify different cognitive biases, such as loss aversion — a phenomenon where a real or potential loss is perceived by individuals as psychologically or emotionally more severe than an equivalent gain — or sunk cost fallacy — phenomenon whereby a person is reluctant to abandon a strategy or course of action because they have invested heavily in it. Gonzalez’s team will evaluate how these biases work in the realm of a cyber attack to develop traps, or cyber defenses, to engage the attacker and thwart the progression of the assault through a network system.
During Phase II, the CMU team will collaborate with their university partners to evaluate and capture data for each bias as a means of defense during a series of capture-the-flag experiments and develop cognitive models of attackers. During the final phase, the three university partners will input their cognitive models into CyberVan, Peraton Lab’s simulated network, to predict cognitive vulnerabilities of the adversary and demonstrate cyber defense strategies. Through this process, Gonzalez and her partners aim to develop new psychology-inspired approaches to protect an organization’s network.
“We are demonstrating that basic scientific biases exist in real-world, complex situations,” said Gonzalez. “We can use [these biases] to the benefit of our society by creating better defenses.”