Themes

We conceptualize Dynamic Decision Making research into two groups: Experience-Based Choice and Decision Making as Control. To learn about these two basic research themes see: Gonzalez, Fakhari, Busemeyer, 2017. All our basic and applied research programs rely on these two forms of dynamic decisions.

Experience-Based Choice

In sequential decision making, the goal is often to maximize long-term rewards while making selections among explicitly provided options. Under this theme, we investigate uncertainty, risk, context, and constraints of decision making including: time limits, feedback delays, and cognitive workload.

Initial contributions to understanding the process by which people make Decisions from Experience (DfE) emerged from the use of complex, dynamic decision making tasks (i.e., Microworlds). Microworlds represent highly complex tasks in computer simulations, such as dynamic resource allocation in real time and under time constraints. Currently, we also use simplified decision tasks in our research. For example, repeated choice tasks such as binary choice or bandit tasks.

In every-day life, we often make choices from experience in at least two ways:

Decisions from sampling:

where experience is acquired by exploring an environment without significant consequences, before consequential decisions are made. And

Consequential repeated decisions:

where we cannot sample the options, but rather learn while making choices and from past choices, perceiving the outcomes, and adjusting our decisions to the consequences.

At the DDMLab, we study both types of experience-based choice.

Decision Making as Control

Dynamic decision making can be also conceived as a continuous control task rather than discrete sequential choice. In a control task, the goal of the decision maker is to keep a system in “balance.” In fact, many real-life situations demand that a decision maker reduces the gap between the current state of a system and a target state. For example, maintaining a healthy body weight requires that decision makers balance diet and exercise, inventory control requires managers to balance demand and supply, and maintaining the earth’s CO2 within acceptable ranges requires a balance between emissions and absorptions. In other words, control tasks are common at the societal, organizational, and individual levels.

We have examined extreme simplifications of dynamic and complex tasks in contexts, such as global warming and inventory control. For example, we have used graphical representations concerning the accumulation of a quantity (e.g., CO2 levels) over time through an inflow (e.g., carbon emissions) that increases the quantity and an outflow (e.g., carbon absorptions) that decreases it.

At the DDMLab we study how people make choices in dynamic control problems in various contexts.

Model Comparison

Instance-Based Learning Theory (IBLT, Gonzalez, Lerch, Lebiere, 2003) is a general cognitive theory that represents computationally, the process by which people make decisions from experience. This theory has been used to construct a large number of models which apply to particular tasks.

To test and improve IBLT and other theories of decisions from experience, we rely on Model Comparisons. IBL models are judged according to how well they replicate human decisions.

IBL models have been among the top models in modeling competitions. For example, an IBL won a runner-up prize in a market entry choice prediction competition (http://sites.google.com/site/gpredcomp/). Under this theme of research we aim at improving the science of model comparison and theory testing.

Research Programs

Cyber Defense

Cybersecurity is a critical problem in our society. The assessment of the threats, vulnerabilities and successful defense in cyber space is surrounded by many challenges. Some of those challenges are technology-driven but many other challenges are human-driven. For example, how does an analyst evaluate the traffic data observed? How does an analyst assess the risk to the organization? How do defenders design networks to deceive the attackers?

At the DDMLab we develop a theoretical understanding of the socio-cognitive factors that impact decisions of attackers, defenders and end-users. The key advancement we provide to cybersecurity is the integration of models of human behavior to help develop better cyberdefense.

Our work in the area of cybersecurity started with initial research on models that would capture classification decisions made by a security analyst in simplified cyber scenarios. Given the high limitations on running experiments with human cyber analysts, we have largely relied on testbeds and subjects that range in their level of experience on cybersecurity. We have focused on modeling the decisions made by attackers and defenders and compared the predictions from these models against human behavior in equivalent tasks.

More recently, we have been interested in developing adaptive and personalized cyber defense strategies based on principles of human deception. Game Theory and particularly Stackelberg Security Games (SSGs) optimizations led to the development of algorithms that have been tested in the context of physical security (e.g., combating illicit poaching in national parks). SSGs are attacker-defender games, that solve the problem of allocation of limited defense resources in order to minimize the losses of the defender. We have advanced this line of research in AI and Computer Science by proposing the use of signals to deter the attacker instead of the costlier strategy of reallocating defense resources.

Our approach is described in Gonzalez et al 2020 and illustrated in this figure:

Phishing Detection and Training

Phishing emails continue to evade automated detection and are a major way in which attackers get into various organizations’ networks. Phishing is a form of deception that relies largely on social engineering tactics, where attackers take advantage of human weaknesses such as: reacting to familiar senders, to immediate requests, and to emotional requests. Based on IBLT, we know that these phishing classification decisions are influenced by the type of experiences people have. For example, end-users make decisions based on the similarity of features of a current email to features of emails they have received in the past. Importantly, phishing emails often mimic benign emails—meaning that decision makers, who are influenced by typical memory effects such as recency and frequency of past instances, are susceptible to the cognitive biases that emerge from these very processes.

An IBL model was built to emulate end-used classification decisions of emails (i.e., as phishing or benign) and the results from this model were compared to the classification decisions from humans in an email processing task. We are working on building training scenarios that take advantage of the insights of our model.

Human-Machine Collaborations

IBLT is in a theory of individual decision making, and groups learn by the learning of individual group members. We have demonstrated that group effects and dynamics can be captured by the aggregation of individual members of a group and their interdependencies. We first expanded IBL models to capture the interdependencies in social dilemmas the resulting effects on the dynamics of cooperation in dyads.

In this work we use the same essential elements of IBLT with an added dynamic function representing the social value (i.e., the regard that each individual has for the other’s outcomes). We use the Prisoner’s Dilemma and other 2-person games (e.g., Rock-Paper-Scissors). We are also expanding this idea to large networks of various structures.

We are also currently advancing the concept of interactions between pairs of individuals to elucidate interactions between humans and machines in groups. For example, we have constructed an architecture in which IBL models develop Cognitive Machine Theory of Mind by observing other agents.

Current Research Sponsors

Intelligence Advanced Research Projects Activity (IARPA)

Impact of Cognition on Cyber Behavior

The IARPA ReSCIND program aims to improve cybersecurity by understanding how human cognition impacts cyber behavior and could affect cyber actors’ success in network attack activities. Specifically, well-established cognitive patterns, such as loss aversion and the representativeness bias, will be investigated as potentially mitigating factors in the efficacy of cyber attack behavior. Coty Gonzalez is the Principal Investigator of this project.

National Science Foundation

Institute for AI-enabled Societal Decision Making (AI-SDM)

Coty Gonzalez is the Co-Principal Investigator and Institute's Research Co-Director. This collaboration brings together AI researchers and social scientists to develop tools for societal challenges.

Army Research Office

Multi-University Research Initiative (MURI): Cyber Autonomy through Robust Learning and Effective Human-Bot Teaming

Coty Gonzalez is a Co-Investigator working in collaboration with Somesh Jha, University of Wisconsin; Lujo Bauer, CMU, and other universities in the USA and Australia to investigate how humans and bots can collaborate to develop better cyberdefense strategies.

Air Force Office of Scientific Research

TRUST’M: Behavioral Models, Methods & Metrics for Trust Establishment, Maintenance and Repair in Human Machine Co-Training

Coty Gonzalez is the Principal Investigator for this project that aims at developing and advancing theories, methods and technologies to study and address the capabilities that promote and maintain mutual understanding between humans and machines.

Former Research Sponsors

(2019-2024) Defense Advanced Research Projects Agency (DARPA). DARPA ASIST: An Integrated Theory of Human-Machine Teaming. In collaboration with Anita Woolley and Henny Admoni, to develop an Integrated Theory of Human-Machine Teaming that brings together research on individual and team cognition into a Socio-Cognitive Architecture that integrates with a Machine Theory of Mind (M-ToM).

(2015-2024) Army Research Office. Multi-University Research Initiative (MURI): Realizing Cyber Inception: Towards a Science of Personalized Deception for Cyber Defense. In collaboration with Milind Tambe, Harvard University; Christian Lebiere, and Lujo Bauer, CMU, and others to develop adaptive and personalized cyberdefense strategies.

(2020-2022) Air Force Research Laboratory (AFRL). Establishing the Science of Understanding for Effective Human-Autonomy Teaming.

(2021-2022) CyLab Seeding Grant. Personalized Phishing Detection Training Using Cognitive Models.

(2017-2021) Army Research Office. Scaling up Models of Decisions from Experience: Information and Incentives in Networks. In collaboration with The Statistics and Data Science Department CSAFE group, at Carnegie Mellon University.

(2009-2014) Army Research Office- Multi University Research Initiative. Human detection of cyber-attack. In collaboration with Peng Liu, Penn State University; Arizona State, George Mason, and North Carolina State University.

(2009-2014) Defense Threat Reduction Agency. Development of cooperation and conflict in social interactions. In collaboration with Christian Lebiere, at Carnegie Mellon University.

(2009-2011) National Institute of Occupational Safety and Health. Training Dynamic Decision Making in Mine Emergency Situations.

(2006-2009) National Science Foundation. Hypothesis Generation & Feedback in Dynamic Decision Making. In collaboration with Rickey Thomas, Assistant Professor of Cognitive Psychology at the University of Oklahoma & Robert Hamm, Professor of Family and Preventive Medicine and Director of Clinical Decision Making Program at the University of Oklahoma Health Sciences Center.

(2005-2009) Army Research Office. Training Decision Making Skills. In collaboration with Alice Healy, Professor of Psychology at the University of Colorado at Boulder; Lyle Bourne, Professor of Psychology Emeritus and Faculty Fellow of Institute of Cognitive Science at the University of Colorado at Boulder; & Robert Proctor, Professor of Psychology at Purdue University.

(2002-2009) Army Research Laboratory. Cognitive Process Modeling and Measurement in Dynamic Decision Making. In collaboration with Mica Endsley, President at SA Technologies.

(2007-2008) Richard Lounsbery Foundation. PeaceMaker-Based Research for Decision Making and Diplomacy. In collaboration with Kiron Skinner, Associate Professor of Social and Decision Sciences and History & Laurie Eisenberg, Associate Teaching Professor of History at Carnegie Mellon University.

(2008) Argonne National Laboratory. Determinants of Public Confidence in Government to Prevent Terrorism. In collaboration with Ignacio Martinez-Moyano and Michael Samsa of Argonne National Labs.

(2005-2006) Institute for Complex Engineered Systems, Carnegie Mellon University, PITA program. Learning from the Past: Improving Estimation of Future Construction Projects. In collaboration with Burcu Akinci, Associate Professor of Civil and Environmental Engineering at Carnegie Mellon University.

(2002-2006) National Institute of Mental Health, training grant. Training in Combined Computational and Behavioral Approaches to Cognition. In collaboration with Lynne Reder, Professor of Psychology and Director of Memory Lab at Carnegie Mellon University.

(2001-2006) U.S. Army Research Laboratory. Cognitive Process Modeling and Measurement in Dynamic Decision Making. In collaboration with Mica Endsley, President of SA Technologies.

(2001-2006) Office of Naval Research, Multidisciplinary University Research Initiative. Cognitive, Biological and Computational Analyses of Automaticity in Complex Cognition. In collaboration with Marcel Just, D.O. Hebb Professor of Psychology and Co-Director of Center for Cognitive Brain Imaging (CCBI) at Carnegie Mellon University; Walter Schneider, Professor of Psychology at the University of Pittsburgh; & Poornima Madhavan, Assistant Professor of Psychology at the Old Dominion University.

(2004) Office of Naval Research, Small Business Innovation Research. Automated Communication Analysis for Interactive Situation Awareness Assessment. In collaboration with Mica Endsley, President of SA Technologies & Cheryl Bolstad, Senior Research Associate at SA Technologies.

(2001) Carnegie Mellon University Berkman Faculty Development Fund. Perception and Attention Effects on Learning Dynamic Decision Making Tasks.