Human designers’ dynamic confidence and decision-making with an AI teammate
Decision-making assistance by an AI is only effective when human designers properly utilize AI input. Unfortunately, designers often misjudge the AI’s and/or their own ability, leading to erroneous reliance on AI and therefore bad designs. This project seeks to understand the evolution of designers’ confidence in both their AI teammate and themselves during AI-assisted decision-making, when the AI is no perfect. A cognitive study utilized a quantitative model of human confidence to explore how experiencing various and changing (without notice) AI performance levels and feedback affects designers’ confidences and consequently their decisions to accept or reject AI suggestions. One study involves a truss design task in which the participants are asked to consider the AI teammate’s suggestions and make the best next design move that leads to a truss with a high strength-to-weight ratio using a highly competent truss design AI algorithm. A different study uses game states of a chess board to explore the generality of the work.
The results for the truss design study show that designers’ confidence in an AI agent changes with poor, but not with good, AI performance. Interestingly, designers’ self-confidence initially remains unaffected by AI accuracy; however, when the accuracy changes, self-confidence decreases regardless of whether the AI gets better or worse in performance. Moreover, this work finds that designers tend to infer flawed information from feedback, resulting in inappropriate levels of confidence in both the AI and themselves. Confidence in the AI and self-confidence are also shown to affect designers’ probability of accepting AI input in opposite directions in this study. Overall, this work offers valuable insights that may enable detection of designers’ dynamic confidence and their consequent misuse of AI input in design. The chess study specifically shows that poor AI performance decreases human self-confidence which is found to drive the decision to accept or reject AI suggestions.
People
