Doctoral Student Behnam Mohammadi’s Research Explores LLMs and Human-AI Interaction

PhD student Behnam Mohammadi researches human-AI interaction, LLMs, and AI regulation. He develops LLM tools for business and created Pel, a language for coordinated AI agents, aiming to democratize AI for small businesses.

Behnam Mohammadi is a sixth-year doctoral student at the Tepper School of Business. After he graduates this year, he will join the University of Texas at Dallas in the fall of 2025 as a tenure-track faculty member of Quantitative Marketing in the Naveen Jindal School of Management. 

At the Tepper School, Mohammadi has studied human-AI interaction and experience, humans’ bounded rationality, and AI regulations. But LLMs (“lovely little minds” to Mohammadi, large language models to the rest of us) have become his main area of interest. Taken in by the idea that LLMs are general-purpose intelligence tools that are applicable across several domains, he pursued opportunities to study their strengths and weaknesses in practical business settings and developed LLM-powered tools and websites to automate employee and leadership training (for PNC bank) and business operations. His research methodologies draw from computer science, experiment design, psychology, and mathematical frameworks like game theory to understand LLM behavior.

Mohammadi is the lead author of "Regulating Explainable Artificial Intelligence (XAI) May Harm Consumers," published in Marketing Science. Mohammadi and his co-authors examine the impact of mandated transparency in AI, also known as "eXplainable AI" or XAI, on consumer welfare. While XAI policies like Europe’s GDPR often push towards requiring AI systems to explain their decisions, the paper challenges common assumptions about the role of transparency in market dynamics. Using sophisticated economic modeling (game theory), the research shows that in competitive markets, forcing full explanation isn’t always the best outcome for consumers. Sometimes, allowing companies to provide partial explanations can lead to better market dynamics and higher consumer welfare. In other words, rigid, one-size-fits-all regulations demanding maximum transparency might stifle competition and could paradoxically harm the consumers they aim to protect. This research provides vital, nuanced insights for policymakers crafting rules for the AI era, urging flexibility over rigidity.

In his other paper, “Explaining Large Language Models Decisions Using Shapley Values,” Mohammadi introduces a novel approach to interpreting LLM behavior by using Shapley values, a concept from cooperative game theory that allows for fair distribution of gains and losses to actors working together. This method allows for measuring the contribution of different parts of a prompt in influencing the LLM’s outputs. Mohammadi’s Shapley value method leads to the discovery of what he calls the “token noise” effect (tokens are the smallest unit of data AI processes, such as a single word), which is when LLM decisions are heavily influenced by inconsequential tokens (e.g., articles, prepositions, etc.). He also applies his method to investigate the framing effect, a type of cognitive bias, in LLMs.

Mohammadi uses a case study that examines a discrete choice experiment to show how the token noise effect can alter the outcomes of the LLM’s decisions. The results show that the decisions are heavily influenced by low-information tokens, which is problematic because it can lead to questions about the validity of using LLMs as a substitute for human decision-makers. The case study also shows that framing can influence LLM decision-making, but this effect is also influenced by the token noise effect. This leads to the conclusion that Shapley value analysis can lead to a more comprehensive understanding of LLM behavior.

Another study, “Creativity Has Left the Chat: The Price of Debiasing Language Models,” examines whether the process of aligning LLMs with human preferences hurts the model’s ability to be creative. AI companies often use techniques like reinforcement learning from human feedback to make AI safer and less prone to generating harmful, toxic, or biased content. But Mohammadi’s research, experimenting with Meta/Facebook’s Llama models, uncovers an unintended consequence: these alignment techniques significantly reduce the AI’s creativity and the diversity of its outputs. The models become more predictable, less likely to explore novel ideas, and tend to gravitate towards safe and common phrases. Imagine training a wildly imaginative artist to paint only within strict lines. They become more reliable but less innovative. This isn’t just a technical glitch; it’s a fundamental trade-off. For businesses, this means choosing the right tool for the job: a highly aligned, “safer” model might be best for predictable customer service responses while a less-aligned “base” model might be better for brainstorming creative ad copy or developing novel marketing personas.

Another area of Mohammadi’s research goes beyond viewing LLMs as passive tools, such as chatbots waiting for a prompt. Instead, it focuses on AI systems that can take initiative and proactively perform tasks on our behalf like managing our email inbox or even overseeing entire business functions. A central challenge here is coordination. How do we get multiple AI agents to work together effectively and reliably? Mohammadi’s groundbreaking solution is Pel, a programming language he designed from scratch specifically for LLMs. “Traditional languages were built with human programmers in mind,” he notes. “Pel is the first programming language designed for language models.” The language’s key strength lies in being simple enough for LLMs to generate and understand, yet powerful enough to express complex actions, decision-making (“control flow”), and communication among agents. This vision of coordinated AI agents isn’t just theoretical. Mohammadi is leading the BEACON project (Business Enhancement through Adaptive Coordinated Networks), supported by a BNY Foundation of Southwestern Pennsylvania fellowship from the Center for Intelligent Business at the Tepper School, in which he utilizes Pel to build an agentic AI framework aimed at democratizing AI for small and family-owned businesses. Many small businesses lack the resources for dedicated departments in marketing, finance, accounting, or HR. “The goal is to level the playing field,” Mohammadi emphasizes. “BEACON brings the sophisticated intelligence embedded in these AI models to small businesses to help them compete in an increasingly AI-driven world.” It’s like giving every small shop owner access to a team of expert consultants, powered by AI.

Mohammadi’s future work includes advancing the BEACON framework, enhancing Pel’s capabilities, and exploring how agentic AI can transform various business operations. He’s particularly excited about a future when AI takes care of mundane tasks, allowing humans to focus on more meaningful activities. “In five years, we might find it unbelievable that we once had to manually read and respond to emails in the same way that today it’s hard to imagine having to visit a store to buy software on CDs back in the ’90s.”