Carnegie Mellon University

Risk and Regulatory Services Innovation Center

Sponsored by PwC at Heinz College

Deriving Enhanced Biometric Information from the Human Voice to Assist Forensic Analysis

Principal Investigator: Rita Singh, Senior Systems Scientist, CMU Language Technologies Institute

PwC Sponsors: John Sabatini, Principal, Financial Services Risk, Regulatory, and Financial Crime Technology Leader and Scott Greenfield , Partner, Advanced Risk and Compliance Analytics Leader 

According to research published in more than 400 scientific journals representing over 30 scientific fields, the human voice carries a wealth of information, including signatures of the speaker’s physical characteristics, physiological parameters, behavioral traits, medical condition, social and sociological background, geographical surroundings at the time of speaking, and objects in the speaker’s immediate environment, amongst other things. At present, technologies for extracting and interpreting this information from voice are seriously underdeveloped and often unavailable. This project will address the problem of specifically detecting those physical and physiological parameters that provide cues to fraud, such as stress (which may be indicative of lies, anger, and frustration) and in-vacuo impersonation attempts (detecting an impersonation attempt from an isolated voice recording, without reference to other voice samples from the same speaker). Potential beneficiaries include law enforcement agencies, emergency response agencies, and retail, banking, financial, and other agencies that employ call centers for substantial customer transactions.