This is Jeopardy!

The first Jeopardy! game ever to feature a non-human contestant was televised Feb. 14-16, 2011.

Watson — a Question Answering (QA) system developed by IBM and Carnegie Mellon University — competed against former champions Ken Jennings and Brad Rutter.

Watson won.

"The Jeopardy! problem is an important challenge for the field, because systems have to respond much more quickly than they ever have in the past," said CMU's Eric Nyberg.

Nyberg is a professor at CMU's Language Technologies Institute (LTI) at the School of Computer Science.

Imagine what this computer had to do.

Not only did it have to quickly recall knowledge from an array of topics. It also had to evaluate its own confidence level — to determine whether or not to answer immediately or pass on a particular question.

"In the Jeopardy! game, there's a big penalty if you give a wrong answer. It's much better to remain silent if you don't think you have the right answer," explained Nyberg.

"The only way for a computer to do that is to have a model of its own confidence in its answers. QA systems didn't really have that before Watson came along."

The Jeopardy! language was another challenge.

"It's very tricky, very creative. It requires forms of reasoning that are quite sophisticated — there's lots of use of metaphors, rhyming, puns and puzzles."

The field of QA initially focused on factoid questions, which only have one simple answer such as a date or a time.

Now researchers at LTI are pioneering more complicated algorithms. These will analyze a user's question and understand the underlying meaning of what they're asking, rather than just decipher the words they're using.

"Carnegie Mellon is a place where the research really focuses on real-world applications," said Nyberg.

Nyberg says the thing about Watson is "once you watch it play several games against humans, you'll notice very quickly that it makes mistakes humans wouldn't."

The match was reminiscent of another game that involved CMU alums developing a computer for IBM. In 1997, chess champion Garry Kasparov went up against IBM's Deep Blue.

The computer won.

"I think the best outcome for me in the final match would be one where Watson's strengths are showcased but also its weaknesses are evident, so that no one believes it is more intelligent or more powerful than it really is," he said.

"Still lots of work to be done to make it as smart as we are."

Seven other universities have joined CMU and IBM on the system development.

Homepage Story Archives