Beyond the Oreo
HERB
HERB — the Home Exploring Robot Butler — found fame when he separated the Oreo creme from the cookie. He's now demonstrating a new and impressive skill.
HERB is learning to discover objects all by himself, thanks to a talented research team from Carnegie Mellon University's Robotics Institute.
Previously, HERB recognized objects via digital models and images of objects loaded into his memory — similar to the method virtually all roboticists use. But that practice will be inadequate once robots begin working routinely in homes, where they'll encounter hundreds if not thousands of objects.
"Manually loading digital models of every object of possible relevance simply isn't feasible," said Siddhartha Srinivasa, associate professor of robotics and head of the Personal Robotics Lab where HERB is being developed.
Robots therefore will need to discover and build digital models of objects by themselves. But doing so using computer vision alone is an intractable computational problem in a cluttered environment, Srinivasa said, so his team has enabled HERB to make use of additional information.
The Lifelong Robotic Object Discovery (LROD) process — dubbed HerbDisc — allows HERB to use color video, a Kinect depth camera and other non-visual information to access an object's location, size, shape, and even whether it can be lifted. With time and experience — much like a human infant using multiple senses — he can refine his models and discover objects on his own.
HERB has discovered items such as computer monitors, plants and — branching out from his favorite Oreos — other food items. He's been known to surprise even his own research team.
When students inadvertently forgot a few leftover lunch items in the lab — a pineapple and a bag of bagels — they returned the next morning to find that HERB had built his own digital models of both. And he could pick up each one.
HERB's definition of an object — something he can lift — relates directly to his main goal of assisting people in accomplishing the tasks of daily living, such as retrieving a needed item or microwaving an evening meal.
"Discovering and understanding objects in places filled with hundreds or thousands of things will be a crucial capability once robots begin working in the home and expanding their role in the workplace," said Srinivasa, who jointly supervised the research with Martial Hebert, professor of robotics.
Yesterday a creme-filled cookie, today a pineapple — tomorrow HERB could be preparing the meal.
HERB is a Quality of Life Technology Center project, a National Science Foundation Engineering Research Center dedicated to developing intelligent systems for everyday, independent living. The center is jointly operated by Carnegie Mellon and the University of Pittsburgh.
Related Links: School of Computer Science | Robotics Institute | HERB's webpage | HERB's-eye view of objects [video] | Read press release