Learning-Quality of Life Technology Center - Carnegie Mellon University


A central goal of the perception plan is the use of data-driven approaches that rely on machine learning tools. The long-term objective of the project is to be able to learn predictive models of activities from training sensor data which can be used to predict actions and behaviors based on live data.

The techniques developed in this project are data-driven techniques to learn how to model and predict from a large corpus of training data. Long-term. The resulting models will be instrumental in key QoLT scenarios such as: Detecting anomalies in behavioral routines, which could be a valuable tool for early identification of individuals who are likely to become demented; Task completion capabilities, which are useful in guiding cognitively impaired individuals (e.g., TBI) through task completion. Can be used to re-learn how to perform tasks that have been lost due to injury or brain lesions; and task sequencing aids that would enable people with cognitive impairments to complete a few simple daily tasks.

This project is directly connected with the other components of QoLT: It uses similar techniques for prediction as those used in the Intelligent Driving QoLT Systems for route planning. It will use the Grand Challenge as training data to build prediction models. The behavior predictions will be use in future Cognitive Coach concepts and the use of motion prediction for safe planning in dynamic environments will be used the Active Home and PerMMa concepts that involve robots operating in the same space as users.

The use of machine perception technology relies on the fact that, with emerging sensing and storage technology, it is possible to acquire vast amounts of data which could conceivably cover all the user’s activities and environments. However, mining that data, for example to build statistical models of typical objects or activities, remains a formidable challenge. Specifically, a major obstacle is the lack of effective algorithms to focus the analysis of very large amounts of data on the few tasks relevant to the information to be modeled. This gap is addressed in the perception thrust through the development of novel machine learning tools to 1) extract useful predictive models from recorded training data (e.g., for predicting probable activities based on current observed activities) and 2) extract meaningful, task-specific statistical models from large sets of sensor data.

Project Team

  • Martial Hebert, Lead
  • Drew Bagnell
  • Byron Boots
  • Geoff Gordon
  • Carlos Guestrin
  • Jonathan Huang
  • Christoph Mertz
  • Kevin Peterson
  • Arne Suppe