Cognitive Coach: MemeXerciser-Quality of Life Technology Center - Carnegie Mellon University

Cognitive Coach: MemeXerciser

Supporting Individuals with Episodic Memory Impairment 

Cognitive Coach provides cognitive support to elders with episodic memory impairment (EMI).  EMI is an early symptom of Alzheimer’s Disease and has a huge impact on a sufferer’s (and his/her caregiver’s) quality of life. By capturing past experiences and distilling salient cues, individuals can be supported in remembering past experiences more effectively.

Pervasive lifelogging systems use wearable or embedded sensor technologies such as cameras, audio recorders, location trackers, and physiological sensors to passively and automatically record the user’s personal experiences. Lifelogging technologies allow people with EMI to automatically record, review, and thus regain an awareness of meaningful personal experiences in their lives to maintain their sense of self. To overcome the overwhelmingly large amount of data that is collected, and that can be both tedious and difficult to review, lifelogging systems need to be designed to select the most important information to present in an engaging way to the user.

We have found that the most helpful cues for triggering an episodic recollection of an experience is determined by the type of the experience (such as people-based, location-based, action-based, or object-based experiences). Lifelogging systems could be designed to automatically filter the data they collect using simple heuristics based on the type of experience such as selecting instances of people, faces, or conversations in people-based experiences to present as memory cues. While automation may be effective for significantly reducing the amount of information to review, the best cues within these categories for triggering recollection must also be either personally significant or distinctive. Unfortunately, there are no simple heuristics to determine significance or distinctiveness of content in the lifelog.

We have designed the MemeXerciser, a lifelogging system based on the Microsoft SenseCam system that passively captures people’s experiences, but unlike the traditional SenseCam system, our system uses a hybrid strategy combining automated approaches with the expertise of the caregiver to select the most appropriate content from the lifelog to present as memory cues. Our system also provides a memory-stimulating, self-guided interaction for the person with EMI to recollect their experiences without burdening their caregiver. A comparative evaluation of our system that uses this self-guided review approach supports the memory of people with EMI better and reduces the burden placed on their caregivers.

We have two main goals in this work: 1) to help people with episodic memory impairments improve their ability to remember past experiences, and 2) to reduce the burden placed on caregivers in helping cue and recall past experiences. The narrative created and experienced using MemeXerciser should help individuals remember more aspects of a given experience, than using a memory journal or replaying all the images (salient and not) from an experience (as is done when using the SenseCam alone). Rather than have a caregiver repeatedly and frustratingly have to cue particular memories, or have the caregiver repeatedly sit through and go over past experiences with a subject, MemeXercise simply allows the caregiver to create a narrative once, and then remind the subject to use the narrative, reducing overall caregiver burden.

Project Team

  • Anind Dey, Lead
  • Michael Pramuka
  • Rich Schulz
  • Matthew Lee
  • Akshay Goel
  • Jincao (Joyce) Xu

MemeX User Interface

MemeX User Interface 

[Click on image to enlarge]