First Person Vision, LLC-Quality of Life Technology Center - Carnegie Mellon University

First Person Vision, LLC


Share seeing of who, what and where a person is looking.


First Person Vision (FPV) is a transformative system that can monitor, record and assist people in their daily lives at work or at play in a truly symbiotic manner. First Person Vision captures a person's full field of vision and specific gaze-based intent to provide the user with intelligent cues and guidance, personal assistance, training, information or entertainment.

FPV features two cameras - a first, forward-looking camera that captures the full visual field of the person, and a second, non-active lighting camera with eye tracking software to pinpoint the exact location a person is looking. The system also incorporates audio and movement sensors.

Captured camera feed images and other recorded data are transmitted to an external computer for processing using computer vision and machine learning algorithms. Select data can be reported back to the user for guidance, training, therapy or assistance purposes. FPV can be integreated into ordinary eyeglasses, hats, helmets or other wearable headgear and then optimized for best shape, size and weight.


Core Benefits:

  • Captures user-centric, empathic data
  • Tracks behavior in specific contexts
  • Creates objective, externalized views of human behavior
  • Provides automated coaching or assistance
  • Supports optimal learning, shared entertainment and other uses

Targeted Applications:

  • Sports
  • Security
  • Therapy
  • Entertainment

"Real quality of life is the ability to live as independently as possible - to do what we can do for ourselves.  First Person Vision exemplifies this goal.  As a QoLT system, it seeks to provide real human-computer symbiosis by examining what a person wants to do and then helping that person independently do what he or she can do in the best way possible."

-- Takeo Kanade, Carnegie Mellon University



About the Research:

Recent efforts have focused on reducing the size of the wearable FPV device for improved user comfort and experience.  Other studies underway begin to examine new applications of the gaze-based information captured with FPV and advanced object detection and tracking in near real-time while wearing FPV.

Future work will explore the use of artificial intelligence and recognition systems for virtual coaching that can provide automatic and real-time decision making feedback to users. The system will immediately communicate advice and feedback to the user to optimize performance or everyday activities.


Learn More:


qolt trl 8

first-person-vision logo

First Person Vision Camera Feed Showing Driver Intent
FPV enabled view of driver intent.

first person vision sports application
FPV enabled view for sports training.

Wearable FPV prototype
Wearable prototype of an FPV device.

first person vision composite view
Composite camera view extraction.

Contacts

Brendan McManus,
CEO - First Person Vision, LLC
bmcmanus@cs.cmu.edu
(p): 412-963-0121


Michael Devyver,
Lead Researcher
devyver+@cs.cmu.edu


Kristen Sabol,
QoLT Communications and Media
ksabol@cs.cmu.edu
(p): 412-303-7396