Carnegie Mellon University

Image of a hand on a smartphone

March 01, 2018

Carnegie Mellon Will Help Develop Camera To See Through Skin

NSF awards $10 million to researchers at five universities

By Byron Spice

Byron Spice
  • School of Computer Science
  • 412-268-9068

Carnegie Mellon University is part of a five-year, $10 million program sponsored by the National Science Foundation to develop a new type of camera that peers deep beneath the skin to help diagnose and monitor a wide variety of health conditions.

The interdisciplinary effort, led by Rice University, will combine advanced optics and sophisticated computation to make sense of light that penetrates the skin but scatters off internal tissues and anatomical structures. This will enable noninvasive bio-optical imaging at a cellular scale — something not possible with ultrasound, X-rays and other medical imaging technologies.

"Bioimaging today enables us to see just a few millimeters beneath the skin," said Srinivasa Narasimhan, a computer vision researcher and professor in CMU's Robotics Institute who is associate director of the new project. "We'd like to go five to 10 times deeper. With every additional millimeter we go, this technology becomes more useful. We hope that eventually it might reduce or eliminate the need for biopsies."

The NSF's newly announced Expeditions in Computing program includes four co-investigators at CMU and another seven at Rice, Harvard University, the Massachusetts Institute of Technology and Cornell University.

Images of body parts on the cellular level
Computational illumination and imaging approaches developed at Carnegie Mellon University produce these images of different body parts that show features up to 2 millimeters deep. Researchers expect new computational scatterography techniques will enable images of structures 10 times deeper or more.
Image of a hand over a smartphone
Light from a smartphone LED flash can travel through more than a centimeter of skin and soft tissue. This low-power, non-ionizing light is safe for biomedical imaging and cheap to produce, control and sense. (Image courtesy of A. Sabharwal/Rice University)
Graphic showing how light goes through the skin
On average, a photon of light scatters 10 times as it passes through each millimeter of skin or soft tissue. To use scattered light for medical imaging, scientists have to "de-scatter" the light using advanced computational techniques. (Image courtesy of A. Sabharwal/Rice University)

"Expeditions supports transformative research, and our goal is to create miniaturized, light-based microscopes for use in wearables, point-of-care, bedside diagnostics, ambulances, operating rooms and more," said Ashutosh Sabharwal, professor of electrical and computer engineering at Rice and the principal investigator on the grant.

The key to this effort is development of a technique called "computational scatterography." When light passes through the body, most of that light is scattered. That scattering can cause the tissue to glow, as when a flashlight is pointed at the palm of the hand. Until now, the scattered light was of little use for medical imaging. But new computer vision techniques allow scientists to make more sense of scattered light — essentially de-scattering the light by tracing the paths that photons took before they reached the camera.

Ioannis Gkioulekas, assistant professor of robotics, said CMU researchers have used similar techniques to see through fog, snow and heavy rain and now are applying those lessons to the much more complex task of bioimaging.

Narasimhan, Gkioulekas and their fellow CMU investigators — Artur Dubrawski of the Robotics Institute and Aswin Sankaranarayanan of the Electrical and Computer Engineering Department — will work with researchers at the University of Pittsburgh School of Medicine to explore possible cardiovascular and critical care applications, and with physicians at the Allegheny Health Network on skin cancer applications.

Rice's Sabharwal pointed to white blood cell count (WBC) tests, which require a finger prick or a blood draw, as an example of the project's potential impact. In the U.S., oncologists use millions of WBC tests each week to monitor chemotherapy patients.

"Imagine a wearable device no larger than a watch that uses sensors to continuously measure white blood cell count and wirelessly communicate with the oncologist's office," Sabharwal said. "The patient could go about their daily life. They'd only have to go to the hospital if there was a problem."

Sabharwal said it is crucial to understand that scatterography will not aid in managing just one or two health care problems.

"If we succeed, this isn't just one product," he said. "It's a platform technology that will be able to spinoff into many products that can be used in the care of nearly 100 health conditions."

Co-investigators at Rice include Ashok VeeraraghavanRichard BaraniukRebecca Richards-Kortum and Lin Zhong. Additional co-investigators include Cornell's Al Molnar, Harvard's Latanya Sweeney (formerly the Career Professor of Computer Science, Technology and Policy at CMU) and MIT's Ramesh Raskar. CMU's Disruptive Health Technology Institute sponsored previous work on bio-optical imaging by Narasimhan and his colleagues.