Carnegie Mellon University

CMU Design for America project assists people with language deficits.

September 19, 2018

Education App Uses Photos To Help People Form Proper Sentences

Students in the Carnegie Mellon University chapter of Design for America (DFA @ CMU) have developed a free mobile app that uses visual cues to help children and adults — particularly those with language difficulties — form meaningful, grammatically correct sentences. The education app, called Sentence Mosaics, uses photos and color-coded parts of speech to prompt users as they construct sentences.

The app is useful for elementary school students and for a wide range of children and adults who are on the autism spectrum, or who have developmental disabilities, hearing disorders, stroke-related language deficits, dementia and other problems with language, said Alyse Baker, a certified speech-language pathologist who has honed the system for decades.

"Alyse has used colored blocks, colored cards and foam tiles attached to a foam board with Velcro as aids, but that's not practical outside the classroom," said Mehar Sawhney, a DFA @ CMU member and a senior majoring in mathematics and physics. "We worked with her to translate her system into an iPad app that can be taken anywhere."

A group of DFA members, including Sawhney, computer science majors Elizabeth Board and Sashank Gogula, and cognitive science major Elizabeth La, worked with Baker to translate physical aids into digital ones.


Pictured above: DFA@CMU students Elizabeth Board, Elizabeth La, Sashank Gogula and Mehar Sawhney worked with Alyse Baker (center) to convert her language creation aid into the Sentence Mosaics iPad app.