Semantic Geotagging-Silicon Valley Campus - Carnegie Mellon University

Project: Semantic Hyperlink-Augmented Geotagged Social Media for First Responders

Problem: As mobile devices grow more ubiquitous and powerful, first responders (professionals, partially trained CERT members, members of the public and spontaneous untrained volunteers) are using social media, SMS, Twitter, and geotagged images to produce a large volume of information, some addressed to incident commanders and the rest just pumped into the ether. Our project adds semantic geotagging at the device end, so that new messages are properly situated in the context of the ongoing situation, both geographically, temporally and in appropriate relationship to existing information, thus providing an intuitive local and global organization which benefits both incident commanders and the responders themselves.

Description of Technology: Simple, carefully designed dynamic forms that help responders leverage mobile smart devices to increase the quality of their situation reports by adding context to their messages, and simultaneously help emergency managers make sense of the firehose of reports. A restricted, but semantically powerful, link vocabulary, coupled with these semi-structured input forms, will enable first responders and commanders to collect valuable information and collaboratively construct a structured hyperlinked knowledge base integrating both location-specific and global situational awareness. In-device sensors will location-tag and time stamp every piece of information, enabling automated association of information with logged events in a geospatial Common Operating Picture (COP).  In-the-field navigation of a potentially large corpus of information is simplified by geo location and the restricted hypermedia links (based on a cognitive theory of conversation), allowing easy access to relevant situational information on small mobile devices, and supporting intuitive access to a deeper level of information within the larger COP.

Device images

Research: We developing and plan to test a prototype system for disaster response, working closely with professional responders. Research includes the efficacy of a restricted link vocabulary for facilitating navigation of a large corpus of information, the most appropriate specific vocabulary for this application, user interface design (including text, images, voice, and gesture) to enable the use of such a tool in difficult situations (e.g., responding to a fire), and the overall efficacy of such an approach for creating shared situational awareness among responders, incident commanders and EOC managers. Submitted semi-structured information can be further processed by a combination of language-based text analysis, statistical methods, geospatial visualization and human input to integrate, filter, aggregate and re-distribute relevant information to users in the field and at mobile command centers.

How It Is New and Different: Current use of social media in disaster response provides a primitive organization of information making it difficult to find information relevant to a user’s task and location; it does not enable a complex, multifaceted conversation among distributed responders, i.e., these approaches are “one-way” - users sending messages do not see the larger picture; instead, multiple users send the same information repeatedly, leaving command center workers to make sense of it all. Our approach supports the simple and natural creation of geo-tagged and semantically typed conversational threads, which have been demonstrated in previous web applications to provide an intuitive structure for effectively navigating a large, complex, dynamically evolving corpus of information.

For more information, contact