Carnegie Mellon University

Optimizing Infrastructure Maintenance with AI and Extended Reality - Pengkun Liu

liu-2.png

The average age of Americans in the labor force continues to increase—and at some point, their wealth of knowledge will need to be imparted upon younger workers. This is especially important when it comes to maintaining the nation’s aging critical infrastructure. 

To learn more about this and to propose solutions, PhD candidate Pengkun Liu is researching both how the “Silver Tsunami” aging population phenomenon impacts the increasingly problematic transportation and water infrastructure, and how to most efficiently transfer knowledge to a new generation.

While a challenging dilemma, this problem provides many opportunities according to Liu who focused his research on combining human and artificial intelligence. “This allows us to share information about inspecting and operating crucial civil infrastructure and industrial plants.”

He developed a concept, backed by large language models, to create a cognitive assistant capable of capturing, summarizing, and imparting the experiences of senior workers to newer members of the workforce. 

“Ultimately, the goal is to enhance the efficiency and safety of critical infrastructure systems, assisting in alleviating and navigating the impacts of the Silver Tsunami by reusing operational experiences,” Liu states.

...this problem provides many opportunities, according to Liu, who focused his research on combining human and artificial intelligence. “This allows us to share information about inspecting and operating crucial civil infrastructure and industrial plants.”

liu-demo.jpg

He believes that the design and development of meticulously-crafted extended reality (XR) platforms can aid in recording, explaining, and reusing operational experiences for training purposes. XR platforms can train human personnel while updating artificial intelligence (AI) algorithms in adapting to diverse civil infrastructure inspection and operational scenarios. “These platforms could incorporate large language models to facilitate seamless and intuitive human-computer interactions,” he says.

To support this research, Lui recently participated in projects that model the complex interactions between intelligent agents, human operators, and their physical environments. So far, the research has been employed in data-driven operation and maintenance protocols for critical facilities, including bridges and water treatment plants. 

“I’m expanding this concept to generate data-driven operational process simulations, which serve as cognitive virtual assistants. These applications leverage the capabilities of large language models in understanding and responding to human inputs.” 

Lui believes that the simulations will eventually be able to serve as true assets to public safety. He states they’ll also increase the efficiency of water and energy systems while benefitting societal welfare—spotlighting the time and resource waste in producing water and electricity for future communities. 

“AI-assisted systems can optimize maintenance protocols for vital facilities like bridges and water treatment plants. This improved maintenance and operation lead to safer public infrastructure.”