Thomas Ploetz

Georgia Institute of Technology, United States

Title: “Agents Up Our Sleeves: From Passive Sensing to Autonomous Activity Recognition using (Agentic) AI”

Abstract: For decades, research into sensor-based Human Activity Recognition (HAR) has been a cornerstone of mobile and ubiquitous computing—evolving from early proofs of concept to integration across diverse application domains. While deep learning has rendered many traditional challenges manageable—leading some to claim HAR is a "solved problem"—real-world deployments still struggle with performance bottlenecks and the rigid, passive nature of current systems.

In the era of modern AI, two of the most persistent bottlenecks in sensor-based HAR can finally be addressed head-on. First, data scarcity is being overcome through cross-modality transfer, self-supervised representation learning, and synthetic data generation. Second, the rise of agentic AI is enabling a shift from reactive pipelines—which merely map continuous sensor streams to static, predefined labels—to dynamic, open-ended activity recognition. These agentic workflows allow systems to autonomously adapt to shifting requirements, emerging target activities, and varying sensor availability.

This keynote serves two purposes. First, I will offer a critical overview of where sensor-based HAR stands today, focusing on the practical challenges of deploying robust systems in complex scenarios. Second, I will examine the promises and pitfalls of agentic AI workflows in this domain. Drawing from emerging research, I will outline both a wishlist and a research roadmap for the next generation of sensor-based HAR systems that seamlessly integrate into practical applications.

Speaker Details: Thomas Ploetz is a Computer Scientist with expertise and decades of experience in Pattern Recognition and Machine Learning research (PhD from Bielefeld University, Germany). He works as a Professor of Computing at the School of Interactive Computing at the Georgia Institute of Technology in Atlanta, USA, where he leads the Computational Behavior Analysis research lab (cba.gatech.edu). There he is also the Associate Chair for Graduate Studies. His research agenda focuses on applied machine learning, that is developing systems and innovative sensor data analysis methods for real world applications. Primary application domain for his work is computational behavior analysis where he develops methods for automated and objective behavior assessments in naturalistic environments, thereby making opportunistic use of ubiquitous and wearable sensing methods. Main driving functions for his work are "in the wild" deployments and as such the development of systems and methods that have a real impact on people's lives. Thomas is a passionate educator teaching (very) large classes on Artificial Intelligence and Mobile, and Ubiquitous Computing on a regular basis at Georgia Tech, and worldwide through guest lectures and keynotes.

Thomas has been very active in the mobile and ubiquitous, including wearable computing community. He is co-editor in chief of the Proc. of the ACM on Interactive, Mobile, Wearable, and Ubiquitous computing technology (IMWUT), has twice been co-chair of the technical program committee of the International Symposium on Wearable Computing (ISWC), and was general co-chair of the 2022 Int. Joint Conf. On Pervasive and Ubiquitous Computing (Ubicomp). Thomas is a Distinguished Member of the ACM.