Project Theme 1: Pervasive Sensing
Vibrations of Everything
Physical and physiological information is essential for indoor human monitoring applications. In order to capture such information non-intrusively, we utilize/design sensors that can capture the vibration induced by people/body parts to infer a variance of activities (over different scales, both infrastructure and wearable) in the physical world. We explore new opportunities to utilize complementary information over different scales (on-body, room, city, etc.) to achieve accurate inference.
In-Home Activities of Daily Living (ADL) Monitoring
Latest Paper:
Zhizhang Hu, Yue Zhang, Tong Yu, and Shijia Pan. "VMA: Domain Variance-and Modality-Aware Model Transfer for Fine-Grained Occupant Activity Recognition." In 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), pp. 259-270. IEEE, 2022.
Retainer-Formed Sensor Capturing Oral Activities-Induced Vibration
Latest Papers:
Zhizhang Hu, Amir Mohammad Radmehr, Yue Zhang, Shijia Pan, and Phuc Nguyen. "IOTeeth: Intra-Oral Teeth Sensing System for Dental Occlusal Diseases Recognition." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, no. 1 (2024): 1-29.
Shijia Pan, Dong Yoon Lee, Jun Ho Lee, and Phuc Nguyen. "TeethVib: Monitoring Teeth Functional Occlusion Through Retainer Vibration Sensing." In 2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 92-96. IEEE, 2021.
Repurpose Vibrotactile Intervention for Fine-grained Muscle Profiling
Latest Paper:
Shubham Rohal, Shreya Shriram, V. P. Nguyen, and Shijia Pan. "FinePose: Fine-Grained Postural Muscle Profiling via Haptic Vibration Signals." In Proceedings of the 2022 Workshop on Body-centric Computing Systems, pp. 19-24. 2022.
Award: Best Poster Award IPSN'22, NSF iCorp
🚀 Alumni Spin-off: PosTrue
Structure-Augmented Intelligent Surfaces
To push the boundary of ambient information inference via surface sensing, we further explore methods and apparatus for augmenting the ability to acquire and recognize human-induced surface deformation by changing the physical properties of the surface.
Augment rigid surface with LEGO® bricks enabled metasurface
Latest Paper:
Yue Zhang, Shikha Patel, Dong Yoon Lee, Paolo Celli, Amelie Bonde, and Shijia Pan. 2023. Poster Abstract: LEVO: LEGO® Bricks Enhanced Single-Point Vibration Sensing for Occupant Monitoring. In The 21st ACM Conference on Embedded Networked Sensor Systems (SenSys ’23), November 12–17, 2023, Istanbul, Turkiye. ACM, New York, NY, USA, 2 pages.
Award: Best Poster Award SenSys'23
Augment elastic surface with Miura-Ori Origami metastructure
Latest Papers:
Shubham Rohal, Dong Yoon Lee, Carlos Ruiz, Joshua Zhang, Jonathon Fagert, Jun Han, and Shijia Pan. "Don’t Crosstalk to Me: Origami Structure-Augmented Sensing for Scalable Surface Pressure Monitoring." In Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems, pp. 493-506. 2024.
Shubham Rohal, Carlos Ruiz, Yue Zhang, and Shijia Pan. "MOOCA: Muira-Ori Origami-Based Configurable Shelf-Liner for Autonomous Retail." In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 776-777. 2022.
Patent: "Programmable smart-sensing shelf liner apparatus for autonomous retail." U.S. Patent Application No. 18/487,059.
Award: Best Demo Award SenSys'22
Augment rigid surface with controlled propagation path
Latest Paper:
Yue Zhang, Carlos Ruiz, Shubham Rohal, and Shijia Pan. "CPA: Cyber-Physical Augmentation for Vibration Sensing in Autonomous Retails." In Proceedings of the 24th International Workshop on Mobile Computing Systems and Applications, pp. 8-14. 2023.
Project Theme 2: Autonomous Adaptation via Cross-Modal Intelligence
Cross-Modal Association
Multiple co-located sensing modalities capture the same physical event from different aspects and provide complementary information for inference. However, to accurately associate signals from different modality capturing the same event is challenging, expecially when the modalities are significantly different.
Cross-Modal Association Using Humans as Shared Context
Latest Papers:
Yue Zhang, Zhizhang Hu, Uri Berger, and Shijia Pan. "CMA: Cross-Modal Association Between Wearable and Structural Vibration Signal Segments for Indoor Occupant Sensing." In The 22nd International Conference on Information Processing in Sensor Networks, pp. 96-109. 2023.
Yue Zhang, Zhizhang Hu, Uri Berger, and Shijia Pan. "Poster Abstract: Integrating On-and Off-body Sensing for Young Adults Failure to Launch (FTL) Behavior Profiling." In Proceedings of the 22nd International Conference on Information Processing in Sensor Networks, pp. 320-321. 2023.
Yue Zhang, Zhizhang Hu, Susu Xu, and Shijia Pan. "AutoQual: Task-Oriented Structural Vibration Sensing Quality Assessment Leveraging Co-located Mobile Sensing Context." CCF Transactions on Pervasive Computing and Interaction 3 (2021): 378-396.
Award: Best Poster Award Runner-Up IPSN'23
Adaptation in Sensing
Data acquisition quality directly affects the information representative and the model accuracy in applications of real-world deployed cyber-physical systems. We combine physical and data-driven knowledge to design metrics and methods to assess the signal and dataset quality for particular sensing tasks.
Cross-Modal IoT System Auto Configuration
Latest Papers:
Shubham Rohal, Yue Zhang, Carlos Ruiz, and Shijia Pan. "AutoLoc: Autonomous Sensor Location Configuration via Cross-Modal Sensing." Frontiers in Big Data 5 (2022): 835949.
Yu, Tong, Yue Zhang, Zhizhang Hu, Susu Xu, and Shijia Pan. "Vibration-based indoor human sensing quality reinforcement via Thompson Sampling." In Proceedings of the First International Workshop on Cyber-Physical-Human System Design and Implementation, pp. 33-38. 2021.
Lixing He, Carlos Ruiz, Mostafa Mirshekari, and Shijia Pan. "SCSV2: physics-informed self-configuration sensing through vision and vibration context modeling." In Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, pp. 532-537. 2020.
Adaptation in Learning
The heterogeneity of cyber-physical systems brings challenges and opportunities for various applications. Our goal is to explore new methods to combine the complementary characteristics of multiple sensing modalities for accurate fine-grained learning with limited (labeled) data.
Domain Variance- and Modality-Aware Model Transfer
Latest Paper:
Zhizhang Hu, Yue Zhang, Tong Yu, and Shijia Pan. "VMA: Domain Variance-and Modality-Aware Model Transfer for Fine-Grained Occupant Activity Recognition." In 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), pp. 259-270. IEEE, 2022.
Project Theme 3: Networked Collaborative Intelligence
AIoT Enabled Augmented Air Quality Monitoring
San Joaquin Valley's (SJV) residents face challenges of air pollutant exposure -- the basin terrain of the SJV traps air pollutants easily. At the same time, the local land use (e.g., industry, agriculture) results in a high variation of pollutant sources of PM2.5. We develop Artificial Intelligence of Things (AIoT) with both advanced sensing and learning algorithms to tackle the challenge with low-cost and scalable solutions.
Latest Papers:
Shangjie Du, Zhizhang Hu, and Shijia Pan. 2024. GraPhy: Graph-Based Physics-Guided Urban Air Quality Modeling for Monitoring-Constrained Regions. In Proceedings of the 11th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys ’24), November 7–8, 2024, Hangzhou, China.
Zhizhang Hu∗, Shangjie Du∗, Yuning Chen, Xuan Zhang, Wan Du, Asa Bradman, and Shijia Pan. 2023. Poster Abstract: Enhancing Fault Resilience of Air Quality Monitoring in San Joaquin Valley: A Data Equity Analysis. In Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems (SenSys ’23), November 12–17, 2023, Istanbul, Turkiye. ACM, New York, NY, USA, 2 pages.
Wang, Gang, Shijia Pan, and Susu Xu. "Decoupling the unfairness propagation chain in crowd sensing and learning systems for spatio-temporal urban monitoring." In Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pp. 200-203. 2021.
Award: Best Paper Award Runner-Up BuildSys'24
Smart Farm Data Quality Assessment via Spatial Consistency Modeling
Coming Soon!
Latest Paper:
Yue Zhang, Abdias Tellez Benitez, Reza Ehsani, and Shijia Pan. "DaQual: Data Quality Assessment for Tree Trunk Relative Water Content Sensors in a Pomegranate Orchard." In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 1096-1101. 2022.