Wearable hand & arm tracking
Continuous hand–wrist tracking is key to enabling natural human–machine interaction, yet challenges remain to ensure robustness across users and sensor repositioning. This research combines established modalities (sEMG, IMU) with A-mode ultrasound to capture complementary muscle information for robust, low-power kinematic tracking and gesture recognition, towards reliable real-time interaction across multiple days.
- Wearable A-mode US for kinematic tracking — designed a four-transducer US armband interfaced with the WULPUS acquisition system; integrated motion capture and an instrumented glove for ground truth; led the data collections; developed XGBoost and CNN regression models, INT8-quantized and deployed on nRF52. 3-DoF: 5 subjects over 9 sessions, inter-session RMSE 11.1°. 23-DoF: 11k-parameter CNN, 29.1 ms inference on the nRF52 MCU, < 35 mW for sensing, inference and BLE result transmission.
- Concurrent EMG + US fusion — designed a flexible EMG PCB and wearable armband interfacing with the BioGAP acquisition system; implemented firmware-level synchronisation for concurrent EMG and US acquisition; designed and led data collections; developed a CNN-based EMG–US late-fusion architecture. 23-DoF: inter-session RMSE 10.6° vs. 12.0° (EMG only) and 13.1° (US only).
- US + IMU for real-time VR — led multi-day data collections, developed late-fusion CNNs with personalised recalibration, and built the real-time pipeline into a Unity VR environment. 5 subjects over 5 days: 80% / 77% inter-session accuracy; 88–96% online VR success after 5 min of recalibration.
Now: distributed EMG + US + IMU fusion for VR and robotic control.



