Ongoing project

EMG-Based Target Selection in XR

Data collection is complete. I am now exploring signal-processing and temporal-modeling approaches for EMG-based target selection in XR, using recorded motion as a training and evaluation reference.

Concept illustration of a person reaching toward an XR menu, with signals from two upper-arm EMG sensors leading to the selected Photos button. — Research concept: using EMG signals to select a button in an XR interface. Models are being explored using collected reaching data; this illustration does not depict a validated system.

Explore a recorded reach

Replay one selected successful development example and explore the recorded muscle signals. The saved model uses the complete S3/S8 reach to predict one target coordinate. This example does not represent average performance or live inference.

Open interactive example in a separate tab ↗

Research goal

This project investigates whether EMG can support target selection in XR by identifying which virtual button a user reaches to press. The aim is to infer the target from muscle activity without relying on camera-based hand tracking during use.

Data collection

I collected EMG and motion-capture data from 29 participants performing a seated target-selection task in XR.

Interaction: Participants used their right hand to select a central start point, then reached toward and selected a presented target.
Target layout: The task covered 27 target positions, combining three horizontal positions, three vertical positions, and three depths at approximately 60%, 75%, and 90% of each participant’s arm length.
EMG recordings: Eight EMG channels recorded muscle activity from the shoulder, upper arm, and forearm during the reaching and selection movements.
Motion and task references: Xsens recorded hand, arm, and torso motion at 60 Hz to provide a reference for model training and evaluation. Task logs captured target coordinates and selection times.

Data collection examples

Two views of seated participants wearing XR headsets and motion-capture sensors during the reaching study. — **Experimental setup**Seated participants performed the XR selection task while wearing EMG sensors and an Xsens motion-capture system.

Reaching movementA seated participant reaches toward a target in XR.Shown at 0.3× speed.

Oblique Unity scene view of spherical reaching targets arranged at different heights and depths, with layout guides visible. — **Spatial target layout**An oblique view of the Unity scene shows target positions across horizontal, vertical, and depth dimensions.

Front view of nine white spherical targets arranged in three rows and three columns. — **Front view**The front view shows the three-by-three horizontal and vertical arrangement, with the depth dimension illustrated in the oblique view.

Current progress

Data collection complete · Signal analysis and modeling in progress

I am exploring signal-processing and temporal-modeling approaches to infer target selection from EMG, using the recorded motion as a training and evaluation reference. Working with these recordings has highlighted two challenges: substantial signal noise and differences between participants. My current focus is on handling this variability while preserving information about the target-directed movement.

Beyond recognizing targets at the recorded positions, I am investigating whether the model can support new button locations through their spatial coordinates, without collecting additional EMG examples for every location. Target-recognition performance and generalization to unseen positions have not yet been established.