ReachTouch: Touch Interaction with Distant Objects in XR Based on Hand Tracking

Abstract

This study proposes ReachTouch, a hand-tracking-based interaction technique for selecting distant objects in extended reality (XR). Direct touch interaction is intuitive but limited to objects within physical reach, while conventional raycast techniques can select distant targets but may suffer from cursor instability, particularly when a pinch gesture is used for confirmation. ReachTouch combines ray-based pointing with a natural touch-like gesture to provide intuitive and accurate interaction without handheld controllers. In ReachTouch, a cursor appears only when the user extends the index finger. After positioning the cursor over a target, the user moves the hand forward as if touching the object. Because this motion can shift the cursor, the system uses the cursor position immediately before the touch movement begins as the final selection point, reducing unintended selections. Two lightweight machine-learning models are used: a multilayer perceptron (MLP) for hand-pose recognition and a long short-term memory (LSTM) for touch-gesture detection. Both models operate locally using Unity Sentis. The MLP achieved 94.55% accuracy, while the LSTM achieved 98.10% accuracy in leave-one-subject-out cross-validation. An experiment with sixteen subjects compared ReachTouch with conventional raycast interaction in target-selection and map-based tasks. The experimental results suggested statistically significant differences under some conditions, suggesting that ReachTouch generally achieved shorter selection times and fewer errors compared to raycast. In the selection task, overall selection time was reduced by 26.5%, and by 35.2% for the smallest target. Its System Usability Scale score was also higher than that of raycasting. These results indicate that ReachTouch can improve distant-object interaction in XR, although further evaluation with 3D objects, multiple panels, and more complex operations are required.

The authors' abstract, as published at the source. Applied Sciences, 2026 · DOI ↗