Keylogging in Virtual Reality: Assessing Data Vulnerabilities via Motion-Position Sensors

Abstract

The integration of advanced motion-position sensors in Virtual Reality (VR) systems has significantly enhanced user interaction but has also raised serious privacy concerns. These sensors track the position and orientation of VR controllers, potentially revealing sensitive information about user activities. Existing research has demonstrated that sensor data can lead to privacy attacks, but the methodologies described are often vague and lack critical details. To address these gaps, we developed a comprehensive framework to log VR controller data without requiring user permissions for background operations. We have put significant efforts into understanding and replicating existing methodologies, as well as fixing missing details. Our approach includes identifying typing windows through detailed analysis of trigger presses, allowing for accurate capture and classification of user input. We then estimate 3D cursor positions, which are used to train a K-Nearest Neighbors (KNN) model for sentence reconstruction. Our method achieves a 96.1% accuracy rate in this task. By providing detailed replication methodologies and addressing gaps in existing work, our research benefits the VR security community by ensuring the reproducibility of VR privacy attacks, thereby aiding the development of advanced defense solutions.