Invisible Threats: Uncovering UI Security Vulnerabilities in Augmented Reality Platforms

Abstract

Augmented Reality (AR) experiences place users within a user interface that allow for interactions with three-dimensional virtual content. Extensive research exists for 2D User Interface (UI) security, however the introduction of AR platforms introduces new security conflicts, particularly regarding how virtual content is handled and user interactions are managed. By utilizing existing knowledge of AR properties identified in prior work such as Same Space, Invisibility, and Synthetic Output, these potential vulnerabilities were investigated to address UI security in AR platforms. Such vulnerability experiments were tested on two leading AR platforms, the ARKit (Apple) and Oculus (Meta). In my research, ARKit vulnerabilities were tested with the use of an iPhone 12 and M2 Macbook Air, while Oculus vulnerabilities were tested with the Unity game engine on a Meta Quest 3S. To test each vulnerability, two independent components were tested as one app in addition to a third library which simulated multiple distinct entities interacting and potentially interfering with the user’s perception and input. It was found that Apple’s ARKit was susceptible to clickjacking attacks where two virtual objects (Cube1 & Cube2) are placed in the same coordinates and a hidden object would secretly receive the input. It was also found in both ARKit and Oculus, objects that were entirely transparent could still receive inputs from the user. Further, both ARKit and Oculus allowed fake, invisible user inputs generated by the computer to control virtual objects with no way to verify the validity of the inputs. The findings are ultimately problematic as they demonstrate how malicious AR applications could manipulate user interactions and perceptions in the background which could lead to unintended actions or a compromised user experience without the user’s awareness.