Advancing Cloud-based XR Applications by Decoupling the Physics Engine

2024-04-10

Immersive Extended Reality (XR) experiences, blending virtual and physical worlds, demand robust simulations to ensure a seamless user experience. Particularly in mobile XR environments, efficient physics algorithms play a crucial role in rendering 3D object animations and interactions while maintaining interactivity. However, the limitations of standalone XR headsets, such as processing power constraints, often result in simplified physics models, impacting realism and user immersion. To address these challenges, ORAMA presents a significant leap forward for XR cloud applications, with its use case application decoupling the physics engine from the main Unity XR application, and hosting it as a containerized service in cloud servers. This blog article delves into the significance of this advancement for XR cloud applications, the main challenges encountered, and highlights the architecture of the proposed XR pipeline.

The decoupling of the physics engine from Unity pipeline allows the offloading of intensive physics computations to dedicated cloud resources, enabling untethered XR experiences with enhanced realism and interactivity. This approach not only overcomes the processing limitations of mobile XR devices, but also minimizes the total frame production time on untethered XR devices, and facilitates multi-user XR scenarios without compromising performance. Moreover, the containerized nature of the service allows for seamless deployment and scalability, making it well-suited for cloud environments. Ultimately, this innovation paves the way for more immersive and collaborative XR applications, revolutionizing the way users interact with virtual environments.

One of the primary challenges lies in ensuring optimal Quality of Experience (QoE) under real-time network conditions. Synchronizing the simulation state across distributed environments introduces complexities, requiring sophisticated networking protocols to mitigate latency and bandwidth issues. Additionally, decoupling the physics engine from the main Unity XR application necessitates extensive code modifications and architectural considerations, impacting pipeline functionalities. Furthermore, maintaining a persistent simulation state independent of user actions adds another layer of complexity. Overcoming these challenges demanded a holistic approach addressing both technical and design considerations.

The proposed XR pipeline adopts an N-1 client-server architecture, separating the Graphics unit from the Physics unit The Graphics unit encompasses the graphics rendering pipeline, while the PhyS Physics unit hosts the physics engine. Communication between these units is achieved using a lightweight network connection, via a cloud based Relay server. The pipeline's modular design simplifies development and maintenance, offering scalability and flexibility for multi-user XR experiences. In this architecture, once Physics computations are completed, the produced new transform data required is sent to the Graphics unit to synchronize the two separated units. Multi-user sessions are initiated and managed by the Graphics unit, with the Physics unit serving as a centralized hub for physics computations. As such, user interactions, such as object manipulations, are transmitted to the Physics unit, which orchestrates physics-based movements and synchronizes the results across all participating Graphics units in the multi-user session.

The decoupling of the physics engine into a cloud-native service marks a significant milestone in advancing XR cloud applications. By strategically offloading physics computations to dedicated cloud resources, this approach enhances realism, interactivity, and scalability in XR experiences. Despite challenges in synchronization and architecture, the proposed XR pipeline offers a promising solution for delivering immersive and collaborative XR applications. Looking ahead, further optimizations and integrations hold the potential to redefine the standards for multi-user XR engagements, ushering in a new era of virtual experiences.

Authored by Dr. Antonis  Protopsaltis ORamaVR