Mesh Merger data service

2023-09-25

An augmented Reality Collaborative game, like other augmented reality multi-user applications, needs a "mesh collider" to work. A mesh collider is a polygonal mesh which approximates the environment that can be used to resolve collisions between real and virtual objects. This makes possible to interact in a realistic way with the virtual environment.

In CHARITY, ORBK are developing a Collaborative AR game where the participants are equipped with modern smartphone with Lidar to facilitate the 3D scanning of fragments of the environment. ARKit, a technology for the environment acquisition, is used to provide a proper mesh collider
of a part of the scene. The idea is to create a data service to merge the meshes produced by ARKit coming from different smartphones into a unique mesh collider representing the complete environment. In other words, all mesh colliders are sent to the Mesh Merger service; that merges them and sends the results back to all the devices, this enables all the participants to interact with a continuously updated version of the mixed environment. This functionality significantly enriches the immersion of all participants in the game.


The video presents the idea and then the results of the first working prototype of the Mesh Merger. An user acquires some scans from the same device and sends them to the MEsh Merger, that creates the final one. The fact that the meshes comes from the same device is not a limitation of this demonstration, because no information is used to align and fuse the geometries sent. So, we expect the same results if the mesh colliders arrives from different devices. The results are shown on a web page since, at the moment of writing, the service is not integrated in the applications. The next steps of the development are to connect it with the Game Server of the Collaborative games, and improve the robustness of the results by identifying wrong alignments and discard such meshes.  

 

This video shown the Mesh Merger service, provides a glimpse of the same. Here, a data service for Augmented Reality collaborative applications is developed by two partners of the CHARITY project; the National Research Council of Italy (CNR) and Orbital Knight (ORBK). Orbital Knight is a game developer company based in Warsaw, Poland that is developing innovative games. CNR participates in the project with ISTI, the Computer Science Institute of the CNR. ISTI-CNR conducts research and development activities on data service for XR applications and other activities in the ambit of the project.