Revealed during Micoroft’s annual Ignite 2021 conference, Mesh is a new mixed reality experience that set to shape how people work and socialize online.
As a platform, Mesh is the result of years of Microsoft research and development in areas ranging from hand and eye tracking and HoloLens development to creating persistent holograms and artificial intelligence models that can create expressive avatars.
Mesh is Microsoft’s vision of an evolution in current online work tools, which for most people generally consist of a bunch of shared documents, email, a messaging app (Teams, Slack, etc.), and a seemingly non-stop lineup of video meetings.
Powered by Microsoft’s Azure cloud platform, Mesh is designed to run on a range of devices, including Microsoft’s Hololens headsets, traditional VR goggles, phones, and more. With Mesh, Microsoft is hoping to create a virtual environment capable of sharing data, 3D models, avatars, and more.
According to Microsoft, Mesh will also enable geographically distributed teams to have more collaborative meetings, conduct virtual design sessions, assist others, learn together and host virtual social meetups. People will initially be able to express themselves as avatars in these shared virtual experiences and over time use holoportation to project themselves as their most lifelike, photorealistic selves.
The “holoportation” itself will allow Mesh devices to create photorealistic digital avatars of your body that can appear in virtual spaces anywhere in the world. With Microsoft Mesh-enabled applications, designers or engineers who work with 3D physical models – anything from bicycles to high-end furniture to jet engines to new sports stadiums – could appear as themselves in a shared virtual space to collaborate and iterate on holographic models, regardless of their physical location.
Mesh is an upgrade version of the traditional remote working experience with the power of Augmented Realuty (AR) and Virtual Reality (VR). It will use eye-tracking and facial-monitoring to allow the hologram to mimic a person’s facial expressions and eye contact.
Microsoft hopes holoportation can not only be used for remote work and social gatherings, but also be implemented in gaming and educational experiences.
The Microsoft Mesh platform will in coming months offer developers a full suite of AI-powered tools for avatars, session management, spatial rendering, synchronization across multiple users and holoportation to build collaborative solutions in mixed reality.
Though users will have the richest experiences in mixed or virtual reality, Microsoft Mesh’s open standards will also give developers the freedom to build solutions that will work across many different devices: HoloLens 2, a range of virtual reality headsets, smartphones, tablets and PCs.