Mixed reality architecture design
Mixed reality is a blend of physical and digital worlds that unlocks natural and intuitive 3D human, computer, and environmental interactions. This new reality is based on advancements in computer vision, graphical processing, display technologies, input systems, and cloud computing.
The following Venn diagram illustrates the interaction between computers, humans, and the environment in mixed reality:
Paul Milgram and Fumio Kishino introduced the term mixed reality in a 1994 paper, A Taxonomy of Mixed reality Visual Displays. The paper explored the concept of a virtuality continuum and the taxonomy of visual displays. Since then, the application of mixed reality has gone beyond displays to include:
- Environmental awareness with spatial mapping and anchors.
- Human responses like hand-tracking, eye-tracking, and speech input.
- Spatial sound.
- Locations and positioning in both physical and virtual spaces.
- Collaboration on 3D assets in mixed reality spaces.
Mixed reality consists of several types of apps:
- Enhanced environment apps (HoloLens only) place digital information or content in a user's current environment.
- Blended environment apps create a digital layer that overlays the user's space.
- Immersive apps create an environment that completely changes the user's world and can place them in a different time and space.
The following image shows the continuum of mixed-reality apps from physical to digital reality:
Azure mixed reality services
Several Azure services help developers build compelling mixed reality experiences on various platforms. Azure mixed reality services help people create, learn, and collaborate within their own context by bringing 3D to mobile devices, headsets, and other untethered devices. All Azure services build in comprehensive security and compliance capabilities.
Azure Remote Rendering lets you render highly complex 3D models in real time and stream them to a device. You can add Azure Remote Rendering to Unity or native C++ projects that target HoloLens 2 or Windows desktop PC.
Azure Spatial Anchors is a cross-platform service that lets you build spatially aware mixed reality applications. With Spatial Anchors, you can map, persist, and share holographic content across multiple devices at real-world scale.
Azure Object Anchors is a mixed reality service that helps you create rich, immersive experiences by automatically aligning 3D content with physical objects. Object Anchors lets you gain contextual understanding of objects without the need for markers or manual alignment.
Microsoft Learn training resources
Microsoft Learn is a free online platform that provides interactive learning for Microsoft products.
Introduction to mixed reality
If you're new to mixed reality, the best place to learn about the ecosystem is the Introduction to mixed reality learning path, which provides foundational knowledge about the core concepts of mixed reality, virtual reality, augmented reality, holograms, and creating 3D applications.
Build a mixed-reality experience
If you're an intermediate-to-advanced developer with previous experience with mixed, augmented, or virtual reality, check out the HoloLens 2 fundamentals: develop mixed reality applications tutorial series. With this tutorial, you can build a mixed-reality experience that lets users explore a hologram modeled after NASA's Mars Curiosity Rover. The tutorial gives you a firm grasp of the Mixed Reality Toolkit (MRTK), and shows how the MRTK can speed up your development process.
Mixed reality design and prototyping
Before you start designing and prototyping mixed reality software, read and understand the following articles:
- Mixed reality design guidance
- Mixed reality core concepts
- Types of mixed reality apps
- Interaction models
- UX elements overview
Path to production
Choose the mixed reality engine you want to use for development. You can select from Unity, Unreal, native, or web development paths.
Get the tools you need to build applications for Microsoft HoloLens and Windows Mixed Reality immersive headsets, and get set up.
Best practices
Follow these best practices to design, develop, and deliver high-quality, user-friendly mixed reality apps.
- Design content for holographic display describes elements to consider for the best holographic experience.
- Comfort explains how to create and present content that mimics cues in the natural world and avoids fatiguing motions.
- Spatial sound best practices discusses how to use sound to inform and reinforce the user's mental model in the mixed-reality world.
- App quality criteria overview presents the top factors that affect mixed reality app quality.
Scenarios and solution ideas
The following implementations and ideas illustrate some ways to adapt and configure mixed reality for various scenarios.
- Shared experiences in mixed reality
- Free-roaming multiuser VR experiences
- Prototyping and manufacturing for enterprises
- Training and simulation for enterprises
- Immersive education
- Theme parks and family entertainment centers
- Virtual museums, exhibits, and tourism
- Virtual reality arcades
Solution ideas:
- Design review with mixed reality
- Facilities management with mixed reality and IoT
- Training and procedural guidance with mixed reality
Stay current with mixed reality
Mixed reality is changing fast. Explore the expanding world of mixed reality applications with the Mixed Reality Toolkit (MRTK), Windows Mixed Reality, Unity, Unreal, and more for HoloLens and Windows Immersive Headsets.
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for