What makes the Apple Vision Pro’s visionOS such an ambitious leap

The name of the operating system powering Apple’s Vision Pro headset is visionOS. And in many ways, it’s Apple’s most ambitious OS yet.

At the architecture level, visionOS shares core blocks in common with macOS, iPadOS and iOS — but importantly, visionOS adds a “real-time subsystem” for processing interactive visuals on the Vision Pro. The subsystem drives immersive experiences simply not possible on other Apple hardware, like animated environments with ambient sounds, 3D videos and avatars captured using the Vision Pro’s depth sensors.

If you’re really looking for how well all that software works when combined with the Vision Pro’s hardware, check out our full review here.

But for a deep dive into visionOS, here’s everything you need to know.

Navigating visionOS

The 3D interface of visionOS frees apps from the boundaries of a traditional display so that they can appear side by side at different scales. The UI responds dynamically to natural light, casting shadows to help communicate both scale and distance.

By default, visionOS apps launch into what Apple calls the “Shared Space,” where they exist side by side à la macOS. For a more immersive experience, an app can — if programmed to do so — open a dedicated “Full Space” where only that app’s content will appear.

Thanks to eye tracking, in visionOS, Vision Pro users can navigate by simply looking at apps, buttons and text fields (visionOS allows eye tracking with one dominant eye as an accessibility feature). Or they can tap — or hover over — with a finger to make a selection, flicking to scroll up or down.

In addition, visionOS supports speech recognition, letting Vision Pro wearers voice dictate anywhere they see a microphone icon and, via Siri, use voice commands to open or close apps, queue up media and more.

As an alternative, users can pair Bluetooth accessories — visionOS doesn’t recognize all of them, but (as a support article on Apple’s website outlines) it does work with current models of AirPods and Beats headphones, most first- and third-party Bluetooth keyboards, Apple’s Magic Trackpad, hearing devices approved through Apple’s Made for iPhone (MFi) program and MFi controllers.

Bluetooth peripherals come in especially handy if you link a Mac to visionOS, which can display a Mac’s screen in a 4K virtual window that can be moved around and resized. 

visionOS apps

Native visionOS apps are built using the visionOS SDK and API in tandem with Apple frameworks and APIs that’ve been around for a while, in particular SwiftUI and UIKit (UI frameworks), RealityKit (a 3D framework) and ARKit (an AR platform). SwiftUI and UIKit help craft the user interface, while RealityKit displays 3D content and visual effects and ARKit provides apps with an understanding of real-world depth.

Xcode, Apple’s IDE for macOS, is where the bulk of the app development happens (or Unity, which supports visionOS, also). A new simulator in Xcode lets developers explore various room layouts and lighting conditions, while a tool called Reality Composer Pro lets devs preview and prep 3D content — along with animations, images and sounds — for their visionOS apps.

Many major developers, including Microsoft, Adobe, Slack, Zoom and Webex, have released native visionOS versions of their apps. So have streaming services — TikTok, Max, Disney+, Prime Video, Paramount+ and ESPN clients are available to download from the dedicated visionOS section of the App Store (but not YouTube, Spotify or Netflix — at least not yet). We have a rundown of the apps early Vision Pro buyers should download asap right here, plus a rundown of apps coming from independent developers beyond the expected corporate players.

Apps needn’t be native to be useable on visionOS, which is technically backward compatible with existing iOS and iPadOS apps. However, developers can opt out of compatibility if they so choose.

Beyond apps that require an installation, visionOS users can tap WebXR, an API for web browsers including Apple’s Safari, to run browser-based mixed-reality experiences. WebXR isn’t enabled by default. But once switched on (by following these steps), any WebXR experiences that work with other AR/VR headsets should run just fine on the Vision Pro.

visionOS Environments and spatial video

At launch, visionOS ships with 13 3D, 360-degree background animations of real-world places that Apple calls Environments. Complete with ambient noises, Environments — which span Yosemite National Park, Haleakalā National Park, Joshua Tree National Park, Mount Hood National Forest, Lake Vrangla near Drammen, Norway, and the moon — optionally cycle through day and night sequences, displaying running apps in the foreground.

The immersion of an Environment can be adjusted, including the volume of the ambient sounds, by rotating the Digital Crown on top of the Vision Pro.

Even more immersive than Environments is spatial video, the 3D video format that the iPhone 15 Pro and iPhone 15 Pro Max — and the Vision Pro — can capture. While spatial videos appear as regular 2D videos on the iPhone and other devices, on visionOS, they can be expanded into an “immersive” view.

Spatial videos are recorded with spatial audio, Apple’s binaural audio tech. Thanks to dynamic head tracking, spatial audio can deliver what Apple describes as “theater-like” sound from video that’s playing — so sounds seem like they’re coming from all around.

Personas, SharePlay and Optic ID

Personas, another key visionOS capability, re-create a user’s face digitally by scanning it. Personas can be used during FaceTime calls and users can see other participants’ Personas if they’re using visionOS.

Also, visionOS supports SharePlay, Apple’s tech that lets wearers watch TV shows and movies, listen to music and play games from the App Store with users on other Apple devices. A “Travel Mode” stabilizes the Vision Pro’s visuals for use on planes, while Guest User shares specific apps and experiences with family and friends, like Photos or Safari.

Because Face ID and fingerprint scans in AR would almost certainly be an exercise in futility, Apple’s introducing a new biometric authentication system, Optic ID, with visionOS.

Optic ID analyzes a user’s irises under invisible LED light to unlock visionOS and the Vision Pro, autofill passwords and complete payments with Apple Pay. Apple says that Optic ID data is protected by an encrypted hardware chip in the Vision Pro and that it’s not accessible to apps — and never leaves their device.