Apple Immersive Video (AIV) represents the pinnacle of immersive media technology. At SpatialGen, we’ve recently automated the distribution process while solving many of its unique challenges. In this guide, we’ll walk through the five key steps for streaming Apple Immersive Video:

1. Production (The Camera)
2. Exporting
3. Encoding
4. Segmenting
5. Hosting

Apple’s sample AIV HLS stream

1) The Camera

Apple Immersive Video requires precise lens calibration metadata to carry from capture to playback. This means you’ll need a certified camera that supports the format.

The current standout is the Blackmagic URSA Cine Immersive—a powerhouse camera designed specifically for AIV. If you’re a hardware enthusiast, we recommend exploring the technical specs.

2) Exporting

After shooting, editing can be done in tools like DaVinci Resolve. A crucial but often overlooked step here is denoising.

Denoising reduces unwanted “noise” (speckles or grain) that don’t represent real detail. In immersive workflows, denoising is essential as it ensures a more efficent and high quality encode in the next step.

Denoising is often the “secret sauce” of big production studios and countless hours are spent fine-tuning their workflows. If you want to get a huge boost in your footage, consider denoising and coloring.

Below is an example of the results of denoising. I reccommend zooming into the picture as if you’re watching an immersive video and imaging all the noise magnified in front of your eyes.

If you’re using DaVinci Resolve, you’ll work with two main export types:

Vision Pro Review

• Creates an AIVU file with MV-HEVC video encoded via Apple’s Video Toolbox APIs.
• An AIVU includes dynamic metadata and lens calibration data for native playback on Apple Vision Pro.
• Useful for testing, but the Video Toolbox APIs do not maintain the source image as much as other encoders.

Vision Pro Bundle

• Produces a dual-track ProRes export, plus an FCPXMLD and AIME file.
• The ProRes file is huge (often terabytes). A common hiccup is people will modify their FCPXMLD file then send it to SpatialGen, but that will break the video. So don’t do this!
• They are the standard source for high-quality streams and can be uploaded with the free SpatialGen Uploader app.

3) Encoding

With your export ready, the next step is encoding for different resolutions and bitrates.

High encode quality is crucial in immersive video. Keep in mind that an encoder’s purpose is not to make footage look good, but to maintain the image from the source. Tools like VMAF help measure quality objectively.

For beginners, Apple’s Compressor app is a solid entry point. For example, you can pull each track from a ProRes export and encode them individually:

If you prefer open-source tools, see our guide on encoding MV-HEVC with FFmpeg.

We have a blog overviewing HDR MV-HEVC encoding with the VideoToolbox APIs if you want to build your own solution.

⚠️ Note: Apple’s VideoToolbox APIs don’t yet deliver the best AIV quality. That’s why at SpatialGen AIV uses our V2 encoder for all Professional customers by default.

Muxing

Once encoded, mux your tracks into a single AIVU using the Apple Immersive Video Utility App.

If you exported a Review Bundle, drag the AIME, FCPXMLD, and MV-HEVC files into the app. Right-click the output video, select Show in Finder, and copy the resulting AIVU.

4) Segmenting

Now that you have all your AIVUs in the same place, we must segment into folders. Create a folder for each bitrate ladder, then use a tool like mediafilesegmenter to segment each video file into its folder. For segment sizes of six seconds, the following command will suffice:

mediafilesegmenter your_video.aivu -f your_folder -t 6

Do this for each video.

To link all your streams together, you need a variant playlist file.

Apple has a good tool for creating your variant playlist file, called variantplaylistcreator that I highly reccommend checking out.

Apple’s HTTP Live Streaming (HLS) authoring specification is a great resource to consult as well. A sample playlist file from Apple can be found here.

#EXTM3U
#EXT-X-VERSION:12

#EXT-X-SESSION-DATA:DATA-ID="com.apple.hls.venue-description",URI="primary.aime"

#EXT-X-SESSION-DATA:DATA-ID="com.apple.private.content-type",VALUE="Fully Immersive"

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="English",DEFAULT=YES,CHANNELS="2/BED-2",URI="hls/audio/playlist.m3u8"

#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=99766343,BANDWIDTH=110318142,AUDIO="audio",VIDEO-RANGE=PQ,CODECS="hvc1.2.20000000.H153.B0,apac.31.00",RESOLUTION=4320x4320,FRAME-RATE=90,CLOSED-CAPTIONS=NONE,REQ-VIDEO-LAYOUT="CH-STEREO/PACK-NONE/PROJ-AIV"
hls/AivBeachWWDC_VideoVar_1/playlist.m3u8

#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=79896494,BANDWIDTH=82840942,AUDIO="audio",VIDEO-RANGE=PQ,CODECS="hvc1.2.20000000.H153.B0,apac.31.00",RESOLUTION=4320x4320,FRAME-RATE=90,CLOSED-CAPTIONS=NONE,REQ-VIDEO-LAYOUT="CH-STEREO/PACK-NONE/PROJ-AIV"
hls/AivBeachWWDC_VideoVar_2/playlist.m3u8

#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=49896717,BANDWIDTH=51539011,AUDIO="audio",VIDEO-RANGE=PQ,CODECS="hvc1.2.20000000.H153.B0,apac.31.00",RESOLUTION=4320x4320,FRAME-RATE=90,CLOSED-CAPTIONS=NONE,REQ-VIDEO-LAYOUT="CH-STEREO/PACK-NONE/PROJ-AIV"
hls/AivBeachWWDC_VideoVar_3/playlist.m3u8

#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=31979317,BANDWIDTH=33424900,AUDIO="audio",VIDEO-RANGE=PQ,CODECS="hvc1.2.20000000.H153.B0,apac.31.00",RESOLUTION=4320x4320,FRAME-RATE=90,CLOSED-CAPTIONS=NONE,REQ-VIDEO-LAYOUT="CH-STEREO/PACK-NONE/PROJ-AIV"
hls/AivBeachWWDC_VideoVar_4/playlist.m3u8

#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=25058340,BANDWIDTH=26262613,AUDIO="audio",VIDEO-RANGE=PQ,CODECS="hvc1.2.20000000.H153.B0,apac.31.00",RESOLUTION=4320x4320,FRAME-RATE=90,CLOSED-CAPTIONS=NONE,REQ-VIDEO-LAYOUT="CH-STEREO/PACK-NONE/PROJ-AIV"
hls/AivBeachWWDC_VideoVar_5/playlist.m3u8

Once you get this file created, the last part is easy.

5) Host

Now upload all your files to a cloud provider of your choice and get streaming! Use the URL to the vvariant playlist file in Safari on your Apple Vision Pro and you will be able to enter an immersive view.

If all of this is too costly, too time consuming, or simply not your thing, consider using SpatialGen to upload a video and get streaming. We handle all the complexity of streaming Apple Immersive Video and more so you don’t have to.

Zachary Handshoe & Michael Butterfield