Live Apple Immersive Video encoding.

What live Apple Immersive Video encoding involves

Apple Immersive Video (AIV) is Apple's 180-degree stereoscopic format for Apple Vision Pro. A finished AIV master carries 8160 x 7200 pixels per eye at 90 frames per second, with Apple Spatial Audio and lens calibration data that tells the headset exactly how to project each frame. On demand, that master is encoded over hours. Live, the same work has to finish inside the frame interval, for every frame, for the length of the event.

A live Apple Immersive Video encoder has four jobs, and all four run at once:

• Ingest. Take both eyes from the camera as uncompressed-class video without dropping a frame.
• Encode. Compress the stereo pair as MV-HEVC across a ladder of bitrates.
• Carry the metadata. Keep the venue description (AIME), projection signaling, HDR color, and spatial audio attached to the stream.
• Package and deliver. Segment the result as HLS and push it to an origin or CDN that Apple Vision Pro can play from.

Why AIV is hard to encode live

The difficulty is pixel rate. A 16K immersive camera feed is 16,320 x 7,200 pixels, 90 times a second. That is roughly 10.5 billion pixels every second, arriving whether the encoder is ready or not. A conventional 4K60 broadcast encoder handles about 500 million. Live AIV asks for around twenty times that, in a codec with two dependent views, with no room for a dropped frame because a dropped frame in a headset is something the viewer feels.

Cloud encoders built for flat broadcast video are not designed for that input. Getting a 16K ProRes feed out of a venue and into a cloud region is its own problem before any encoding starts. That is why live AIV encoding happens on site, next to the camera, on hardware built for the throughput.

Live Apple Immersive Video is not a bigger 4K stream. It is a different class of pixel rate, and the encoder has to be built for it.

How Zeus encodes Apple Immersive Video in real time

SpatialGen Zeus is a 3U rackmount live system built for this workload. It ingests ProRes over SMPTE ST 2110 with 100 Gb/s of peak sustained throughput, at up to 16K and 90 FPS. It supports the Blackmagic URSA Cine Immersive 100G, along with the rest of the URSA Cine line.

From that feed Zeus encodes MV-HEVC in real time, builds the bitrate ladder, packages the stream, and delivers it to SpatialGen, AWS, GCP, Azure, Akamai, or another cloud or CDN. Zeus also encodes HEVC, H.264, and AV1, so the same box can feed a flat simulcast to YouTube or Twitch alongside the immersive stream.

Those numbers come from the Spectrum case study, where SpatialGen engineers built the live encoding pipeline on site for courtside Lakers games.

What the output stream looks like

Apple Vision Pro plays Apple Immersive Video over HTTP Live Streaming. Apple's HLS authoring specification requires stereo video to be MV-HEVC, and an AIV stream is signaled with a stereo channel layout and the AIV projection. The multivariant playlist also points to the venue description, the AIME file that holds the camera's lens calibration. Without it, the headset cannot project the image correctly.

For delivery, Apple recommends 4320 x 4320 per eye at 90 frames per second in P3-D65 PQ, with average bitrates between roughly 25 and 150 Mbps across the ladder. Zeus builds the ladder live, so a viewer on a slower connection gets a lower rung instead of a stalled stream.

Owning the encoder

Zeus is a one-time purchase at $65,000 with a lifetime software and hardware license and a year of direct access to the SpatialGen engineers who built it. There are no per-channel, per-hour, or per-output charges for encoding. Rehearsals, extra camera positions, and test streams cost nothing extra to run, which matters in a format where teams are still learning what works. The full system details are in theZeus technical specifications.

For the wider workflow around the encoder, including capture, playback, and distribution, see live AIV, end to end.