Live MV-HEVC, encoded in real time.

What MV-HEVC is

MV-HEVC is Multiview High Efficiency Video Coding, an extension of HEVC for video with more than one view. For stereo content, one eye is encoded as a normal HEVC layer. The other eye is encoded as a second layer that can borrow from the first, because the two eyes see nearly the same picture. The result is one stream that holds both views and costs less bitrate than two separate encodes or a side-by-side frame.

It is the codec Apple uses for spatial video and for Apple Immersive Video, and Apple's HLS authoring specification requires stereo video to be MV-HEVC. If a live 3D stream is going to Apple Vision Pro, it is going as MV-HEVC.

Why live MV-HEVC is harder than file-based MV-HEVC

SpatialGen has written about encoding MV-HEVC from files, both with FFmpeg andin HDR with VideoToolbox. A file encode can take as long as it needs. A live encode cannot.

• Two views, one deadline. Both eyes must be encoded within the same frame interval. At 90 FPS that is about 11 milliseconds per frame pair.
• The second view depends on the first. Inter-view prediction is where the bitrate savings come from, and it also means the two layers cannot simply be handed to two independent encoders.
• A ladder, not a single encode. Adaptive streaming needs several bitrates at once, so the live system runs the stereo encode several times in parallel.
• No second pass. File encoders look ahead and revisit decisions. A live encoder commits to each frame as it arrives and still has to hold quality in fast motion.

A file encode is judged on quality. A live encode is judged on quality at a fixed deadline, 90 times a second.

How Zeus encodes MV-HEVC live

SpatialGen Zeus is a 3U rackmount live encoding system. It takes ProRes over SMPTE ST 2110 at up to 16K and 90 FPS, with 100 Gb/s of peak sustained throughput, and encodes MV-HEVC in real time. It then packages the stream and delivers it to SpatialGen, AWS, GCP, Azure, Akamai, or another cloud or CDN.

Zeus also encodes HEVC, H.264, and AV1. A production can send MV-HEVC to Apple Vision Pro and a flat feed to YouTube or Twitch from the same system, without a second encoder in the rack.

Live MV-HEVC over HLS

Live MV-HEVC reaches viewers through HTTP Live Streaming. Each rung of the ladder is segmented, and the multivariant playlist tells the player the stream is stereo so it requests the right variant. HDR adds its own requirements. SpatialGen documented the details that makeHDR10 MV-HEVC play correctly over HLS, and the same requirements apply when the stream is live.

For Apple Immersive Video, the playlist also carries the AIV projection signaling and the venue description file. That workflow is covered inlive Apple Immersive Video encoding.

What live MV-HEVC is used for

• Apple Immersive Video. 180-degree, 90 FPS live events for Apple Vision Pro.
• Live sports. Courtside and front-row positions where depth is the point of the shot.
• Concerts, comedy, and stage events. A fixed seat that viewers anywhere can take.
• Stereo broadcast feeds. Any live 3D production that needs to reach Apple devices over HLS.

Zeus is a one-time purchase with a lifetime software and hardware license and a year of direct SpatialGen engineer access. The full details are in theZeus technical specifications.