By now, people understand spatial video. As production picks up and people fight for the latest edge, many ask - “Why not make audio spatial too?” Well let’s talk.
What Is Spatial Audio?
Spatial audio is a surround sound technology that uses on-device algorithms to create an immersive 3D audio experience. Apple, has popularized spatial audio by tracking head movement with AirPods and dynamically adjusting the audio to match the listener’s position. This allows for a 360-degree audio environment. The result is a more realistic and engaging auditory experience.
Because of this, one cannot easily say a video file has spatial audio, as it is Apple’s on-device processing that transforms regular audio into their branded “spatial audio.” The magic happens on the device, making the audio spatial and dynamically adjusting it based on the user’s head movements and position.
What Codecs Are Compatible With Spatial Audio?
-
Dolby Atmos: Widely used in cinema and home theaters, Dolby Atmos is supported by Apple devices for spatial audio experiences.
-
AAC (Advanced Audio Codec): Used in Apple Music and other streaming services, AAC can be enhanced with spatial audio on compatible devices.
-
HE-AAC (High-Efficiency Advanced Audio Codec): An extension of AAC, used for streaming high-quality audio at lower bitrates.
-
ALAC (Apple Lossless Audio Codec): Provides lossless audio compression, ensuring high-quality sound that can be spatialized.
Spatial Audio Requirements
Creating and experiencing spatial audio requires specific technical pre-requisites. At its core, spatial audio typically requires at least six audio channels to create a fully immersive experience. These channels include the front left, front right, center, rear left, rear right, and a subwoofer channel (often referred to as 5.1 surround sound). Additionally, the content must be encoded in a format that supports multi-channel audio, such as Dolby Atmos or AAC.
Compatible hardware is also essential: devices must support spatial audio playback, and users need headphones or speakers that can accurately render the 3D audio effect. Lastly, software on the playback device must be capable of processing and dynamically adjusting the audio based on the listener’s movements and environment.
Here’s How To Encode Spatial Audio
A video with 6 audio channels can be encoded into 5.1 surround sound using ffmpeg. This will put the audio of the video in the correct format for Apple to consider it spatial.
ffmpeg <input-video.mp4> -c:v copy -c:a eac3 -b:a 640k -ac 6 spatial_audio_compatible.mp4
Now with a file encoded in 5.1 or more, you can convert this video to Spatial MV-HEVC and have support for spatial audio.
Please reach out to us for questions or feature requests at contact@spatialgen.com
Thank you for being part of the SpatialGen community.
SpatialGen
Zachary Handshoe & Michael Butterfield