Apple’s 3D spatial audio format is under process for AR devices

Apple is developing a new audio format that will help in placing sounds around a user with Apple Glass, even while the user in motion. This endeavor is to produce high-quality audio and to implement the standardized audio format such as MP3 or AAC.

This new format will further incorporate extra spatial information for AR requirements. Earlier, Apple’s collaboration with Pixar for placing audio on a 3d space, using a format called USDZ has influenced the later petition for this new audio format. However, with this new format, it will not only determine the actual audio but will go beyond playing audio on the left or the right side of the stereo picture.

Working on the patent entitled, ‘File format for spatial audio,’ Apple mentions:

“Producing three-dimensional (3D) sound effects in augmented reality (AR), virtual reality (VR), and mixed reality (MR) applications… is challenging because existing audio formats were originally designed for producing 3D sound in a physical environment with fixed speaker locations and stationary listeners,such as in a movie theater.”

Apple has also confirmed about working on a wide range of 3D audio format, categorized under Simulated Reality (SR)-

“spatial audio formats designed to produce 3D sound include MPEG-H (Moving Picture Experts Group) 3D Audio standards, HOA (Higher-order Ambisonics) spatial audio techniques, and DOLBY ATMOS surround sound technology,.

Apple’s further proposal is an attempt to create a format that is similar to the .m4v and .mp4. For the audio spatial requirements, Apple wants to build a new “audio asset library… [which] includes asset metadata that enables simulated reality (SR) application developers to compose sounds for use in SR applications.”

The patent acknowledges Stephen E. Pinto, crediting him for spatial audio navigation for “Apple Glass” on the one hand, and Christopher T. Eubank, who has been working on high-resolution images for AR devices. The patent further has published an innumerable number of images and has provided specific details based on those figures, specifically the spatial audio bed, speaking of which it writes:

A spatial bed is a multi-channel audio content that represents a complete sound field description, e.g., a virtual sphere of sound, for example surrounding a simulated reality listener in a simulated reality environment,” it says. “A new spatial bed is generated by combining sections of at least two of such spatial beds.” It further continues, “The new spatial audio object may comprise a spherical array of virtual sound sources (virtual sphere) that define a sound field surrounding a listening position of the new spatial audio object, e.g., at a center of the custom mix sphere.”

This patent granted by the USPTO further mentions 3D sound positioning via Head related transfer function (HRTF), which will help in amplifying a sound from a distant position to straight into the ear. The patent further shows an elongated process, which might take some time. For now, it remains to be a developmental approach as a published medium, but for the development to take place, all one can do is wait.