Consumer audio reality is the logical follow-up behind VR, and it’s what Facebook’s developmental arm is currently workshopping.
These are not grandpa’s waxy hearing aids. Sure, they currently look like something Elroy Jetson would wear tearing around the family’s cloud-brushing Orbit City home, but augmented reality (AR) could be defined as pioneer tech for those suffering from hearing loss, while doubling as an enhanced hearing aid for everyone else.
Facebook may be on the outs with the younger crowd and the domain of Gen X and boomers, but the social media’s Reality Labs (FRL) hopes to get the attention of audiophiles with its deep-dive research into AR, with variations of headset, glasses, and IEMs.
Noisy backgrounds in-person and on virtual calls leave even those who hear in the 2,000 to 5,000 range (the most sensitive hearing) struggling to hone in on a speaker’s voice. The inability to distinguish words amid the din can easily shut down what would have been pleasant or significant social interaction. If that noise was plaguing a Zoom call with work, it might mean missing critical information or being left out of an appealing project.
SEE: Quick glossary: Augmented Reality (TechRepublic Premium)
A recent post on Facebook’s newsroom page described its research arm, FRL (formerly the AR/VR team, renamed at the end of August) as an “interdisciplinary audio team made up of research scientists, engineers, designers, and more,” currently at work and “striving to improve human communication through radical audio innovation.”
Better hearing means better interactions in work and social life
Johns Hopkins’ said about one-in-five people in the US have hearing loss, but they eschew hearing aids for many reasons, including expense, social stigma, discomfort, and lack of reliability.
FRL team member Thomas Lunner (whose work was the basis for the world’s first digital hearing aid in 1995) was quoted in the post, “By putting hearing impaired people on par with people with normal hearing, we could help them become more socially engaged,” he said, “…hearing loss often keeps people away from social situations.”
The goals of FRL are:
- To create virtual sounds that are perceptually indistinguishable from reality.
- To redefine human hearing.
They plan to deliver two new capabilities:
- Audio presence: The listener feels physically present through high-fidelity audio indistinguishable from a real-world source.
- “Perceptual superpowers:” A tech advancement in which a listener can shut out the unnecessary surrounding babble and focus on a companion, and turn up the volume of their voice, while turning down unwanted background noises.
The post stated that a demo which uses FRL’s novel algorithm and software processing technologies, as well as off-the-shelf hardware, to illustrate the experience of personalized audio, recreated a room’s acoustics, and made a hardware device acoustically transparent allowed FRL to develop an always-on audio calibration system to hear sounds in ultra-high fidelity through a pair of headphones with no coloration from the hardware.
The sound is so crisp and clear, the author of the post asserts, that it becomes indistinguishable from reality.
FRL’s Redmond, CA location has a multi-million dollar anechoic chamber, which can test sound to an individual’s specific ear geometry; it might be likened to an ear version of the orthotics’ machines at pharmacies and the podiatrist’s office which create a person’s literal footprint. Basically, it’s a personalized experience of hearing spatialized audio head-related transfer function (HRTF), something the post said was developed by Facebook’s AR/VR in 2017.
With today’s VR and gaming, the HRTF is generic, and everyone participating hears the sound in the exact same way, but with customized HRTF, sounds are individualized.
For sound to be replicated as real, a room’s acoustic properties place a virtual sound source into the real space, as spatial audio mimics the direction of where the sound comes from. The currently-in-prototype headphones combined with an eye-movement tracking device can explore “what a person wants to hear.” This means that in a virtual call or meeting, the social interaction will sound as if the speakers were in each others’ presence.
Researchers quoted in the FB post said there’s a “higher purpose” at stake, to improve human hearing, and the new tech “can extend and enhance hearing ability” for “increased concentration and focus.”
It’s not news that loud noise contributes to hearing loss, prolonged exposure to noise levels more than 85 decibels (apparently common in crowded bars and restaurants), and dimming the noise may help protect people’s hearing over time, FB asserts in the post.
In the development and prototypes are “tiny” “perceptually transparent hear-through” in-ear monitors (IEM) to cover the complete range of sounds humans hear, 20 to 20,000 Hz. The IEM paired with FRL’s input prototype (including the microphone array, capturing surrounding sound), FRL’s researchers said “we can deliver the full experience of auditory superpowers.”
Because the device can pick up and focus on surrounding conversations, FRL are concerned with “guardrails around our innovation” to protect privacy, the post said. One way around this is something similar to Apple’s AirDrop and it puts a protocol in which one users’ glasses can request permission from another.
FRL hopes to build in other privacy features and identity verification tech with headsets and glasses which can only be accessed by the devices’ owner.