Why are captions suddenly everywhere and how did they get there

People with hearing loss have a new ally in their efforts to navigate the world: captions that aren’t limited to their television screens and streaming services.

The COVID pandemic disrupted daily life for people everywhere, but many people with hearing loss found the resulting isolation especially difficult.

“When everyone wears a mask they are completely incomprehensible to me,” said Pat Olken, of Sharon, Massachusetts, whose hearing aids were insufficient. A new cochlear implant has helped her a lot.

So when her grandson’s bar mitzvah was streamed on Zoom at the start of the pandemic, Olken turned to Otter, an app built to transcribe, before offering the service’s captions. Business Meetings Reading with speakers of the ceremony made the app “a tremendous resource,” she said.

People with hearing loss, a group of nearly 40 million American adults, have long adopted technologies To help them make their way into the world of hearing, from Victorian-era ear trumpets to modern digital hearing aids and cochlear implants.

But today’s hearing aids can cost more than $5,000, are often not covered by insurance, and don’t work for everyone. Just as glasses correct vision immediately, the devices also do not pick up audible sound in focus. Instead, hearing aids and cochlear implants require the brain to interpret sound in a new way.

“The solutions are clearly not a one-size-fits-all model and don’t meet the needs of a lot of people based on cost, accessibility, many different things,” said Frank Lynn, director of the Cochlear Center. for Hearing and Public Health at Johns Hopkins University. It’s not just a communication problem; Researchers have found a link between untreated hearing loss and a higher risk of dementia.

Inexpensive over-the-counter hearing aids are on the way. But for now, only 20% of those who benefit from hearing aids use one.

Captions, in contrast, are usually very easy to access. They have long been available on modern television sets and are cropping up more frequently in videoconferencing apps like Zoom, streaming services like Netflix, social media videos on TikTok and YouTube, movie theaters, and live art venues.

In recent years, smartphone apps such as Otter; Google’s Live Transcribe; Ava; InnoCaption for phone calls; and GalaPro, for live theater performances, have appeared. Some are aimed at people with deafness and use human reviewers to ensure that captions are accurate.

Others, such as Otter and Live Transcribe, instead rely on what is called automatic speech recognition, which uses artificial intelligence to learn and capture speech. ASR has issues with accuracy and lags in transcribing the spoken word; Inherent bias can make transcription less accurate for voices of women, people of color and deaf people, said Professor Christian Vollger of Gallaudet University, who specializes in accessible technology.

Jargon and slang can also be a hindrance. But users and experts say that the ASR has improved a lot.

While welcome, none of these solutions are perfect. Tony Iacolucci of New York says that while she was using Otter to transcribe conversations, her book club may still be over. He said captions aren’t always accurate and don’t identify individual speakers, which can make it difficult to maintain.

“It took a little work,” said Iacolucci, who lost his hearing nearly two decades ago. After coming home, she was so tired of trying to follow up on the conversation that she had to lie down. “It just takes so much energy.” She received a cochlear implant a year ago, which has greatly improved her hearing, where she can now speak face-to-face without captions. He said they still help with group discussions.

Otter said in a statement that it welcomes feedback from the deaf and hard of hearing community and noted that it now offers a paid software assistant that can join virtual meetings and transfer them automatically. .

Transcription lag can present other problems – among them, there is a concern that conversation partners may become impatient with the delay.

“Sometimes you say, ‘I’m sorry, I just need to look at your caption to hear,'” said Richard Einhorn, a musician and composer in New York. “That doesn’t mean I don’t know that sometimes it causes trouble for other people.”

Other issues come to the fore. When Disciple Wyatt of Salt Lake City went to her doctor’s office, the Wi-Fi wasn’t strong enough for the transcription app to work. “It was writing gestures and things and making sure I got a written report later, so I knew what was said,” she said.

Movie theaters provide equipment that amplifies the sound, as well as glasses and separate screens that show captions to go with the movie. But they aren’t always intuitive and sometimes aren’t well made or just don’t work. Many people with hearing loss wish to have more movies playing captions on the big screen, just like you would do in the comfort of your own home.

A new law that took effect in New York City on May 15 requires movie theaters to offer on-screen captions for four showtimes per movie each week, including during the most popular hours to go to movies – Fridays. Includes nights and weekends. Hawaii passed a state law in 2015 that required two screenings a week of each film with on-screen captions. Large film chain AMC also says it screens some films with captions in about a third of its US theaters.

Captions are now more available for live performances as well. Many Broadway theaters promote a smartphone app that captions live performances; There are also separate handheld devices that show captions. The theater also has some performances with “open captions” that everyone can watch.

During the pandemic, online meetings and school changes meant videoconferencing services became a tool of existence – but the caption came only after a major push. Zoom only added live transcription to its free service in October 2021, but the host of the meeting will have to enable them. Google Meet was quick to make captions available to everyone for free in May 2020; Workplace messaging app Microsoft Teams did this in June.

“We need captioning everywhere and we need people to be more sensitive,” Olken said. “The more I advocate the more other people benefit.”