New Facebook tool for blind
Helps users ‘see’ photos,
A new Facebook tool will help users who are blind or have visual impairments “see” photos by describing what’s in them.
With automatic alternative text, people using screen readers on iOS devices will hear a list of items that a photo on Facebook may contain, such as “three people, smiling, outdoors.” Without the new automated photo captioning tool, screen-reader users skimming their news feed would be told only that a friend posted a photo. Screen-reader software turns Web pages and documents into synthesized speech.
“We really want to start with a set of concepts that frequently appear in photos that add a lot of value to the narrative of the photo,” Jeff Wieland, Facebook’s head of accessibility, told USA TODAY.
More than two billion photos are shared every day on Facebook and Facebook-owned Instagram, Messenger and WhatsApp services. The flood of imagery means people who are blind or have visual impairments often are left out of the conversation.
In its first iteration, Facebook’s new automated photo captioning tool can identify rudimentary things: cars and boats, basketball and baseball, ice cream and sushi, beards and eyeglasses. In time, Facebook aims to include a much fuller description of what’s in a photo, including the identity of the people in it. It also plans to expand the tool to more devices and languages, as well as to all of Facebook’s family of apps. For now, the tool is only being tested on iOS screen readers set to English.
Facebook and other tech giants are tackling a major challenge: The World Wide Web is anything but wide for people with visual impairments and other disabilities, cutting them off from the basics of everyday life such as applying for college or jobs, making major purchases or getting health information.
Facebook is taking steps to reengineer its website and mobile apps to make them more accessible. It’s also brainstorming a new generation of products that har- ness the power of artificial intelligence to improve the experience of Facebook for people with disabilities.
Automatic alternative text generates a description of a photo using advances in object recognition technology. The tool was requested by users with visual impairments frustrated that they were excluded from the conversation surrounding photos in their news feeds.
Even small bits of information can be helpful when a friend uploads a photo without a caption or the caption does not describe what’s in the picture, says Facebook accessibility specialist Matt King, who is blind and has been using a screen reader since college. King says he gleans clues from the captions and comments, but “you really feel excluded when you can’t see the picture.”
“Obviously, photos and videos are a lost cause because I can’t see those anymore,” says Marco Salsiccia, a former animator and visual effects artist who lost his sight two years ago due to medical complications from surgery. Today, he works as an advocate for the blind and on a contract basis as an accessibility specialist at Lyft. He took part in a group that tested the new photo-captioning tool for Facebook.
With the new tool, Salsiccia says: “I am less prone to skip over a photo now.”
Salsiccia says he welcomes innovation from Facebook, but says the company has a long road ahead to give equal access to people with disabilities.
“It’s still not perfect, but within the past year or so Facebook has made some decent strides in fixing some of the access issues,” he said.
Among his major gripes when it comes to photos: His screen reader still can’t read him text that appears in them, such as screenshots, inspirational quotes or funny memes that pop up all the time in Salsiccia’s news feed.
King says that is a common source of frustration for users who are visually impaired, himself included, but illustrates the sometimes daunting technological challenges Facebook has encountered as it builds technology to make the giant social network more inclusive.
“Everybody is thinking this should be simple, but it’s not necessarily simple when you are trying to sort text out from pictures and you don’t necessarily know if there is text in them,” he said.