Facebook helps blind users 'see' photos with AI and image-recognition technology

Facebook is rolling out a new feature that will automatically describe the content of photos to blind and visually impaired users. Called automatic alternative text, the feature uses artificial intelligence to identify visual content and provide a description for people using screen readers.

While scrolling through Facebook, blind and visually impaired users will hear the name of the person followed by the word "photo" when they scroll past an image post by a user. Automatic alt text will then describe a list of themes of the image, such as "three people, smiling, outdoors" or "two people, smiling, sunglasses, sky, tree, outdoor".

According to Facebook, more than two million photos are shared on social media every day, yet as content becomes more visual, many blind and visually impaired users are left feeling excluded. Facebook itself has come under fire for isolating users by failing to address basic accessibility issues in its mobile app and has since been working to remedy this. Automatic alt text is the fruit of a 10-month effort by the company's Accessibility team to "build technology that helps the blind community experience Facebook the same way others enjoy it".

With automatic alt text, image descriptions are generated as image alt text, an HTML attribute designed to provide text alternatives for images that can be easily picked up by screen readers. The feature is powered by Facebook's object recognition technology, which has learned to recognise images after being trained with millions of examples. The company said automatic alt text had also been developed to be contextually-aware, in that it is able to identify what makes an image "interesting or significant".

Facebook continues to make strides in artificial intelligence. Aside from object recognition, the company is also trying to discover whether AI can learn semantic connections within language by teaching its neural network to read children's stories.

The latest development from the company comes after Microsoft unveiled its Seeing AI project at Build 2016, which uses machine learning and artificial intelligence to translate real-world events into audio messages for the blind and visually impaired.

Facebook is launching automatic alt text first on iOS screen readers set to English for users in the UK, US, Canada, Australia and New Zealand, before bringing the function to other platforms, markets and languages in the near future.

The company said: "While this technology is still nascent, tapping its current capabilities to describe photos is a huge step toward providing our visually impaired community with the same benefits and enjoyment that everyone else gets from photos on Facebook."