The Web is an increasingly important resource in many aspects of life: government information and services, education and training, commerce, news, workplace interaction, civic participation, health care, recreation, entertainment, and more. In some cases, the Web is replacing traditional resources.
Therefore, it is essential that the Web be accessible in order to provide equal access and equal opportunity to people with diverse abilities. The Web is an opportunity for unprecedented access to information and interaction for many people with disabilities. The accessibility barriers to print, audio, and visual media can be much more easily overcome through web technologies. Accessible products and services offer people with disabilities to use things with the same level of efficiency, usability and satisfaction as for people without disabilities.
Digital accessibility ensures that websites, web apps, and digital content can be used by people with a diverse range of hearing, movement, sight or cognitive abilities.
Scope
- Accessibility
- Google Chrome Extension
- Javascript
- Machine Learning
PROBLEM
Many websites have accessibility barriers that make it difficult or impossible for some people with disabilities to use them. And many web software tools are not sufficiently accessible to people with disabilities, making it difficult or impossible for them to contribute to the Web. This is a very big deal. Many millions of people have disabilities that affect their use of the Web.
Web accessibility is about removing those barriers so that people with disabilities can use and contribute to the Web.
My Solution
A11Y ML
A Chrome Extension that uses machine learning to auto caption images and fix missing Alt Texts
How does this extension help?
One way to promote digital accessibility is by providing alt-text (alternative text), which provides a text alternative to non-text content in web pages including images, media, etc. Alt text can be challenging to create, audit, edit and/or update in existing websites. This Chrome extension will automate this process by using machine learning and image detection.
One of the most common pitfalls in enabling accessible content is the lack of alt text in the HTML attribute in the source. Screen-reader software uses alt text to enable someone who’s listening to the content of a web page, such as a blind person, to understand and interact with the content.
Through a machine-learning model, this add-on automatically generates alt text keywords and phrases for a particular image.
HTML code
<img height="600" width="800" src="https://haykmikayel.com/wp-content/uploads/2019/09/zebra.jpg" alt="" />
alt-text is missing
HTML code
<img height="600" width="800" src="https://haykmikayel.com/wp-content/uploads/2019/09/zebra.jpg" alt="a zebra standing on top of a dry grass field." />
alt-text has been generated and ingested in DOM
Why Machine Learning?
There are many ways to put captions on images. However, most share the following disadvantages:
- They are not responsive and take a long time to return a caption.
- They are semi-automated. They rely on humans to manually place captions on images.
- They are expensive to create and maintain.
Considering those, it might be a great use of Machine Learning, as ML now drives a huge number of day-to-day interactions on the web.
As of now, a machine can never truly guess the intent of a content author. However, we all agree that a computer-generated description is better than no description. And captioning tools like this, if incorporated with automated accessibility testing, can assist us in generating alternative text for an image or act as a fallback for assistive technology when there is no alt text available.
ML Model
IM2TXT captioning is the model used in this project. The add-on supports Hosted and Local models that run on Runway. Most ML solutions including Runway require computational power, which usually costs money. So, it might not be available at the moment as Runway requires credits to run Hosted models.
References
- Auto Alt Text – Need to click on the image before alt text appears
- Wave Evaluation Tool – Evaluate web accessibility within the Chrome browser
- Cloudinary Add-ons – Cloudinary takes care of your entire image management pipeline. With Cloudinary Add-ons, you can enhance your images even further with powerful functionality
- ACM SIGACCESS Conference on Computers and Accessibility
- Automatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service – Designed and deployed automatic alt-text (AAT), a system that applies computer vision technology to identify faces, objects, and themes from photos to generate photo alt-text for screen reader users on Facebook
- Emojis and screenreaders – “This is social media when you are blind.”
Further development
I am considering developing this plugin further and add new features. Here are some points I am planning to work on:
- Create a WordPress plugin
- Generate images based on labels
- Use ML to provide a better screen reader experience
- Retain model on web semantics
Special thanks to the following people for their generous support: Abi Muñoz, Yining Shi, Lauren Race, Ellen Nickels.