Post Date
May 07 2024

Graph2Speech: Making Graphs accessible to Persons with Vision Impairment Using Computer Vision and Natural Language Processing

Persons with vision impairment (PVIs) face numerous hurdles in education and employment. This is especially true for science, technology, engineering, and math (STEM) fields that involve scientific graphs and charts for information summarization and communication. Traditionally, two methods are employed to produce accessible graphs. Both methods require specialized hardware (braille embosser with cardboard, thermal embosser with swell sheets) that are difficult to procure. More recent methods based on sonification systems and refreshable tactile graphics are either experimental or extremely expensive. This project aims to develop a software tool, called Graph2Speech (G2S), that leverages artificial intelligence (AI) for understanding and communicating the information contained in graphs to PVIs. This project is a continuation of previous works on accessible math and technical document authoring and presentation from the same research group. It is a step towards making STEM fields more accessible to all.