Portable form filling assistant for the visually impaired

Abstract

The filling of printed forms has always been an issue for the visually impaired. Though optical character recognition technology has helped many blind people to ‘read’ the world, there is not a single device that allows them to fill out a paper-based form without a human assistant. The task of filling forms is however an essential part of their daily lives, for example, for access to social security or benefits. This paper describes a solution that allows a blind person to complete paper-based forms, pervasively and independently, using only off-the-shelf equipment including a Smartphone, a clipboard with sliding ruler, and a ballpoint pen. A dynamic color fiduciary (point of reference) marker is designed so that it can be moved by the user to any part of the form such that all regions can be “visited”. This dynamic color fiduciary marker is robust to camera focus and partial occlusion, allowing flexibility in handling the Smartphone with embedded camera. Feedback is given to the blind user via both voice and tone to facilitate efficient guidance in filling out the form. Experimental results have shown that this prototype can help visually impaired people to fill out a form independently.