Computer Vision-Based Assistance System for Visually Impaired Individuals in Vending Machine Interactions

Vending machines are widely known for their convenient accessibility to a wide range of products, eliminating the need for human assistance. However, visually impaired individuals often encounter difficulties when it comes to independently selecting, paying for, and retrieving items from these machines due to their heavy reliance on visual cues. This project aimed to address this issue by developing a computer vision-based model that would aid visually impaired individuals in acquiring products from vending machines. The research involved curating a comprehensive dataset, which consisted of images depicting various types of vending machines and the objects they dispense such as beverages and snacks, also labels for beverages. This dataset served as the foundation for training a custom object detection models using the state- of-the-art framework from YOLO family, applying supervised learning and semi supervised learning. By leveraging these models, the aim was to enable the system to accurately identify and locate products within vending machines. To evaluate the performance of the developed model, a series of tests were conducted. The results obtained from these assessments showcased the potential of the model in facilitating a system that could significantly enhance accessibility and independence for visually impaired individuals who interact with vending machines. The positive outcomes indicated that the model had the ability to successfully identify and track products within the vending machines, thus enabling users to navigate the purchasing process more effectively. The model demonstrated promising performance and exhibited potential for integration into a larger system aimed at improving accessibility and independence for visually impaired individuals when interacting with vending machines.