Camera and ultrasonic sensor fusion for electronic travel aid: Design and performance analysis

Department of Mechatronics, Nelson Mandela University, Gqeberha, 6013, South Africa

^* Corresponding author: stefan.vanaardt@mandela.ac.za

Abstract

Millions of people globally experience visual impairment, severely limiting safe and independent mobility. This paper presents the design, development, and validation of an AI-enabled smart cane integrating ultrasonic and vision-based sensing for obstacle detection and object recognition. The system combines four HC-SR04 ultrasonic sensors with a camera connected to a Raspberry Pi 4B, running a custom-trained YOLOv4-tiny convolutional neural network (CNN) model to identify key navigational features such as doors, stairs, and ramps. The Arduino Mega handles low-latency ultrasonic sensing and vibration feedback, while the Pi processes visual frames and provides audio cues using an offline text-to-speech engine. In trials, the ultrasonic module achieved near-perfect obstacle detection within 1.2 m, and the vision system correctly identified trained objects under standard lighting, though accuracy declined under strong glare. Despite hardware limitations, the system effectively fuses both sensing modes to alert users via haptic and voice feedback. The paper evaluates all subsystems individually, verifying sensor accuracy, object detection reliability, and thermal performance under continuous load. This research contributes a portable, affordable solution toward safer navigation for the visually impaired using real-time embedded AI.