An economical eye-tracking algorithm for assistive wheelchair control using MediaPipe’s facial landmarks

Department of Mechatronics, Nelson Mandela University, Gquberha, 6013, South Africa

^* Corresponding authors: s220672326@mandela.ac.za, farouk.smith@mandela.ac.za

Abstract

We present the design, implementation, and evaluation of a novel eye-controlled wheelchair interface using MediaPipe’s face mesh for robust, low-cost operation. The system interprets horizontal gaze shifts for steering and intentional one-eye blinks for forward/reverse commands, enabling hands-free mobility for users with severe disabilities. The hardware comprises a 5 MP infrared (IR) camera on a Raspberry Pi 4, two 24 V 250 W DC drive motors, two 20 Ah LiFePO₄ batteries, and four ultrasonic collision sensors. Face and iris landmarks (478 total, including 10 iris points) are detected in real time; gaze direction is computed relative to eye corners, and blink detection uses the Eye Aspect Ratio. We calibrated thresholds empirically (gaze offset > 15% of eye width triggers a turn; EAR < 0.18 triggers a blink). In tests conducted by the author under well-lit (≈1000 lux), dim (≈200 lux), and pitch-dark (~0 lux) conditions, our algorithm achieved up to 98.71% overall command-recognition accuracy using the IR camera (with slight degradation to ≈91% under low visible light). These results, corroborated by confusion matrices, indicate reliable performance comparable to recent deep-learning approaches. The mechanical design meets expected torque needs (~25 N·m per wheel) and the collision avoidance worked reliably (albeit with limited testing). We discuss limitations (lighting sensitivity, head-movement constraints) and propose improvements like active IR illumination and user-specific calibration. This work demonstrates an effective, affordable assistive interface aligning with best practices in assistive robotics.