waypoint

The world's first AI-powered game controller for visually impaired players

the Waypoint controller and Dock - two objects, one system

overview

2.2 billion people worldwide live with visual impairments, yet fewer than 0.000001% of video games are fully accessible to them. Waypoint is a hardware and AI system built from scratch to close that gap, giving blind and visually impaired players a genuinely independent path to mainstream gaming without requiring sighted assistance.

The system is two objects: a custom tactile controller and the Waypoint Dock, a camera unit that reads the game screen in real time. Object detection and OCR running locally on an embedded compute unit translate visual game state into haptic, audio and tactile cues through the controller. The player stays in control. Waypoint narrates and guides but never plays for them.

early prototype testing - iterating on form and button layout directly with users

hardware and interaction design

The controller went through over a dozen physical iterations, from a bare foam block to a fully functional wireless prototype. Each version was tested with visually impaired users, with feedback directly shaping the next form. Button layout, grip geometry, and tactile differentiation were all refined through hands-on sessions rather than assumptions.

Every button has a distinct texture and position so it can be identified by touch without looking down. The controller works in a single hand, keeping the other free. The Dock sits beside the screen with a wide-angle camera and houses the local compute unit, running AI inference entirely on-device with no cloud dependency and no noticeable latency.

the full system loop - game screen, Dock, AI, controller

AI system

The Waypoint Dock reads the screen continuously in the background, monitoring for gameplay-relevant events and storing contextual information about the game world. When the player queries the controller, the AI has enough context to give a meaningful, specific response rather than a generic description of what's on screen.

The system runs on a local compute unit embedded in the Dock. No cloud calls, no privacy exposure, no dependency on internet connectivity. The AI agent successfully guided a player through a full game level in under six minutes during structured validation testing.

validation session at RSBC - testing with visually impaired players on real games

validation and funding

Waypoint was validated through structured workshops with 14 visually impaired players at RSBC, testing across multiple game titles and genres. The sessions shaped everything from audio cue timing to button sensitivity thresholds.

The project secured incubator backing through WEInnovate was shortlisted for InnovationRCA, and won for the Hult Prize Imperial Campus Competition, all pitched with working prototypes rather than slide decks.

waypoint in use