Built at ExecuTorch Hackathon — Mobile & XR Track

Vysion XR

Read a sign, understand a room, navigation mode, hands-free. Vysion XR is an AI accessibility companion for blind and low-vision people on the Samsung Galaxy XR headset. Say "Vysion, read this", "describe", or "find the door" and it answers out loud. Today, reading a sign means waiting for a stranger, aiming a phone, or uploading a photo to someone's cloud. With Vysion you just ask. Three modes: - Read: reads signs, menus and labels aloud in about 1.5 s (measured 1.27–1.72 s on the headset; target was 6 s). - Describe: a short description of the scene, hazards first. - Navigate: finds the door, then guides you with a spatial-audio ping from its direction and clock-face directions ("door, 2 o'clock, 3 meters"). Obstacle warnings cut in over everything. Everything runs on the headset. Five models run on ExecuTorch with XNNPACK on the Galaxy XR's Snapdragon XR2+ Gen 2: LFM2.5-VL-450M (4-bit) for scenes and doors, PP-OCRv6 for text, YOLOv8n (Open Images V7, int8) for people, obstacles and doors at about 11 fps, and Whisper-tiny with Silero VAD for voice commands. Read, Describe and Find the door need no connection, and the photo never leaves the device. An opt-in cloud Describe and a "where am I" map are optional extras, each with an on-device fallback. Designed for the ear first, on two devices. Every state has its own sound and color, "stop" interrupts any speech, and hand gestures have spoken fallbacks. The same app also runs as a phone app on the Galaxy S25 Ultra, with a gesture-first design: the whole screen is the button. Tap anywhere to read, swipe to switch modes, swipe up to repeat, press and hold to talk, and two-finger double-tap to stop. Every gesture also has a labeled control for TalkBack users. One app, one set of sounds and commands: learn it once, start with the phone you already have, and keep a backup in your pocket when the headset is off. (Navigation needs the headset's depth sensing; Read and Describe run on both.) Numbers are from development runs on the device; the formal 20-run benchmark is still pending. Vysion XR helps you read, understand, and move through a space. It is not validated for unassisted travel by blind users.