2025-11-24 ペンシルベニア州立大学 (Penn State)
NaviSense can identify objects in a users environment, based solely off their voice commands, without requiring users to preload 3D models of the objects prior to use. Upon identification, vibration and audio feedback guides users directly to the desired object. These quality of life features distinguish NaviSense from other forms of guidance technology. Credit: Caleb Craig/Penn State. All Rights Reserved.
＜関連情報＞
- https://www.psu.edu/news/engineering/story/ai-tool-helps-visually-impaired-users-feel-where-objects-are-real-time
- https://dl.acm.org/doi/10.1145/3663547.3759726
NaviSense: 視覚障害者による物体検索を支援するマルチモーダルモバイル支援アプリケーション NaviSense: A Multimodal Assistive Mobile application for Object Retrieval by Persons with Visual Impairment
Ajay Narayanan Sridhar, Fuli Qiao, Nelson Daniel Troncoso Aldas, Yanpei Shi, Mehrdad Mahdavi, Laurent Itti, Vijaykrishnan Narayanan
ASSETS ’25: Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility Published: 22 October 2025
DOI:https://doi.org/10.1145/3663547.3759726
Abstract
People with visual impairments often face significant challenges in locating and retrieving objects in their surroundings. Existing assistive technologies present a trade-off: systems that offer precise guidance typically require pre-scanning or support only fixed object categories, while those with open-world object recognition lack spatial feedback for reaching the object. To address this gap, we introduce NaviSense, a mobile assistive system that combines conversational AI, vision-language models, augmented reality (AR), and LiDAR to support open-world object detection with real-time audio-haptic guidance. Users specify objects via natural language and receive continuous spatial feedback to navigate toward the target without needing prior setup. Designed with insights from a formative study and evaluated with 12 blind and low-vision participants, NaviSense significantly reduced object retrieval time and was preferred over existing tools, demonstrating the value of integrating open-world perception with precise, accessible guidance.
