A Large-Scale Dataset and Synthetic Data
Generation Pipeline for Blind Navigation
NeurIPS 2026 ED Track
Abstract
Reliable recognition of accessibility-critical objects, such as accessible pedestrian signals, door-activation buttons, and handrails, is essential for technologies that support independent mobility for blind and low-vision (BLV) people. Yet these objects are often missing or underrepresented in existing vision datasets, and available data offer limited diversity, inconsistent annotations, and viewpoints that do not match assistive devices.
We introduce NavAble, a large-scale dataset and synthetic data generation pipeline for BLV navigation. NavAble combines 36,466 curated images with 5,681 collected image records, including 100 supplementary revolving-door frames, across 11 accessibility-critical categories. Its generator transforms web images into simulation-ready 3D assets and renders them in NVIDIA Isaac Sim. The current synthetic release contains 450,860 frames across 37 environments, with semantic segmentation, depth, surface normals, and 2D/3D bounding boxes.
Experiments across seven segmentation architectures reach 71.70% foreground mIoU on the paper's real-world benchmark. Adding synthetic data to real-only training improves five of the seven architectures by up to 7.19 mIoU points. Evaluation on a quadruped guide dog robot and a cane-shaped mobility-assistive device shows relative transfer gains on their egocentric viewpoints, while absolute accuracy remains insufficient for autonomous deployment.
- 11
- accessibility object classes
- 450.9K
- synthetic frames
- 37
- simulation environments
- 500
- released 3D assets
Key Contributions
- A dataset built around accessibility. Curated and newly collected real-world images with dense annotations for navigation-relevant infrastructure.
- A reusable synthetic data generator. Adapt the object classes, scenes, and camera viewpoints to your own perception task. Record a trajectory once, then replay it across assets, environments, and lighting with simulator-generated labels.
- Evaluation beyond a single viewpoint. Segmentation benchmarks and transfer evaluation on two different mobility-assistive robot platforms.
The NavAble Dataset
Three complementary data sources expand coverage of the objects that support everyday mobility decisions.
| Component | Count | Role |
|---|---|---|
| Curated real-world data | 36,466 images | Existing sources remapped to a shared class taxonomy. |
| Newly collected real-world data | 5,681 records | 5,581 released records plus 100 supplementary revolving-door images. |
| Synthetic data | 450,860 frames | Controllable appearance, lighting, environments, and viewpoints. |
Counts follow the current Hugging Face release. The 5,581 released collected records are split into 3,703 train, 396 validation, and 1,482 test records. The 100 supplementary revolving-door images are separate from these splits and await split assignment in the release.
The released-mask audit identifies 5,479 distinct photos within those 5,581 records. Per-class counts reflect images containing a class; classes can overlap, so their counts should not be summed to obtain the dataset size.
APS: accessible pedestrian signal. The main benchmark evaluates 10 foreground classes; revolving door is evaluated separately on the 100 supplementary frames (paper, Appendix H).
Manually Collected, Densely Annotated Data
We captured egocentric views across 50 indoor and outdoor locations with a chest-mounted RealSense D435. Every visible target instance receives a dense, pixel-level mask under shared annotation rules, followed by three full passes of cross-checking and correction.
SAM 3: 19.06% → 70.49% foreground mIoUA 51.43-point gain after fine-tuning on our collected real data alone.
Ground-truth masks are overlaid on the six collected images. Select an image to inspect it at full size. Collection and annotation: paper, Section 2.2 and Appendix B. SAM 3 comparison: Tables 2 and 4; same architecture, 1,482 real test images, 10 foreground classes.
Generate Synthetic Data for Your Own Task
Use NavAble’s Isaac Sim extension and companion command-line interface to build a dataset for your own object classes and viewpoints. The reusable workflow extends beyond BLV navigation to other perception tasks.
- Choose your objects. Turn class-specific image queries into simulation-ready 3D assets with automated filtering and reconstruction, followed by mesh review.
- Set up once. Place an asset and record a camera trajectory with simple teleoperation. Replay the path across object variants, environments, and lighting.
- Export the supervision you need. Get paired RGB, semantic segmentation, depth, surface normals, and 2D/3D boxes directly from the simulator.
5 trajectories → up to 60,000 labelled frames
The paper’s example replays 5 trajectories across 4 environments and 10 asset variants to produce 20,000 annotated frames. Repeating the renders under 3 lighting conditions expands this to 60,000 frames without recording new paths.
Workflow and scaling example: paper, Section 3.1. Object placement and camera trajectory example from the NeurIPS presentation.
Synthetic Data Demo
Lighting variation, annotation modalities, and a nine-object montage from the synthetic data generator.
The video first shows a pedestrian-signal button under changing illumination, then paired renderings and annotations, and finally nine examples of accessibility-related infrastructure.
Segmentation Evaluation
We benchmark seven segmentation architectures: SegMAN, Mask2Former, SAM 3, SAN, SegFormer, EncNet, and DeepLabv3+. Four training configurations separate the effects of collected data, curated data, and synthetic supervision, with 28 trained checkpoints released. The paper's main benchmark uses 1,482 real-world test images and reports foreground mIoU over 10 classes, excluding background and revolving door.
SegMAN with real, curated, and synthetic data reaches 71.70% foreground mIoU, compared with 26.07% for the strongest evaluated zero-shot baseline, CAT-Seg. Adding synthetic data to real-only training improves five of seven architectures, with a maximum gain of 7.19 mIoU points for SegMAN. EncNet also improves (+2.34 points); DeepLabv3+ and SAM 3 perform best on real-only training. Combining curated and synthetic data yields SegMAN's largest gain over real-only training: +9.50 points.
Results follow Tables 2 and 4 of the paper, using its training data and benchmark masks. Foreground mIoU excludes background; background-inclusive averages in the checkpoint documentation use a different denominator. Dataset counts above describe the current release.
Transfer to Assistive Robots
Offline evaluation on egocentric footage from Glide (157 images, 5 labelled classes) and a quadruped guide dog robot (155 images, 9 labelled classes).
Adding synthetic supervision to SegFormer improves foreground mIoU from 11.94% to 16.17% on Glide (+4.23 points) and from 21.56% to 23.08% on the quadruped (+1.52 points), as reported in Table 5 of the paper. Mean precision also improves on both platforms.
These are relative transfer gains. Absolute accuracy of 16.17%–23.08% foreground mIoU does not establish readiness for autonomous assistive navigation; broader class coverage, geographic diversity, and reliable closed-loop navigation remain open directions.
BibTeX
If you use NavAble in your research, please cite our paper.
@inproceedings{hwang2026navable,
title = {{NavAble}: A Large-Scale Dataset and Synthetic Data Generation Pipeline for Blind Navigation},
author = {Hwang, Hochul and Monon, Jahir Sadik and Yang, Soowan and
Patel, Shiven Umeshbhai and Nguyen, Anh Nhat Hong and Nguyen, Kien and
Garg, Keshav and Gage, Dylan and Hordofaa, Duretti and
Anjna, Anshu and Ohn-Bar, Eshed and Kim, Donghyun},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
url = {https://neurips.cc/virtual/2026/loc/sydney/poster/138983}
}