×

SideNav

From Cognition to Precognition:
A Future-Aware Framework for Social Navigation

Zeying Gong     Tianshuai Hu     Ronghe Qiu     Junwei Liang    
Accepted at ICRA 2025
arXiv Video Code Datasets

Overview Video

Abstract

To navigate safely and efficiently in crowded spaces, robots should not only perceive the current state of the environment but also anticipate future human movements.
We integrate trajectory prediction into the SocialNav task.

We integrate trajectory prediction into the SocialNav task.

In this paper, we propose a reinforcement learning architecture, namely Falcon (future-aware SocialNav framework with precognition), to tackle socially-aware navigation by explicitly predicting human trajectories and penalizing actions that block future human paths.
Falcon Overview

Falcon Overview.

To facilitate realistic evaluation, we introduce a novel SocialNav benchmark containing two new datasets, Social-HM3D and Social-MP3D. This benchmark offers large-scale photo-realistic indoor scenes populated with a reasonable amount of human agents based on scene area size, incorporating natural human movements and trajectory patterns.
Benchmark Overview

Benchmark Overview (Social-HM3D & Social-MP3D).

We conduct a detailed experimental analysis with the state-of-the-art learning-based method and two classic rule-based path-planning algorithms on the new benchmark. The results demonstrate the importance of future prediction and our method achieves the best task success rate of 55% while maintaining about 90% personal space compliance.
The four different classes of encounter

More Videos: Classic Encounters in SocialNav

We show our method on four classic categories of encounters for SocialNav (inspired by Cancelli et al. [1] and Pirk et al. [2]). Each encounter displays three videos from each of our two datasets (Social-HM3D & Social-MP3D).

The four different classes of encounter

The four different classes of encounter. The dashed line represents the general direction of the agent and the person involved. The red area represents the agent’s field of view at the beginning of the encounter.

(a) Frontal Approach

In this case, the robot and a human are approaching each other from opposite directions.
Our method uses 'Step Aside' to maintain a safe distance or 'Predictive Shift' to proactively avoid potential collisions by predicting the human's path.

Social-HM3D

Step Aside
Predictive Shift
Predictive Shift

Social-MP3D

Step Aside
Predictive Shift
Predictive Shift

(b) Intersection

In an intersection encounter, the robot and human paths cross each other.
Our approach utilizes 'Early Pass' to pass ahead when it is safe, 'Predictive Bypass' to adjust its path based on the predicted movement, or 'Proactive Yield' where the robot either stops to let the human pass or senses it is in the way and moves aside to yield the path to the human.

Social-HM3D

Early Pass
Predictive Bypass
Proactive Yield

Social-MP3D

Early Pass
Predictive Bypass
Proactive Yield

(c) Blind Corner

This occurs when visibility is limited, and the robot may encounter a human unexpectedly.
Our method involves 'Proactive Avoidance' when a human is detected to safely move aside, or 'Reactive Sidewalk' to instinctively walk closer to the side when visibility is low.

Social-HM3D

Reactive Sidewalk
Proactive Avoidance
Proactive Avoidance

Social-MP3D

Reactive Sidewalk
Proactive Avoidance
Proactive Avoidance

(d) Person Following

This encounter occurs when a human is in the robot's path.
The robot either 'Brief Tail' until the human leaves its path or 'Follow & Overtake' when the human remains in its way.

Social-HM3D

Brief Tail
Follow & Overtake
Follow & Overtake

Social-MP3D

Brief Tail
Follow & Overtake
Follow & Overtake

BibTeX

@misc{gong2024cognitionprecognitionfutureawareframework,
  title={From Cognition to Precognition: A Future-Aware Framework for Social Navigation}, 
  author={Zeying Gong and Tianshuai Hu and Ronghe Qiu and Junwei Liang},
  year={2024},
  eprint={2409.13244},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2409.13244},  
}

References

[1] Cancelli, Enrico, et al. "Exploiting Proximity-Aware Tasks for Embodied Social Navigation." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[2] Pirk S, Lee E, Xiao X, et al. A protocol for validating social navigation policies[J]. arXiv preprint arXiv:2204.05443, 2022.