Safe Navigation With Human Instructions in Complex Scenes

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

7 Scopus Citations
View graph of relations

Author(s)

  • Zhe Hu
  • Jia Pan
  • Tingxiang Fan
  • Ruigang Yang
  • Dinesh Manocha

Detail(s)

Original languageEnglish
Pages (from-to)753-760
Journal / PublicationIEEE Robotics and Automation Letters
Volume4
Issue number2
Online published16 Jan 2019
Publication statusPublished - Apr 2019

Abstract

In this letter, we present a robotic navigation algorithm with natural language interfaces that enables a robot to safely walk through a changing environment with moving persons by following human instructions such as "go to the restaurant and keep away from people." We first classify human instructions into three types: goal, constraints, and uninformative phrases. Next, we provide grounding in a dynamic manner for the extracted goal and constraint items along with the navigation process to deal with target objects that are too far away for sensor observation and the appearance of moving obstacles such as humans. In particular, for a goal phrase (e.g., "go to the restaurant"), we ground it to a location in a predefined semantic map and treat it as a goal for a global motion planner, which plans a collision-free path in the workspace for the robot to follow. For a constraint phrase (e.g., "keep away from people"), we dynamically add the corresponding constraint into a local planner by adjusting the values of a local costmap according to the results returned by the object detection module. The updated costmap is then used to compute a local collision avoidance control for the safe navigation of the robot. By combining natural language processing, motion planning, and computer vision, our developed system can successfully follow natural language navigation instructions to achieve navigation tasks in both simulated and real-world scenarios. Videos are available at https://sites.google.com/view/snhi.

Research Area(s)

  • Social human-robot interaction, visual-based navigation, motion and path planning, semantic scene understanding, robot audition

Citation Format(s)

Safe Navigation With Human Instructions in Complex Scenes. / Hu, Zhe; Pan, Jia; Fan, Tingxiang; Yang, Ruigang; Manocha, Dinesh.

In: IEEE Robotics and Automation Letters, Vol. 4, No. 2, 04.2019, p. 753-760.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review