Analytic Approach for Natural Language based Supervisory Control of Robotic Manipulations

Robots have been widely used in industrial and domestic areas. To control their behaviors, multiple approaches have been proposed to reduce the complexity of controlling robots. Among them, natural language (NL) control is attracting increasingly more attention due to its convenience and friendliness for the lay users. Existed approaches of natural language control focus on translating linguistic input into implementable action plans, while less attention were put on model check and property analysis, which are valued important and necessary for practical applications. To provide partial remedies to the problem, we propose to use State Transition Matrix (STM) to model the system behavior at task level. The matrix can be used to analyze the system properties from the control perspective, which provides reference for system design. In addition, STM supports to learn new skill in a hierarchical way with one-shot online interactive training. In this paper, we introduce the STM framework, describe how to analyze system property with STM, elaborate the learning algorithm, and illustrate the utility of this approach with experimental results.