Distilling information for crowd-aware robot navigation
A navigating device includes: a camera configured to capture images within a field of view, the field of view depicting a scene including humans; a feature module configured to generate feature vectors based on the scene of humans in the images and to specify latent vectors that summarize movement of the humans in the scene based only on the camera images, the feature vectors summarizing the movement of the humans in the scene, and the latent vectors capturing a latent representation of trajectories in the scene of humans; a policy module configured to generate actions to be taken by the navigating device to navigate the scene of humans based on the feature vectors; and a propulsion control module configured to control one or more propulsion devices of the navigating device based on the actions to be taken generated by the policy module to navigate the scene of humans.
1 . A navigating device, comprising:
a camera configured to capture images within a field of view of the navigating device, the field of view depicting a scene including objects;
an object detection module configured to generate detections for the objects in the scene, the detections including at least one of position, size, and bounding box confidence associated with the objects;
a feature module configured to generate feature vectors based on and the detections and to specify latent vectors that summarize movement of the objects in the scene based only on the camera images, the feature vectors summarizing the movement of the objects in the scene, and the latent vectors capturing a latent representation of trajectories in the scene of the objects;
a policy module configured to generate actions to be taken by the navigating device to navigate the scene based on the feature vectors and the latent vectors; and
a propulsion control module configured to control one or more propulsion devices of the navigating device based on the actions to be taken generated by the policy module to navigate the scene.
2 . The navigating device of claim 1 wherein the navigating device is one of a robot and an autonomous vehicle.
3 . The navigating device of claim 1 wherein the policy module is trained jointly with an encoder module using reinforcement learning, the encoder module being configured to generate feature vectors based on training data including positions of the objects that are humans; and
wherein the feature module is trained using supervised learning to approximate the output of the encoder module based on training data without explicit positions of humans.
4 . The navigating device of claim 1 wherein the policy module is trained jointly with the feature module using reinforcement learning based on training data that includes detections of the objects that are humans without explicit positions of humans.
5 . A navigating robot, comprising:
a camera configured to capture images within a field of view in front of the navigating robot;
a feature module configured to generate feature vectors based on objects in the images;
a policy module configured to generate actions to be taken by the navigating robot based on the feature vectors; and
a propulsion control module configured to control one or more propulsion devices of the navigating robot based on the actions to be taken,
wherein the policy module is trained jointly with an encoder module using reinforcement learning, the encoder module being configured to generate feature vectors based on training data including positions of the objects; and
the feature module is trained using supervised learning based on training data that does not include positions of the objects and that includes one of (a) detections of the objects in images and (b) depth images including the objects.
6 . The navigating robot of claim 5 wherein the policy module and the encoder module are trained jointly using a proximal policy optimization (PPO) algorithm based on maximizing a reward.
7 . The navigating robot of claim 5 wherein the feature module is trained using supervised learning based on training data that does not include positions of the objects and that includes depth images.
8 . The navigating robot of claim 7 wherein the feature module is trained based on minimizing a loss determined based on differences between (a) outputs of the feature module generated based on samples of the training data, respectively, and (b) stored outputs associated with the ones of the samples, respectively.
9 . The navigating robot of claim 7 wherein the feature module includes:
a convolutional neural network (CNN) module;
a flattening module;
an attention module; and
a multilayer perceptron (MLP) module configured to generate the feature vectors.
10 . The navigating robot of claim 9 wherein:
the CNN module is configured to extract features from the depth images; and
the MLP module configured to generate the feature vectors based on outputs of the attention module.
11 . The navigating robot of claim 5 wherein the feature module is trained using supervised learning based on training data that does not include positions of the objects and that includes detections of the objects.
12 . The navigating robot of claim 11 further comprising a detector module configured to generate the detections in the images.
13 . The navigating robot of claim 12 wherein the detector module is configured to generate the detections using the YOLOv4 detector algorithm.
14 . The navigating robot of claim 13 wherein the feature module includes:
a first multilayer perceptron (MLP) module configured to receive the detections;
a concatenation module;
a second MLP module;
an attention module; and
a third MLP module configured to generate the feature vectors.
15 . The navigating robot of claim 5 wherein the navigating robot does not include any light detection and ranging (LIDAR) sensors.
16 . A training system, comprising:
a training dataset including:
privileged training samples including positions of humans in images; and
non-privileged training samples that do not include positions of humans and that include one of (a) detections of humans in images and (b) depth images including humans; and
a training module configured to:
during a first portion of training, jointly train an encoder module and a policy module for a navigating robot using reinforcement learning based on ones of the privileged training samples of the training dataset,
wherein the policy module is configured to generate actions to be taken by the navigating robot based on feature vectors generated by a feature module, and
wherein the feature module is configured to generate the feature vectors based on humans in images captured using a camera of the navigating robot; and
during a second portion of the training after the first portion of the training, train the feature module based on ones of the non-privileged training samples.
17 . The training system of claim 16 wherein the training module is configured to, during the second portion of the training, train the feature module using supervised learning.
18 . The training system of claim 16 wherein the training module is configured to jointly train the policy module and the encoder module during the first portion of the training using a proximal policy optimization (PPO) algorithm based on maximizing a reward.
19 . The training system of claim 16 wherein the non-privileged training samples include one of (a) detections of humans in images and (b) depth images including humans.
20 . The training system of claim 16 wherein the training module is configured to train the feature module during the second portion of the training based on minimizing a loss determined based on differences between (a) outputs of the feature module generated based on ones of the non-privileged training samples, respectively, and (b) stored outputs associated with the ones of the non-privileged training samples, respectively.
21 . A navigating robot, comprising:
a camera configured to capture images within a field of view in front of the navigating robot;
a feature module configured to generate feature vectors based on objects in the images;
a policy module configured to generate actions to be taken by the navigating robot based on the feature vectors; and
a propulsion control module configured to control one or more propulsion devices of the navigating robot based on the actions to be taken;
wherein the policy module is trained jointly with the feature module using reinforcement learning based on training data that includes detections of the objects without explicit positions of the objects.
22 . The navigating robot of claim 21 wherein the feature module is configured to:
generate detections of the objects in images using the YOLOv4 detector algorithm; and
generate the feature vectors based on the detections.
23 . The navigating robot of claim 21 wherein the policy module and the feature module are trained using a proximal policy optimization (PPO) algorithm based on maximizing a reward.
24 . A training system, comprising:
a training dataset including:
non-privileged training samples that do not include positions of humans and that include detections of humans in images; and
a training module configured to:
jointly train a feature module and a policy module for a navigating robot using reinforcement learning based on ones of the non-privileged training samples of the training dataset using reinforcement learning,
wherein the policy module is configured to generate actions to be taken by the navigating robot based on feature vectors generated by the feature module, and
wherein the feature module is configured to generate the feature vectors based on detections of humans in images captured using a camera of the navigating robot.
25 . The training system of claim 24 wherein the training module is configured to jointly train the policy module and the feature module using a proximal policy optimization (PPO) algorithm based on maximizing a reward.
26 . A navigating device, comprising:
the encoder module and the policy module trained by the training system of claim 16 ;
a camera configured to capture images within a field of view of the navigating device, the field of view depicting a scene including objects;
the encoder module configured to generate feature vectors based on the scene of the objects in the images captured by the camera and to specify latent vectors that summarize movement of the objects in the scene based only on the camera images, the feature vectors summarizing the movement of the objects in the scene, and the latent vectors capturing a latent representation of trajectories in the scene of the objects;
the policy module configured to generate actions to be taken by the navigating device to navigate the scene based on the feature vectors; and
a propulsion control module configured to control one or more propulsion devices of the navigating device based on the actions to be taken generated by the policy module to navigate the scene.