Robotic navigation and transport of objects
Navigational systems and methods include building a topological graph of an environment using nodes that represent locations in the space and associated directions, with frontiers associated with particular nodes and directions within the topological graph. An action is determined using a policy trained with an action reward function that weighs exploration to find new objects and moving objects to a goal. An agent navigates within the environment in accordance with the determined action.
1. A navigation method, comprising:
building a topological graph of an environment using nodes that represent locations in the space and associated directions, with frontiers associated with particular nodes and directions within the topological graph;
determining an action using a policy trained with an action reward function that weighs exploration to find new objects and moving objects to a goal, including calculating an exploration score that represents a likelihood of finding an object upon exploring a frontier and predicting frontier scores on a per-frame basis using a convolutional neural network; and
navigating an agent within the environment in accordance with the determined action.
2. The method of claim 1 , wherein calculating the exploration score includes calculating a respective exploration score for each node-direction combination in the topological graph.
3. The method of claim 1 , wherein calculating the exploration score includes processing the topological graph using a graph convolutional neural network.
4. The method of claim 1 , wherein determining the action selects an action from the group consisting of explore, pickup, and drop and wherein the pickup and drop actions interact with a target object to move the target object from an origin location to a goal location.
5. The method of claim 1 , wherein the agent is an autonomous device and navigating the agent causes the agent to move within the environment.
6. The method of claim 1 , wherein the action reward function is expressed as:
r
t
e
=
success
·
r
s
u
c
c
e
s
s
e
+
r
slack
e
+
∑
o
found
o
·
r
found
e
where found o found is an indicator if an object o was found at a timestep t, r found e is a reward for finding a new object, r stack e is a time penalty for every step that encourages finding objects faster, success is an indicator if all objects were found, and r success e is an associated success bonus.
7. The method of claim 1 , wherein navigating includes using a navigation policy trained with a navigation reward function that weighs distances to objects.
8. The method of claim 7 , wherein the navigation reward function is expressed as:
r t n = [reached-obj] ·r obj n +r slack n +r d2o n +r collision n
where [reached-obj] is an indicator that a target object has been reached, r obj n is the success reward if the agent reaches closer than a threshold distance d th from the target object, r slack n is a constant time penalty for each step, r d2o n =(d t-1 −d t ) is the decrease in geodesic distance with the target object and r collision n is a penalty for collision with an object in the environment.
9. The method of claim 1 , further comprising using the exploration score to identify a frontier sub-goal.
10. The method of claim 1 , further comprising using an object closeness score to indicate a distance to an object within the agent's field of view.
11. A navigation system, comprising:
a hardware processor; and
a memory to store a computer program that, when executed by the hardware processor, causes the hardware processor to:
build a topological graph of an environment using nodes that represent locations in the space and associated directions, with frontiers associated with particular nodes and directions within the topological graph;
determine an action using a policy trained with an action reward function that weighs exploration to find new objects and moving objects to a goal and calculates an exploration score that represents a likelihood of finding an object upon exploring a frontier using a convolutional neural network to predict frontier scores on a per-frame basis; and
navigate an agent within the environment in accordance with the determined action.
12. The system of claim 11 , wherein the calculation of the exploration score includes a respective exploration score for each node-direction combination in the topological graph.
13. The system of claim 11 , wherein the calculation of the exploration score includes a graph convolutional neural network to process the topological graph.
14. The system of claim 11 , wherein the computer program further causes the hardware processor to select an action from the group consisting of explore, pickup, and drop, and wherein the pickup and drop actions interact with a target object to move the target object from an origin location to a goal location.
15. The system of claim 11 , wherein the agent is an autonomous device and navigating the agent causes the agent to move within the environment.
16. The system of claim 11 , wherein the action reward function is expressed as:
r
t
e
=
success
·
r
s
u
c
c
e
s
s
e
+
r
slack
e
+
∑
o
found
o
·
r
found
e
where found o is an indicator if an object o was found at a timestep t, r found e is a reward for finding a new object, r stack e is a time penalty for every step that encourages finding objects faster, success is an indicator if all objects were found, and r success e is an associated success bonus.
17. The system of claim 11 , wherein navigating includes using a navigation policy trained with a navigation reward function that weighs distances to objects.
18. The system of claim 17 , wherein the navigation reward function is expressed as:
r t n = [reached-obj] ·r obj n +r slack n +r d2o n +r collision n
where [reached-obj] is an indicator that a target object has been reached, r obj n is the success reward if the agent reaches closer than a threshold distance d th from the target object, r slack n is a constant time penalty for each step, r d2o n =(d t-1 −d t ) is the decrease in geodesic distance with the target object and r collision n is a penalty for collision with an object in the environment.
19. The system of claim 11 , wherein the exploration score is used to identify a frontier sub-goal.
20. The system of claim 11 , wherein an object closeness score is used to indicate a distance to an object within the agent's field of view.