Learning-based techniques for autonomous agent task allocation
Techniques are disclosed to perform task allocation for autonomous systems by implementing machine-learning to perform task allocation to Autonomous Mobile Robots (AMRs) in an environment. The disclosed techniques also provide for enhanced path planning and the identification of AMR health and failure prediction to further improve upon task allocation and system efficiency.
1 . A computing device, comprising:
a memory configured to store computer-readable instructions; and
processing circuitry configured to execute the computer-readable instructions to cause the computing device to:
receive autonomous agent task data including (i) a set of tasks to be performed by a plurality of autonomous agents operating within an environment, and (ii) a set of task parameters identified with each respective one of the set of tasks,
train a neural network using a training dataset that includes a set of target vectors representing an allocation of an autonomous agent to each respective one of the set of tasks based upon a set of reward metrics,
wherein the training dataset is generated by:
calculating, for each one of the set of tasks, a respective set of reward metrics that assigns a reward metric to each one of the plurality of autonomous agents for potentially performing the respective task by computationally determining a result of allocating each autonomous agent to the respective task based on (i) the task parameters, and (ii) cost factors associated with respective ones of the plurality of autonomous agents; and
determining a respective target vector for each one of the set of tasks by allocating an autonomous agent from among the plurality of autonomous agents having the highest respective calculated reward metric for each one of the set of tasks, and
wherein each reward metric from among the set of reward metrics corresponds to a result of allocating one of the plurality of autonomous agents to perform each respective one of the set of tasks;
generate, by providing (i) the autonomous agent task data and the set of target vectors as inputs to an input layer of the trained neural network, and (ii) autonomous agent task allocation data as an output of an output layer of the trained neural network, the autonomous agent task allocation data representing an allocation of each one of the plurality of autonomous agents to a respective one of the set of tasks; and
calculate a route for the plurality of autonomous agents to follow to perform their respectively allocated one of the set of tasks, and transmit route data to the plurality of autonomous agents to cause the plurality of autonomous agents to traverse respective routes,
wherein the calculation of the route is performed based upon a model of the environment that is shared among the plurality of autonomous agents and comprises a plurality of cells having a predetermined size, the model of the environment including (i) a cell location of the plurality of cells that is occupied by the plurality of autonomous agents within the environment, and (ii) a mapping of one or more wireless communication metric values to respective cells of the plurality of cells based upon measurements performed by the plurality of autonomous agents in the environment as commanded by the computing device,
wherein the model of the environment used for the calculation of the route is updated over time as the plurality of autonomous agents move among different cells within the environment, and
wherein the route traversed by the plurality of autonomous agents comprises respective cells in the environment having one or more wireless communication metrics in excess of one or more respective predetermined thresholds.
2 . The computing device of claim 1 , wherein each reward metric corresponds to a result of allocating, for each one of the set of tasks, one of the plurality of autonomous agents to perform the each respective one of the set of tasks using (i) a task priority parameter that represents a weighted sum of the set of task parameters for each respective one of the set of tasks, and (ii) a cost parameter that represents a weighted sum of a set of cost factors, each respective one of the set of cost factors being identified with a different state of a respective one of the plurality of autonomous agents when allocated to perform each respective one of the set of tasks.
3 . The computing device of claim 2 , wherein the set of reward metrics represent, for each one of the set of tasks, a further weighted sum of the task priority parameter and the cost parameter.
4 . The computing device of claim 1 , wherein the processing circuitry is configured to calculate the route for one of the plurality of autonomous agents to follow to perform the respective allocated one of the set of tasks further based upon a presence of obstacles in the environment.
5 . The computing device of claim 4 , wherein the processing circuitry is configured to calculate the route by:
eliminating cells having a wireless connectivity metric that is less than a threshold value to provide a subset of cells;
calculating a plurality of routes within the subset of cells;
for each one of the plurality of routes, calculating a congestion factor based upon cells along each respective route that are occupied by objects; and
selecting, as the calculated route, a route from among the plurality of routes having a lowest congestion factor.
6 . The computing device of claim 5 ,
wherein the processing circuitry is configured to calculate the congestion factor for each one of the plurality of routes by:
computing, for each cell of a respective route, a congestion parameter based upon a proportion of a number of adjacent cells occupied by an object to a total number of the adjacent cells; and
computing the congestion factor by aggregating calculated congestion parameters of cells along the respective route.
7 . The computing device of claim 1 , wherein the processing circuitry is further configured to:
calculate an expected probability of task completion for one of the plurality of autonomous agents to perform an allocated one of the set of tasks by calculating an error metric with respect to a deviation between a predetermined movement profile and a measured movement profile while previously performing tasks from among the set of tasks, and
wherein the generated autonomous agent allocation data that allocates each one of the plurality of autonomous agents to a respective one of the set of tasks is further based upon the expected probability of the one of the plurality of autonomous agents performing the allocated one of the set of tasks.
8 . The computing device of claim 7 , wherein the error metric is from among a set of error metrics calculated based upon a respective predetermined movement profile and a respective measured movement profile with respect to when the one of the plurality of autonomous agents previously performed different portions of the set of tasks, and
wherein the processing circuitry is further configured to subsequently generate further autonomous agent allocation data that allocates a respective one of the set of tasks to the one of the plurality of autonomous agents based upon the set of error metrics.
9 . The computing device of claim 1 , wherein the processing circuitry is configured to calculate a health metric for an autonomous agent from among the plurality of autonomous agents based upon a measured performance of the autonomous agent when performing previous tasks.
10 . The computing device of claim 9 , wherein the processing circuitry is further configured to:
calculate an expected probability of task completion for the autonomous agent from among the plurality of autonomous agents to perform an allocated one of the set of tasks, and
subsequently generate further autonomous agent allocation data that allocates a respective one of the set of tasks to the one of the plurality of autonomous agents based upon the health metric and/or the expected probability of task completion for the autonomous agent.
11 . The computing device of claim 1 , further comprising:
a communication interface configured to receive task-identification data transmitted by an autonomous agent from among the plurality of autonomous agents, the task-identification data being indicative of a further task detected by the autonomous agent, wherein:
the autonomous agent acquires an image of an object in the environment,
the autonomous agent detects the further task by comparing features extracted from the acquired image to features extracted from a matching image of the object accessed via a memory, and
the processing circuitry is further configured to subsequently generate further autonomous agent task allocation data that allocates the further task to one of the plurality of autonomous agents based upon the task-identification data.
12 . An edge network computing device, comprising:
processing circuitry configured to execute computer-readable instructions to cause the edge network computing device to:
receive autonomous agent task data including (i) a set of tasks to be performed by a plurality of autonomous agents operating within an environment, and (ii) a set of task parameters identified with each respective one of the set of tasks,
train a neural network using a training dataset that includes a set of target vectors representing an allocation of an autonomous agent to each respective one of the set of tasks based upon a set of reward metrics, wherein the training dataset is generated by:
calculating, for each one of the set of tasks, a respective set of reward metrics that assigns a reward metric to each one of the plurality of autonomous agents for potentially performing the respective task by computationally determining a result of allocating each autonomous agent to the respective task based on (i) the task parameters, and (ii) cost factors associated with respective ones of the plurality of autonomous agents; and
determining a respective target vector for each one of the set of tasks by allocating an autonomous agent from among the plurality of autonomous agents having the highest respective calculated reward metric for each one of the set of tasks, and
wherein each reward metric from among the set of reward metrics corresponds to a result of allocating one of the plurality of autonomous agents to perform each respective one of the set of tasks; and
generate, by providing the (i) autonomous agent task data and the set of target vectors as inputs to an input layer of the trained neural network, and (ii) autonomous agent task allocation data as an output of an output layer of the trained neural network, the autonomous agent task allocation data representing an allocation of each one of the plurality of autonomous agents to a respective one of the set of tasks; and
calculate a route for the plurality of autonomous agents to follow to perform their respectively allocated one of the set of tasks; and
a communication interface configured to transmit commands to each one of the plurality of autonomous agents to cause the plurality of autonomous agents to traverse their respectively calculated routes and to perform their respectively allocated tasks along their respective routes,
wherein the calculation of the route is performed based upon a model of the environment that is shared among the plurality of autonomous agents and comprises a plurality of cells having a predetermined size, the model of the environment including (i) a cell location of the plurality of cells that is occupied by the plurality of autonomous agents within the environment, and (ii) a mapping of one or more wireless communication metric values to respective cells of the plurality of cells based upon measurements performed by the plurality of autonomous agents in the environment as commanded by the computing device,
wherein the model of the environment used for the calculation of the route is updated over time as the plurality of autonomous agents move among different cells within the environment, and
wherein the route traversed by the plurality of autonomous agents comprises respective cells in the environment having one or more wireless communication metrics in excess of one or more respective predetermined thresholds.
13 . The edge network computing device of claim 12 , wherein each reward metric corresponds to a result of allocating, for each one of the set of tasks, one of the plurality of autonomous agents to perform the respective one of the set of tasks using (i) a task priority parameter that represents a weighted sum of the set of task parameters for the respective one of the set of tasks, and (ii) a cost parameter that represents a weighted sum of a set of cost factors, each respective one of the set of cost factors being identified with a different state of a respective one of the plurality of autonomous agents when allocated to perform the respective one of the set of tasks.
14 . The edge network computing device of claim 13 , wherein the set of reward metrics represent, for each one of the set of tasks, a further weighted sum of the task priority parameter and the cost parameter.
15 . The edge network computing device of claim 12 , wherein the processing circuitry is configured to calculate the route for one of the plurality of autonomous agents to follow to perform the respective allocated one of the set of tasks further based upon congestion along the route caused by a presence of obstacles in the environment.
16 . The edge network computing device of claim 15 , wherein the processing circuitry is configured to calculate the route by:
eliminating cells having a wireless connectivity metric that is less than a threshold value to provide a subset of cells;
calculating a plurality of routes within the subset of cells;
for each one of the plurality of routes, calculating a congestion factor based upon cells along each respective route that are occupied by objects; and
selecting, as the calculated route, a route from among the plurality of routes having a lowest congestion factor.
17 . The edge network computing device of claim 16 ,
wherein the processing circuitry is configured to calculate the congestion factor for each one of the plurality of routes by:
computing, for each cell of a respective route, a congestion parameter based upon a proportion of a number of adjacent cells occupied by an object to a total number of the adjacent cells; and
computing the congestion factor by aggregating calculated congestion parameters of cells along the respective route.
18 . The edge network computing device of claim 12 , wherein the processing circuitry is configured to:
calculate an expected probability of task completion for one of the plurality of autonomous agents to perform an allocated one of the set of tasks by calculating an error metric with respect to a deviation between a predetermined movement profile and a measured movement profile while previously performing tasks from among the set of tasks, and
wherein the generated autonomous agent allocation data that allocates each one of the plurality of autonomous agents to a respective one of the set of tasks is further based upon the expected probability of the one of the plurality of autonomous agents performing the allocated one of the set of tasks.
19 . The edge network computing device of claim 18 , wherein the error metric is from among a set of error metrics calculated based upon a respective predetermined movement profile and a respective measured movement profile with respect to when the one of the plurality of autonomous agents previously performed different portions of the set of tasks, and
wherein the processing circuitry is further configured to subsequently generate further autonomous agent allocation data that allocates a respective one of the set of tasks to the one of the plurality of autonomous agents based upon the set of error metrics.
20 . The edge network computing device of claim 12 , wherein the processing circuitry is configured to calculate a health metric for an autonomous agent from among the plurality of autonomous agents based upon a measured performance of the autonomous agent when performing previous tasks.
21 . The edge network computing device of claim 20 , wherein the processing circuitry is further configured to:
calculate an expected probability of task completion for the autonomous agent from among the plurality of autonomous agents to perform an allocated one of the set of tasks, and
subsequently generate further autonomous agent allocation data that allocates a respective one of the set of tasks to the one of the plurality of autonomous agents based upon the health metric and/or the expected probability of task completion for the autonomous agent.
22 . The edge network computing device of claim 12 , wherein:
the communication interface is configured to receive task-identification data transmitted by an autonomous agent from among the plurality of autonomous agents, the task-identification data being indicative of a further task detected by the autonomous agent,
the autonomous agent acquires an image of an object in the environment,
the autonomous agent detects the further task by comparing features extracted from the acquired image to features extracted from a matching image of the object accessed via a memory, and
the neural network is further configured to subsequently generate further autonomous agent task allocation data that allocates the further task to one of the plurality of autonomous agents based upon the task-identification data.
23 . The computing device of claim 1 , wherein each target vector represents a calculated allocation of an autonomous agent to a respective task.
24 . The computing device of claim 1 , wherein the mapping of the one or more wireless communication metric values to the respective cells of the plurality of cells is based upon an aggregation of measurements performed by the plurality of autonomous agents in response to commands transmitted by the computing device to the plurality of autonomous agents to perform the measurements at specific cell locations over a period of time as the plurality of autonomous agents traverse different cells.