Intelligent routing method for routing items through a network
A method for selecting routes for items being sent from a given beginning node to a given destination node within a network. Each segment of a trip is evaluated and assigned a negative reward based on the deviation from the ideal length of time to complete that segment. Upon arrival of the item at the destination node, the route is assigned a large positive reward. The system is configured to attempt to maximize rewards at all times by selecting the routes with the lowest likelihood of delay. As more and more trips are completed, the control system may define a policy that selects routes based on their probability of yielding the highest reward.
1. A method for determining a route from a beginning node to a destination node in a distribution network, the method comprising:
providing a plurality of nodes connected by a plurality of links that provide a plurality of routes from the beginning node to the destination node, wherein
each of the plurality of nodes represents a transfer point between links, and wherein
each link represents a segment of the plurality of routes;
providing an item for traversing the distribution network;
selecting one of the plurality of routes;
traversing the selected route by the item;
evaluating an efficiency of each segment of the selected route as traversed by the item;
calculating a reward based on the item traversing the selected route, wherein calculating the reward based on the item traversing the selected route comprises:
assigning a negative reward for each segment of the selected route based on any delay on the segment; and
assigning a positive reward for each segment of the selected route based on completion of the segment; and
summing the positive and negative rewards for all segments of the selected route;
tabulating the reward;
iteratively traversing the distribution network by other items along others of the plurality of routes and tabulating the rewards; and
selecting a preferred route corresponding with a highest reward based on tabulated rewards for future items to traverse the distribution network.
2. The method of claim 1 , wherein the distribution network is selected from the group consisting of: a pneumatic tube network, a rail network, and a conveyor belt network.
3. The method of claim 1 , wherein the assigning a negative reward is calculated by the formula:
[ W (estimated)− W (actual)] R =reward, where
W (estimated) is a first amount of time to complete the segment with no delays;
W(actual) is a second amount of time to actually complete the segment; and
R is a predetermined reward rate.
4. The method of claim 1 , further comprising:
assigning a value to the selected route;
calculating a policy corresponding with values of all selected routes; and
selecting the policy to maximize the reward.
5. The method of claim 4 , wherein the policy defines a best route discovered represented by the formula:
π( s )=argmax s′ Σ,P ( s′|s 1 a )∨( s ′)
where
π is the policy;
P is a state transition function;
s (state) is a possible trip segment of a route; and
a is the action of selecting a route.