Coordinated load balancing in mobile edge computing network
A method includes obtaining at least one policy parameter of a neural network corresponding to a load balancing policy, receiving trajectories for each mobile device in a plurality of mobile devices of the wireless network, each trajectory corresponding to a sequence of states of a respective mobile device, wherein the sequence of states is generated based on a continuous interaction of an existing policy of the respective mobile device with the wireless network, estimating advantage functions for each mobile device in the plurality of mobile devices based on the trajectories for each respective mobile device, and updating the at least one policy parameter based on the estimated advantage functions such that the load balancing policy is determined based on states of each mobile device in the plurality of mobile devices.
1 . A method comprising:
obtaining at least one policy parameter of a neural network corresponding to a load balancing policy;
receiving trajectories for each mobile device in a plurality of mobile devices of the wireless network, each trajectory corresponding to a sequence of states, actions and rewards for the actions of a respective mobile device, wherein the sequence of states is generated based on a continuous interaction of an existing policy of the respective mobile device with the wireless network, the actions indicating a base station to which the mobile device should connect for a request and the rewards being received after the actions guide a learning process;
estimating advantage functions for each mobile device in the plurality of mobile devices based on the trajectories for each respective mobile device; and
updating the at least one policy parameter based on the estimated advantage functions such that the load balancing policy is determined based on states of each mobile device in the plurality of mobile devices,
wherein the advantage functions are determined based on a difference between a cost-to-go function and a value function, and
wherein the sequence of states of each trajectory corresponds to states over a predetermined number of time steps for each mobile device of the plurality of mobile devices.
2 . The method of claim 1 , further comprising:
obtaining at least one value parameter of the neural network corresponding to the load balancing policy; and
updating the at least one value parameter based on the estimated advantage functions.
3 . The method of claim 1 , further comprising deploying the neural network corresponding to the load balancing policy to each mobile device of the plurality of mobile devices in the wireless network.
4 . The method of claim 1 , further comprising:
receiving, as a first input to the neural network corresponding to the load balancing policy, statuses of queues of each base station of a plurality of base stations in the wireless network; and
receiving, as a second input to the neural network corresponding to the load balancing policy, a task request from a first mobile device of the plurality of mobile devices.
5 . The method of claim 4 , further comprising determining a base station of the plurality of base stations for performing the requested task based on the first input and the second input, and
performing a handover operation connecting the first mobile device to the determined base station for performing the requested task.
6 . The method of claim 1 , wherein the wireless network comprising a mobile edge computing (MEC) network.
7 . A system comprising:
a memory storing instructions; and
a processor configured to execute the instructions to:
obtain at least one policy parameter of a neural network corresponding to a load balancing policy;
receive trajectories for each mobile device in a plurality of mobile devices of a mobile edge computing (MEC) network, each trajectory corresponding to a sequence of states, actions and rewards for the actions of a respective mobile device, the actions indicating a base station to which the mobile device should connect for a request and the rewards being received after the actions guide a learning process, wherein the sequence of states is generated based on a continuous interaction of an existing policy of the respective mobile device with the MEC network;
estimate advantage functions for each mobile device in the plurality of mobile devices based on the trajectories for each respective mobile device; and
update the at least one policy parameter based on the estimated advantage functions such that the load balancing policy is determined based on states of each mobile device in the plurality of mobile devices,
wherein the advantage functions are determined based on a difference between a cost-to-go function and a value function, and
wherein the sequence of states of each trajectory corresponds to states over a predetermined number of time steps for each mobile device of the plurality of mobile devices.
8 . The system of claim 7 , wherein the processor is further configured to execute the instructions to:
obtain at least one value parameter of the neural network corresponding to the load balancing policy; and
update the at least one value parameter based on the estimated advantage functions.
9 . The system of claim 7 , wherein the processor is further configured to execute the instructions to deploy the neural network corresponding to the load balancing policy to each mobile device of the plurality of mobile devices in the MEC network.
10 . The system of claim 7 , wherein the processor is further configured to execute the instructions to:
receive, as a first input to the neural network corresponding to the load balancing policy, statuses of queues of each base station of a plurality of base stations in the MEC network; and
receive, as a second input to the neural network corresponding to the load balancing policy, a task request from a first mobile device of the plurality of mobile devices.
11 . The system of claim 10 , wherein the processor is further configured to execute the instructions to determine a base station of the plurality of base stations for performing the requested task based on the first input and the second input, and
perform a handover operation connecting the first mobile device to the determined base station for performing the requested task.
12 . The system of claim 11 , wherein the base station for performing the requested task with the first mobile device is determined at the first mobile device.
13 . A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to:
obtain at least one policy parameter of a neural network corresponding to a load balancing policy;
receive trajectories for each mobile device in a plurality of mobile devices of a mobile edge computing (MEC) network, each trajectory corresponding to a sequence of states, actions and rewards for the actions of a respective mobile device, the actions indicating a base station to which the mobile device should connect for a request and the rewards being received after the actions guide a learning process, wherein the sequence of states is generated based on a continuous interaction of an existing policy of the respective mobile device with the MEC network;
estimate advantage functions for each mobile device in the plurality of mobile devices based on the trajectories for each respective mobile device; and
update the at least one policy parameter based on the estimated advantage functions such that the load balancing policy is determined based on states of each mobile device in the plurality of mobile devices,
wherein the advantage functions are determined based on a difference between a cost-to-go function and a value function, and
wherein the sequence of states of each trajectory corresponds to states over a predetermined number of time steps for each mobile device of the plurality of mobile devices.
14 . The storage medium of claim 13 , wherein the instructions, when executed, further cause the at least one processor to:
obtain at least one value parameter of the neural network corresponding to the load balancing policy; and
update the at least one value parameter based on the estimated advantage functions.
15 . The storage medium of claim 13 , wherein the instructions, when executed, further cause the at least one processor to deploy the neural network corresponding to the load balancing policy to each mobile device of the plurality of mobile devices in the MEC network.