Edge processing of sensor data using a neural network to reduce data traffic on a communication network
Methods, systems, and apparatuses related to edge processing of sensor data using a neural network to reduce network traffic to and/or from a server. In one approach, a cloud server processes sensor data from a vehicle using an artificial neural network (ANN). The ANN has several layers. Based on analyzing at least one characteristic of the sensor data received from the vehicle and/or a context associated with processing the sensor data, the cloud server determines to send one or more of the layers of the ANN for edge processing on the vehicle itself. In other cases, the cloud server decides to send the one or more layers to an edge server device located on a communication path between the vehicle and the cloud server. The edge processing reduces network data traffic.
1. A system comprising:
at least one processing device; and
memory containing instructions configured to instruct the at least one processing device to:
receive data over a network from a computing device, wherein the received data is processed using an artificial neural network (ANN), the ANN comprises a neuron model used for processing the data, and the computing device provides the data from processing sensor data obtained from at least one sensing device of a vehicle, wherein the sensor data corresponds to an image collected by an image sensor of the vehicle;
send, to the computing device, a first portion of the neuron model, wherein the first portion is selected based on predicted data traffic associated with processing the sensor data, the predicted data traffic including at least one of data traffic to the computing device, or data traffic from the computing device;
receive, from the computing device, an intermediate result, wherein the computing device provides the intermediate result based on processing the sensor data using the first portion, wherein the intermediate result comprises at least one of an identification of an object in the image, or a classification of the object; and
provide, by processing the intermediate result using a second portion of the neuron model, an output.
2. The system of claim 1 , wherein selecting the first portion of the neuron model based on predicted data traffic comprises selecting a set of neurons from the neuron model, and wherein selecting the set of neurons is based on at least one of comparing a data size of input data to be processed using the first portion to a data size of output data to be obtained from processing using the first portion, or comparing a data size of input data to be processed using the first portion to a data size of the first portion.
3. The system of claim 1 , wherein the instructions are further configured to instruct the at least one processing device to:
predict, using at least a portion of the ANN, that the data size of the sensor data processed using the first portion is greater than the data size of the first portion;
wherein the first portion is sent to the computing device in response to the predicting that the data size of the sensor data processed using the first portion is greater than the data size of the first portion.
4. A method comprising:
processing, by a first computing device and using an artificial neural network (ANN), data received over a network from a second computing device, wherein the ANN comprises a neuron model used for processing the data, the second computing device processes sensor data from at least one sensing device of a vehicle, and the second computing device provides the received data from processing the sensor data, wherein the sensor data corresponds to an image collected by an image sensor of the vehicle;
determining, by the first computing device, to perform processing for a first portion of the neuron model on the second computing device;
in response to determining to perform processing for the first portion of the neuron model on the second computing device, sending the first portion over the network to the second computing device, wherein the second computing device provides an intermediate result based on processing the sensor data using the first portion, wherein the intermediate result comprises at least one of an identification of an object in the image, or a classification of the object;
receiving, from the second computing device, the intermediate result; and
processing, by the first computing device, the intermediate result using a second portion of the neuron model to provide an output.
5. The method of claim 4 , wherein the first computing device is a first server, and the second computing device is a second server on a communication path between the vehicle and the first server.
6. The method of claim 5 , wherein the determining to perform the processing for the first portion on the second computing device comprises at least one of comparing a data size of the sensor data to a data size of the intermediate result, comparing a data size of the sensor data to a data size of the first portion, or comparing a data size of the intermediate result to a data size of the first portion.
7. The method of claim 6 , wherein the sensor data is an first input vector, and the intermediate result is a first output vector.
8. The method of claim 7 , wherein the determining to perform the processing for the first portion of the neuron model on the second computing device further comprises predicting at least one of the data size of the first input vector, or the data size of the first output vector.
9. The method of claim 4 , wherein the determining to perform the processing for the first portion on the second computing device is based on determining whether the sensor data will be uploaded to the first computing device after the intermediate result is provided.
10. The method of claim 4 , wherein the at least one sensing device comprises an image sensor.
11. The method of claim 4 , further comprising:
determining, by the first computing device, to perform processing for a third portion of the ANN on the vehicle; and
in response to determining to perform processing for the third portion on the vehicle, sending the third portion over a network to the at least one sensing device, wherein:
the at least one sensing device generates the sensor data using the third portion to process at least one image from an image sensor; and
the at least one sensing device sends the generated sensor data to the second computing device for use in determining the intermediate result.
12. The method of claim 4 , further comprising sending, to the vehicle, the output to control an action on the vehicle.
13. The method of claim 4 , wherein the second computing device is a controller in a computer system of the vehicle.
14. The method of claim 4 , wherein the determining to perform the processing for the first portion of the neuron model on the second computing device comprises evaluating data traffic between the first computing device and the second computing device that is associated with providing the output.
15. The method of claim 14 , wherein the data traffic is associated with at least one of receiving the sensor data from the second computing device after determining that the sensor data requires further processing by the ANN at the first computing device, sending the first portion to the second computing device, or receiving the intermediate result from the second computing device.
16. The method of claim 4 , further comprising selecting the first portion of the neuron model for sending to the second computing device, wherein the selecting is based on predicted data traffic between the first computing device and the second computing device.
17. A system comprising:
at least one processing device; and
memory containing instructions configured to instruct the at least one processing device to:
process, using at least one portion of an artificial neural network (ANN), sensor data obtained from at least one sensing device of a vehicle;
send, over a network to a computing device, results from processing the sensor data, wherein the sensor data corresponds to an image collected by an image sensor of the vehicle;
receive, from the computing device, a first portion of the ANN, the first portion selected based on a prediction of data traffic associated with processing the sensor data, the predicted data traffic including at least one of data traffic to the computing device, or data traffic from the computing device; and
send, to the computing device, an intermediate result determined based on processing the sensor data using the first portion, wherein the computing device provides an output used to control the vehicle, and the output is obtained by processing the intermediate result using a second portion of the ANN and wherein the intermediate result comprises at least one of an identification of an object in the image, or a classification of the object.
18. The system of claim 17 , wherein the instructions are further configured to instruct the at least one processing device to:
compare a data size of the sensor data to a data size of the first portion of the ANN; and
send, to the computing device and based on the comparing the data size of the sensor data to the data size of the first portion, a request for the first portion;
wherein the first portion is received from the computing device in response to the request.
19. The system of claim 17 , wherein the first portion is selected to reduce predicted data traffic to and from the computing device when processing the sensor data using the ANN.