Machine-learning application proxy for IoT devices including large-scale data collection using dynamic servlets with access control
An apparatus and method for providing ML processing for one or more ML applications operating on one or more Internet of Things (IoT) devices includes receiving a ML request from an IoT device. The ML request can be generated by a ML application operating on the IoT device and include input data collected by the first ML application. A ML model to perform ML processing of the input data included in the ML request is identified and provided to an ML core for ML processing along with the input data included in the first ML request. The ML core produces ML processing output data based on ML processing by the ML core of input data included in the ML request using the ML model. The ML processing output data can be transmitted to the IoT device.
1. A machine-learning (ML) proxy device providing ML processing for one or more ML applications operating on one or more Internet of Things (IoT) devices, the ML proxy device configured to:
at one or more processors:
receive an ML request from an IoT device, the ML request being generated by an ML application operating on the IoT device and including first input data collected by the ML application;
receive second input data from one or more transmitted servlets, wherein the one or more transmitted servlets are dynamically allocated according to ML data collection requirements of the ML application operating on the IoT device;
receive, from a configuration device external to the ML proxy device and the IoT device, a configuration file;
identify, based on the configuration file, an ML model to perform ML processing of the first input data and the second input data;
provide the ML model, the first input data, and the second input data to a ML core for ML processing;
receive ML processing output data from the ML core as a result of ML processing by the ML core of the first input data and the second input data using the ML model; and
transmit the ML processing output data to the IoT device.
2. The ML proxy device of claim 1 , wherein the one or more processors is further configured to:
transmit one or more servlets to collect data from one or more data sources; and
provide the ML core the collected data from the one or more servlets for the ML processing.
3. The ML proxy device of claim 1 , wherein the one or more processors is further configured to convert the first input data collected by the ML application into a format used by the ML core.
4. The ML proxy device of claim 1 , wherein the one or more processors is further configured to:
receive a plurality of ML requests from one or more IoT devices;
generate a schedule for ML processing of the plurality of ML requests based on priority queues; and
perform ML processing by one or more ML cores of the plurality of ML requests according to the priority queues.
5. The ML proxy device of claim 4 , wherein the one or more processors is further configured to:
assign at least one of the plurality of ML requests to a session in the generated schedule; and
complete all of the ML processing for the ML request assigned to the session when the ML core begins the ML processing of that ML request according to the generated schedule.
6. The ML proxy device of claim 4 , wherein the one or more processors is further configured to:
assign at least one of the plurality of ML requests to at least first and second segments in the generated schedule; and
perform ML processing by an ML core of at least one other ML request between the ML processing of the first and second segments by the ML core according to the generated schedule.
7. The ML proxy device of claim 4 , wherein:
each of the plurality of ML requests includes indicating priority information for processing the ML request, and
the one or more processors is further configured to generate the schedule for ML processing of the plurality of ML requests based on the priority information.
8. The ML proxy device of claim 1 , wherein the one or more processors is further configured to:
communicate with the IoT device using data transport protocol; and
bridge the communication with the IoT device with an IoT adaptor when the IoT device does not support the data transport protocol.
9. The ML proxy device of claim 1 , wherein the one or more processors is further configured to progressively pre-compile the ML model into specific binary codes of the ML core during a downloading the ML model to the ML proxy device.
10. The ML proxy device of claim 1 , wherein the ML application collects data in accordance with information provided by the configuration device.
11. The ML proxy device of claim 1 , wherein the configuration device is separate from the IoT device.
12. The ML proxy device of claim 1 , wherein the one or more processors is further configured to responsive to a data request, determine the one or more servlets responsible for processing the data request.
13. The ML proxy device of claim 1 , wherein:
the one or more transmitted servlets configured with data access control rules, and
the one or more transmitted servlets are configured to collect the first input data generated by ML application operating on the IoT device only if the data access control rules provide permission to make the collection.
14. A method for providing machine-learning (ML) processing for one or more ML applications operating on one or more Internet of Things (IoT) devices, the method comprising, by an ML proxy device:
receiving an ML request from an IoT device, the ML request being generated by an ML application operating on the IoT device and including first input data collected by the ML application;
transmitting one or more servlets configured with data access control rules, wherein the one or more servlets are further configured to collect second input data generated by the ML application in response to the data access control rules providing permission to make the collection;
receiving, from a configuration device external to the ML proxy device and the IoT device, a configuration file;
identifying, based on the configuration file, an ML model to perform ML processing of the input data;
providing the ML model, the first input data, and the second input data to a ML core for ML processing;
receiving ML processing output data from the ML core as a result of ML processing by the ML core of the first input data and the second input data using the ML model; and
transmitting the ML processing output data to the IoT device.
15. The method of claim 14 , the method further comprising:
transmitting one or more servlets to collect data from one or more data sources; and
providing the ML core the collected data from the one or more servlets for the ML processing.
16. The method of claim 14 , the method further comprising:
receiving a plurality of ML requests from one or more IoT devices at the ML proxy device;
generating a schedule for ML processing of the plurality of ML requests; and
performing ML processing by one or more ML cores of the plurality of ML requests according to the generated schedule.
17. A non-transitory, computer-readable medium storing instructions that, when executed by a machine-learning (ML) proxy device providing ML processing for one or more ML applications operating on one or more Internet of Things (IoT) devices, control the ML proxy device to implement a method comprising:
receiving an ML request from an IoT device, the ML request being generated by an ML application operating on the IoT device and including first input data collected by the ML application;
receiving second input data from one or more transmitted servlets, wherein the one or more transmitted servlets are dynamically allocated according to ML data collection requirements of the ML application operating on the IoT device;
receiving, from a configuration device external to the ML proxy device and the IoT device, a configuration file;
identifying, based on the configuration file, an ML model to perform ML processing of the input data;
providing the ML model, the input data, and the second input data to a ML core for ML processing;
receiving ML processing output data from the ML core as a result of ML processing by the ML core of the first input data and the second input data using the ML model; and
transmitting the ML processing output data to the IoT device.
18. The non-transitory, computer-readable medium of claim 17 , the method further comprising:
transmitting the one or more servlets to collect data from one or more data sources; and
providing the ML core the collected data from the one or more servlets for the ML processing.
19. The non-transitory, computer-readable medium of claim 17 , the method further comprising:
receiving a plurality of ML requests from one or more IoT devices at the ML proxy device;
generating a schedule for ML processing of the plurality of ML requests based on priority queues; and
performing ML processing by one or more ML cores of the plurality of ML requests according to the priority queues.