Automatic Generation Of Labeled Data In IOT Systems
A labeled data generation service provides an Internet-of-Things (IoT) system with a capability whereby users may configure how the system gathers, processes, and generates labeled data instances by: collecting and processing the data into a format required by supervised learning algorithms; generating expected outputs from data available in the IoT system; supporting the linking of collected inputs with generated expected outputs; forming labeled data instances; cleaning the labeled data set appropriately; sending the labeled data set to target nodes; and/or communicating with target nodes regarding improving the data processing and labeling processes, as required.
1 . An apparatus for a service supporting service capabilities through a set of Application Programming Interfaces (APIs), the service being provided as middleware between application protocols and applications, the apparatus comprising circuitry configured to:
maintain a configuration, the configuration comprising design information for a labeled data set, the labeled data set comprising a plurality of labeled data instances, wherein each labeled data instance comprises a plurality of data values relating to one or more data inputs and one or more expected data outputs associated with the one or more data inputs;
acquire a plurality of raw data inputs from data sources;
process, according to the configuration, the raw data inputs to create processed data inputs, wherein the processing of the raw data inputs comprises pre-processing based on first parameters indicated in the configuration and data transformation based on second parameters indicated in the configuration;
generate, according to the configuration, labeled data instances, wherein a labeled data instance comprises one or more processed data inputs and one or more expected data output values;
store the labeled instances in a labeled data set; and
send the labeled data set to a repository.
2 . The apparatus of claim 1 , wherein the middleware comprises a service layer defined according to ETSI/oneM2M standards.
3 . The apparatus of claim 1 , wherein, for one or more raw data inputs, the processing of the raw data inputs comprises scaling a processed data input value for each of the raw data inputs in accordance with one or more statistical observations of the plurality of raw data inputs.
4 . The apparatus of claim 1 , wherein, for one or more sets of raw data inputs, the processing of the raw data inputs comprises deriving a processed data input value for each plurality of raw data inputs in accordance with one or more statistical observations of the plurality of raw data inputs.
5 . The apparatus of claim 1 , wherein the labeled data instances are generated with data cleaning based on one or more cleaning rules indicated by the configuration.
6 . The apparatus of claim 5 , wherein the data cleaning comprises one or more of:
identifying duplicate labeled data instances in the labeled data set;
removing the identified duplicate labeled data instances from the labeled data set;
verifying data is valid from the labeled data set;
monitoring for mandatory data in the labeled data set;
detecting conflicts with data instances in the labeled data set; and
informing the repository of the identified duplicate labeled data instances.
7 . The apparatus of claim 1 , wherein:
the configuration comprises an output time requirement parameter; and
the operations further comprise acquiring an expected data output in accordance with the output time requirement parameter.
8 . The apparatus of claim 1 , wherein the pre-processing comprises one or more of:
measurement unit conversion, data type conversion, or data aggregation.
9 . The apparatus of claim 8 , wherein the data aggregation comprises one or more of: a sum, an average, a minimum, a maximum, or a count.
10 . The apparatus of claim 1 , wherein the data transformation comprises one or more of:
normalization, standardization, or binning.
11 . A method for a service supporting service capabilities through a set of Application Programming Interfaces (APIs), the service being provided as middleware between application protocols and applications, the method comprising:
maintaining a configuration, the configuration comprising design information for a labeled data set, the labeled data set comprising a plurality of labeled data instances, wherein each labeled data instance comprises a plurality of data values relating to one or more data inputs and one or more expected data outputs associated with the one or more data inputs;
acquiring a plurality of raw data inputs from data sources;
processing, according to the configuration, the raw data inputs to create processed data inputs, wherein the processing of the raw data inputs comprises pre-processing based on first parameters indicated in the configuration and data transformation based on second parameters indicated in the configuration;
generating, according to the configuration, labeled data instances, wherein a labeled data instance comprises one or more processed data inputs and one or more expected data output values;
storing the labeled data instances in a labeled data set; and
sending the labeled data set to a repository.
12 . The method of claim 11 , wherein the middleware comprises a service layer defined according to ETSI/oneM2M standards.
13 . The method of claim 11 , wherein, for one or more raw data inputs, the processing of the raw data inputs comprises scaling a processed data input value for each of the raw data inputs in accordance with one or more statistical observations of the plurality of raw data inputs.
14 . The method of claim 11 , wherein, for one or more sets of raw data inputs, the processing of the raw data inputs comprises deriving a processed data input value for each plurality of raw data inputs in accordance with one or more statistical observations of the plurality of raw data inputs.
15 . The method of claim 11 , wherein the labeled data instances are generated with data cleaning based on one or more cleaning rules indicated by the configuration.
16 . The method of claim 15 , wherein the data cleaning comprises one or more of:
identifying duplicate labeled data instances in the labeled data set;
removing the identified duplicate labeled data instances from the labeled data set;
verifying data is valid from the labeled data set;
monitoring for mandatory data in the labeled data set;
detecting conflicts with data instances in the labeled data set; and
informing the repository of the identified duplicate labeled data instances.
17 . The method of claim 11 , wherein:
the configuration comprises an output time requirement parameter; and
the operations further comprise acquiring an expected data output in accordance with the output time requirement parameter.
18 . The method of claim 11 , wherein the pre-processing comprises one or more of:
measurement unit conversion, data type conversion, or data aggregation.
19 . The method of claim 18 , wherein the data aggregation comprises one or more of: a sum, an average, a minimum, a maximum, or a count.
20 . The method of claim 11 , wherein the data transformation comprises one or more of:
normalization, standardization, or binning.