Automatic dataset creation using software tags
Traditionally, a software application is developed, tested, and then published for use by end users. Any subsequent update made to the software application is generally in the form of a human programmed modification made to the code in the software application itself, and further only becomes usable once tested, published, and installed by end users having the previous version of the software application. This typical software application lifecycle causes delays in not only generating improvements to software applications, but also to those improvements being made accessible to end users. To help avoid these delays and improve performance of software applications, deep learning models may be made accessible to the software applications for use in providing inferenced data to the software applications, which the software applications may then use as desired. These deep learning models can furthermore be improved independently of the software applications using manual and/or automated processes.
1. A method, comprising:
causing an application to generate metadata to indicate one or more locations, in memory of the application, of neural network training data generated by the application.
2. The method of claim 1 , wherein one or more identifiers correlated with the metadata are defined in a dataset definition file defined in accordance with a neural network using the neural network training data.
3. The method of claim 2 , wherein the one or more identifiers are specified as tags.
4. The method of claim 2 , further comprising using the metadata to retrieve the neural network training data from the application by at least:
matching an identifier of the one or more identifiers to metadata associated with certain data of the application; and
retrieving the certain data of the application.
5. The method of claim 1 , wherein the metadata is associated with the neural network training data by being inserted in a portion of code of the application that defines the one or more locations, in the memory of the application, in which the neural network training data is to be stored.
6. The method of claim 5 , wherein the one or more locations, in the memory of the application, include at least one of: a data structure; or a buffer.
7. The method of claim 1 , further comprising creating a dataset from the neural network training data by at least saving the neural network training data in a new dataset.
8. The method of claim 7 , further comprising:
storing the new dataset locally.
9. The method of claim 7 , further comprising:
causing the new dataset to be stored remotely at a server; and
wherein the server uses the new dataset to perform at least one of:
retraining a neural network trained using the neural network training data;
or
training another neural network.
10. The method of claim 1 , wherein a dataset collector that interfaces the application retrieves the neural network training data.
11. The method of claim 10 , wherein the dataset collector, the application, and a neural network to use the neural network training data are instantiated on a client system.
12. The method of claim 1 , further comprising:
receiving an improved version of a neural network resulting from training of the neural network using the neural network training data.
13. The method of claim 12 , further comprising:
providing at least a portion of the neural network training data as input to the improved version of the neural network, wherein the improved version of the neural network processes the input to generate inferenced data; and
receiving the inferenced data as output of the neural network, wherein the inferenced data is provided to the application.
14. The method of claim 1 , wherein a neural network trained using the neural network training data is executable to perform inferencing operations that provide inferenced data to the application.
15. The method of claim 1 , wherein the neural network training data is data processed using the application.
16. The method of claim 1 , wherein the neural network training data is a set of configuration or calibration settings of an instantiation of the application.
17. The method of claim 1 , further comprising:
storing the metadata in association with the neural network training data.
18. A system, comprising:
a memory storing instructions; and
one or more processors that execute the instructions to cause an application to generate metadata to indicate one or more locations, in memory of the application, of neural network training data generated by the application.
19. The system of claim 18 , wherein the memory further stores the metadata and associated data of the application.
20. A processor, comprising: one or more circuits to cause an application to generate metadata to indicate one or more locations, in memory of the application, of neural network training data generated by the application.