Method and system for preprocessing optimization of streaming video data using machine learning

1 . A method of improving a main output of a main AI model processing first video data using a machine learning model, the method comprising:

continuously analyzing incoming video data via a first processing pipeline that comprises:

continuously receiving incoming video data from a camera;

preprocessing the incoming video data according to first preprocessing parameters defined within a first runtime configuration file, wherein the preprocessing includes formatting the incoming video data to create the first video data;

accessing the first video data by the main AI model;

processing the first video data by the main AI model to determine the main output that is indicative of a first inference dependent upon the first video data;

continuously analyzing incoming video data via a second processing pipeline that includes the machine learning model, wherein the second processing pipeline is performed concurrently with the performance of the first processing pipeline, the second processing pipeline comprising:

continuously receiving the incoming video data from the camera;

identifying, by the machine learning model that includes a computer processor, first test preprocessing parameters based upon a database in communication with the machine learning model, the first test preprocessing parameters being predictive to produce a first test output that satisfies a baseline criterion, wherein the first test preprocessing parameters include video edits and have independent variables that include an order in which the video edits are performed on the incoming video data;

preprocessing the incoming video data according to the first test preprocessing parameters defined within a test runtime configuration file, wherein the first test preprocessing includes formatting the incoming video data to create first test video data;

accessing the first test video data by the test AI model;

processing the first test video data by the test AI model to determine the first test output that is indicative of a first test inference dependent upon the first test video data;

comparing the first test output to the baseline criterion;

in response to the first test output satisfying the baseline criterion:

altering the first preprocessing parameters in the first runtime configuration file to be similar to the first test preprocessing parameters in the test runtime configuration file; and

continuing to analyze the incoming video data via the altered first preprocessing parameters; and

in response to the first test output failing to satisfy the baseline criterion, identifying, by the machine learning model, second test preprocessing parameters based upon the first test preprocessing parameters and the database with the second test preprocessing parameters being predictive to produce a second test output that satisfies the baseline criterion and repeating the steps of analyzing the incoming video data via the second preprocessing pipeline.

2 . The method of claim 1 , wherein the baseline criterion is a value representative of an accuracy of the main output.

3 . The method of claim 2 , wherein the baseline criterion is satisfied when a value representative of the accuracy of the first test output is greater than the value representative of the accuracy of the main output.

4 . The method of claim 1 , wherein the baseline criterion is a value representative of correctly identified inferences by the main AI model.

5 . The method of claim 4 , wherein the baseline criterion is satisfied when a value representative of correctly identified inferences by the test AI model is greater than the value representative of correctly identified inferences by the main AI model.

6 . The method of claim 1 , wherein the baseline criterion is a value representative of a latency of the processing of the main video data by the main AI model.

7 . The method of claim 6 , wherein the baseline criterion is satisfied when a value representative of a latency of the processing of the test video data by the test AI model is less than the latency of the processing of the main video data by the main AI model.

8 . The method of claim 1 , wherein the main AI model and the test AI model are the same AI model such that accessing the first video data, processing the first video data to determine the main output, accessing the test video data, and processing the test video data to determine the test output are performed by one AI model.

9 . The method of claim 1 , further comprising:

adding the first test preprocessing parameters and the first test output to the database.

10 . The method of claim 1 , wherein the video edits and independent variables further include at least one of the following: the video edits and a value of each of the video edits.

11 . The method of claim 10 , wherein the video edits include at least one of the following: crop, grayscale, contrast, brightness, color threshold, resize, blur, hue saturation value, sharpen, erosion, dilation, Laplacian image processing, Sobel image processing, pyramid up, and pyramid down.

12 . The method of claim 10 , wherein the first test output has dependent variables that are at least one of the following: correctly identified inferences by the test AI model, accuracy of the test AI model, and latency of the test AI model.

13 . The method of claim 12 , further comprising:

adding the dependent variables and the independent variables to the database,

wherein the database includes additional independent variables and dependent variables from previous instances of analyzing the incoming video data via the second processing pipeline that correspond to the independent variables and dependent variables from the first test preprocessing parameters and the first test output, respectively.

14 . A method for training a machine learning model for improving a main output of a main AI model processing first video data, the machine learning model includes a hardware processor and machine learning training software code stored in memory with the hardware processor configured to execute the machine learning training software code, the method comprising:

continuously receiving incoming video data from a camera;

identifying, by the machine learning training software code as executed by the hardware processor, first test preprocessing parameters having video edits defined within a test runtime configuration file, the first test preprocessing parameters having independent variables that include an order in which the video edits are performed on the incoming video data;

preprocessing the incoming video data according to the first test preprocessing parameters to create first test video data;

accessing the first test video data by a test AI model;

processing the first test video data by the test AI model to determine a first test output that is indicative of a first inference dependent upon the first test video data, the first test output having dependent variables that are at least one of the following:

correctly identified inferences by the test AI model, accuracy of the test AI model, and latency of the test AI model;

adding the first test preprocessing parameters and the first test output to a database;

identifying, by the machine learning model, second test preprocessing parameters based on the first test preprocessing parameters and the first test output, the second test preprocessing parameters having different values of independent variables than first test preprocessing parameters;

preprocessing the incoming video data according to the second test preprocessing parameters to create second test video data;

processing the second test video data by the test AI model to determine a second test output that is indicative of a second inference dependent upon the second test video data, the second test output having dependent variables;

adding the second test preprocessing parameters and the second test output to the database;

continuously adjusting the test preprocessing parameters to create further test video data;

processing the further test video data by the test AI model to determine further test outputs indicative of further inferences dependent upon the further test video data, the further test outputs each having dependent variables;

adding the test preprocessing parameters and the further test outputs to the database;

continuously extracting, by the machine learning model, main preprocessing parameters having independent variables that are predictive of a change in the dependent variables based on the test preprocessing parameters and the test outputs in the database; and

continuously analyzing the incoming video data via a main processing pipeline to determine the main output of the main AI model at the same time as the continual adjustment of the test preprocessing parameters to create further test video data and the continual extraction of the main preprocessing parameters, the main processing pipeline comprising:

receiving the incoming video data;

preprocessing the incoming video data according to the main preprocessing parameters to format the incoming video data to create main video data; and

processing the main video data by the main AI model to determine the main output that is indicative of a first inference dependent upon the first main video data, the main output having the independent variables.

15 . The method of claim 14 , further comprising:

analyzing incoming video data via a second processing pipeline that includes the machine learning model, the second processing pipeline comprising:

receiving the incoming video data from the camera;

identifying, by the machine learning model, optimal test preprocessing parameters based upon the database in communication with the machine learning model, the optimal test preprocessing parameters being predictive to produce an optimal test output that satisfies a baseline criterion;

preprocessing the incoming video data according to the optimal test preprocessing parameters defined within an optimal test runtime configuration file, wherein the optimal preprocessing includes formatting the incoming video data to create optimal test video data;

accessing the optimal test video data by the test AI model;

processing the optimal test video data by the test AI model to determine the optimal test output that is indicative of an optimal test inference dependent upon the optimal test video data;

comparing the optimal test output to the baseline criterion;

in response to the optimal test output satisfying the baseline criterion, altering the main preprocessing parameters in the main runtime configuration file to be similar to the optimal test preprocessing parameters in the optimal test runtime configuration file; and

adding the optimal test preprocessing parameters and the optimal test output to the database.

16 . The method of claim 14 , wherein the video edits include at least one of the following: crop, grayscale, contrast, brightness, color threshold, resize, blur, hue saturation value, sharpen, erosion, dilation, Laplacian image processing, Sobel image processing, pyramid up, and pyramid down.

17 . The method of claim 14 , wherein the independent variables are at least one of the following: the video edits, a value of each of the video edits, and an order in which the video edits are performed on the incoming video.

18 . The method of claim 14 , wherein the dependent variables are at least one of the following: correctly identified inferences by the test AI model, accuracy of the test AI model, and latency of the test AI model.