System and method for vision based graphical fluid flow anomaly detection for display verification

1. A system for fluid flow verification, the system comprising:

a video recording device operative to capture video frames of an actual fluid or a fluid display; and

at least one processor operatively coupled to the video recording device, wherein the at least one processor hosts a plurality of fluid verification modules comprising:

a fluid flow rate verification module including instructions to cause the at least one processor to:

receive a plurality of video frames captured by the video recording device;

for each of the video frames, binarize a region of interest and compute therefrom a histogram of white pixels and black pixels;

count the white pixels in each of the video frames to detect an actual fluid quantity;

determine whether the actual fluid quantity matches an expected fluid quantity at a given time;

if the actual fluid quantity matches the expected fluid quantity, measure a fluid rate of change;

determine whether there is a steady fluid flow based on the fluid rate of change; and

if there is not a steady fluid flow, report a presence of a fluid flow rate anomaly; and

a fluid color verification module including instructions to cause the at least one processor to:

receive the plurality of video frames captured by the video recording device;

receive the binarized region of interest and the histogram of white pixels and black pixels, for each of the video frames, from the fluid flow rate verification module;

detect a fluid color by extracting a sample color mask of pixels from a position of the white pixels in each binarized region of interest;

determine whether the detected fluid color matches an expected fluid color at the given time; and

if the detected fluid color does not match the expected fluid color, report a presence of a fluid color anomaly.

2. The system of claim 1 , wherein the fluid verification modules further comprise:

a fluid type verification module that includes instructions to cause the at least one processor to:

receive the plurality of video frames captured by the video recording device;

for each of the video frames, binarize a region of interest and perform an edge pattern analysis to detect a fluid type;

determine whether the detected fluid type matches an expected fluid type; and

if the detected fluid type does not match the expected fluid type, report a presence of a fluid type anomaly.

3. The system of claim 1 , wherein if a direction of fluid flow or fluid fill is incorrect, the fluid flow rate verification module reports a fluid direction anomaly.

4. The system of claim 1 , wherein the video recording device is operative to capture video frames of an actual fluid flow, or a graphical fluid flow display.

5. The system of claim 1 , wherein the white pixels in the histogram represent presence of fluid, and the black pixels in the histogram represent absence of fluid.

6. The system of claim 1 , wherein the ratio of white pixels to black pixels for each frame corresponds to a relative quantity of fluid.

7. The system of claim 1 , wherein the expected fluid quantity at the given time is retrieved from a database and correlated with the actual fluid quantity using a pixel map.

8. The system of claim 1 , wherein the fluid rate of change is measured from a last video frame to a present video frame, by a detecting a fluid flow direction and a fluid flow magnitude.

9. The system of claim 8 , wherein the fluid flow magnitude is detected using a time series prediction module operative to:

receive a fluid quantity value from previous frames;

calculate and output a model predicted value for a fluid quantity at a current frame;

compare the model predicted value with an actual fluid quantity at the current frame; and

if the actual fluid quantity is deviated from the model predicted value by more than a threshold, an anomaly is detected for the current frame.

10. A method for fluid flow verification, the method comprising:

providing a fluid flow rate verification module including instructions to cause at least one processor to perform a method comprising:

receiving a plurality of video frames showing an actual fluid or a fluid display;

binarizing a region of interest for each of the video frames;

computing a histogram of white pixels and black pixels for each binarized region of interest;

counting the white pixels in each histogram to detect an actual fluid quantity;

retrieving an expected fluid quantity at a given time from a database;

determining whether the actual fluid quantity matches an expected fluid quantity at a given time;

if the actual fluid quantity matches the expected fluid quantity, measuring a fluid rate of change;

determining whether there is a steady fluid flow based on the fluid rate of change; and

if there is not a steady fluid flow, reporting a fluid flow rate anomaly.

11. The method of claim 10 , further comprising:

providing a fluid color verification module including instructions to cause the at least one processor to perform a method comprising:

receiving the plurality of video frames showing the actual fluid or the fluid display;

receiving each binarized region of interest and the histogram of white pixels and black pixels, from the fluid flow rate verification module;

detecting a fluid color by extracting a sample color mask of pixels from a position of the white pixels in each binarized region of interest;

retrieving an expected fluid color at a given time from a database;

determining whether the detected fluid color matches the expected fluid color at the given time; and

if the detected fluid color does not match the expected fluid color, reporting a fluid color anomaly.

12. The method of claim 10 , further comprising:

providing a fluid type verification module that includes instructions to cause at least one processor to perform a method comprising:

obtaining a plurality of video frames of a fluid;

inputting the video frames into a trained classifier model operative to generate a detected fluid type of the fluid based on the video frames;

outputting the detected fluid type from the trained classifier model;

retrieving an expected fluid type from a database;

determining whether the detected fluid type matches the expected fluid type; and

if the detected fluid type does not match the expected fluid type, reporting a fluid type anomaly.

13. The method of claim 12 , wherein the trained classifier model is produced by a process comprising:

obtaining an image set for each of a plurality of different chemical classes; and

inputting each image set into a model neural network operative to process each image set with machine learning to generate the trained classifier model.

14. The method of claim 13 , wherein one or more of the image sets show images of chemicals under reaction or perturbation.

15. The method of claim 13 , wherein the trained classifier model is operative to classify a chemical under test at each video frame.

16. The method of claim 10 , wherein the video frames depict an actual fluid flow, or a graphical fluid flow display.

17. The method of claim 10 , wherein the white pixels in the histogram represent presence of fluid, and the black pixels in the histogram represent absence of fluid.

18. The method of claim 17 , wherein the ratio of white pixels to black pixels for each frame corresponds to a relative quantity of fluid.

19. The method of claim 10 , wherein the expected fluid quantity at the given time is retrieved from a database and correlated with the actual fluid quantity using a pixel map.

20. The method of claim 10 , wherein the fluid rate of change is measured from a last video frame to a present video frame, by a detecting a fluid flow direction and a fluid flow magnitude.