Real-time deployment of machine learning systems

1. A system comprising one or more processors, and a non-transitory computer-readable medium including one or more sequences of instructions that, when executed by the one or more processors, cause the system to perform operations comprising:

receiving multiple video frames for video data obtained from an image capture device, wherein the video frames depict imagery of an internal body cavity of a person;

rendering a first display output depicting the imagery of the multiple video frames;

performing an inference on the multiple video frames using a machine learning network;

determining the occurrence of one or more objects in the multiple video frames based on the performed inference on the multiple video frames;

in response to determining the occurrence of one or more objects, generating for a determined object, coordinates describing a bounding perimeter about the determined object;

rendering a second display output depicting graphical information in a form corresponding to the coordinates of the bounding perimeter for the determined one or more objects from the multiple video frames; and

generating a composite display output, wherein the composite display output includes the imagery of the first display output overlaid with the graphical information of the second display output.

2. The system of claim 1 , wherein the multiple video frames depict imagery obtained during an endoscopy procedure of the person, and the determined one or more objects are polyps or abnormalities.

3. The system of claim 2 , wherein the machine learning network has been trained to determine the probability that a polyp or an abnormality exists in a digital image.

4. The system of claim 1 , further comprising the operations of:

evaluating the multiple video frames to determine whether a colon is clean enough to examine for polyps or abnormalities; and

performing the inference of the multiple video frames if the colon is determined to be clean enough to examine for polyps or abnormalities.

5. The system of claim 4 , wherein evaluating the multiple video frames comprises:

evaluating the multiple video frames by determining a quality score of an image depicted in a video frame; and

determining that the colon is clean enough if the quality score meets a pre-determined quality value.

6. The system of claim 1 , further comprising the operations of:

determining the start of an endoscopy procedure where a change is detected in an image characteristic of the multiple video frames;

starting recording of the multiple video frames if the change in the image characteristic is detected; and

generating a video file from the recorded multiple video frames.

7. The system of claim 1 , further comprising the operations of:

storing the multiple video frames in a memory cache of a video capture card; and

obtaining the multiple video frames directly from the memory cache of the video capture card to perform the inference of the multiple video frames.

8. The system of claim 1 , further comprising the operations of:

determining the type or class of the one or more objects, the one or more objects being a polyp.

9. A method implemented by a system comprising of one or more processors, the method comprising:

receiving multiple video frames for video data obtained from an image capture device, wherein the video frames depict imagery of an internal body cavity of a person;

rendering a first display output depicting the imagery of the multiple video frames;

performing an inference on the multiple video frames using a machine learning network;

determining the occurrence of one or more objects in the multiple video frames based on the performed inference on the multiple video frames;

in response to determining the occurrence of one or more objects, generating for a determined object, coordinates describing a bounding perimeter about the determined object;

rendering a second display output depicting graphical information in a form corresponding to the coordinates of the bounding perimeter for the determined one or more objects from the multiple video frames; and

generating a composite display output, wherein the composite display output includes the imagery of the first display output overlaid with the graphical information of the second display output.

10. The method of claim 9 , wherein the multiple video frames depict imagery obtained during an endoscopy procedure of the person, and the determined one or more objects are polyps or abnormalities.

11. The method of claim 10 , wherein the machine learning network has been trained to determine the probability that a polyp or an or abnormality exists in a digital image.

12. The method of claim 9 , further comprising the operations of:

evaluating the multiple video frames to determine whether a colon is clean enough to examine for polyps or abnormalities; and

performing the inference of the multiple video frames if the colon is determined to be clean enough to examine for polyps.

13. The method of claim 12 , wherein evaluating the multiple video frames comprises:

evaluating the multiple video frames by determining a quality score of an image depicted in a video frame; and

determining that the colon is clean enough if the quality score meets a pre-determined quality value.

14. The method of claim 9 , further comprising the operations of:

determining the start of an endoscopy procedure where a change is detected in an image characteristic of the multiple video frames;

starting recording of the multiple video frames if the change in the image characteristic is detected; and

generating a video file from the recorded multiple video frames.

15. The method of claim 9 , further comprising the operations of:

storing the multiple video frames in a memory cache of a video capture card; and

obtaining the multiple video frames directly from the memory cache of the video capture card to perform the inference of the multiple video frames.

16. The system of claim 1 , further comprising the operations of:

determining the type or class of the one or more objects, the one or more objects being a polyp.

17. A non-transitory computer storage medium comprising instructions that when executed by a system comprising one or more processors, cause the one or more processors to perform operations comprising:

receiving multiple video frames for video data obtained from an image capture device, wherein the video frames depict imagery of an internal body cavity of a person;

rendering a first display output depicting the imagery of the multiple video frames;

performing an inference on the multiple video frames using a machine learning network;

determining the occurrence of one or more objects in the multiple video frames based on the performed inference on the multiple video frames;

in response to determining the occurrence of one or more objects, generating for a determined object, coordinates describing a bounding perimeter about the determined object;

rendering a second display output depicting graphical information in a form corresponding to the coordinates of the bounding perimeter for the determined one or more objects from the multiple video frames; and

generating a composite display output, wherein the composite display output includes the imagery of the first display output overlaid with the graphical information of the second display output.

18. The non-transitory computer storage medium of claim 17 , wherein the multiple video frames depict imagery obtained during an endoscopy procedure of the person, and the determined one or more objects are polyps or abnormalities.

19. The non-transitory computer storage medium of claim 18 , wherein the machine learning network has been trained to determine the probability that a polyp or an abnormality exists in a digital image.

20. The non-transitory computer storage medium of claim 17 , further comprising the operations of:

evaluating the multiple video frames to determine whether a colon is clean enough to examine for polyps; and

performing the inference of the multiple video frames if the colon is determined to be clean enough to examine for polyps or abnormalities.

21. The non-transitory computer storage medium of claim 20 , wherein evaluating the multiple video frames comprises:

evaluating the multiple video frames by determining a quality score of an image depicted in a video frame; and

determining that the colon is clean enough if the quality score meets a pre-determined quality value.

22. The non-transitory computer storage medium of claim 18 , further comprising the operations of:

determining the start of an endoscopy procedure where a change is detected in an image characteristic of the multiple video frames;

starting recording of the multiple video frames if the change in the image characteristic is detected; and

generating a video file from the recorded multiple video frames.

23. The non-transitory computer storage medium of claim 17 , further comprising the operations of:

storing the multiple video frames in a memory cache of a video capture card; and

obtaining the multiple video frames directly from the memory cache of the video capture card to perform the inference of the multiple video frames.

24. The non-transitory computer storage medium of claim 17 , further comprising the operations of:

determining the type or class of the one or more objects, the one or more objects being a polyp.