IP Library Granted Patent US 11,875,480
Granted Patent B1
US 11,875,480 · App. 17/321,134 · Granted Jan 16, 2024

Distinguishing artifacts from pathological features in digital images

Inventors: Sam Kavusi (Menlo Park, CA); Sunny Virmani (Fremont, CA); Eliezer Glik (San Francisco, CA); Kira Whitehouse (Palo Alto, CA)
Assignee: Verily Life Sciences LLC
G06T5/002A61B3/0008A61B3/0041A61B3/12A61B3/14G06T5/50G06T7/11G06T7/80G06T2207/10024G06T2207/20081G06T2207/20182G06T2207/30041
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,875,480
App. No.
17/321,134
Granted
Jan 16, 2024
Kind
B1
Abstract

Introduced here are approaches to assessing whether digital features (or simply “features”) detected in digital images by detection models are representative of artifacts that can obscure actual pathologies. A diagnostic platform may characterize each digital feature detected in a digital image based on its likelihood of being an artifact. For instance, a digital feature could be characterized as being representative of an artifact caused by improper illumination, an artifact caused by a physical element that is adhered to the lens through which light is collected by an imaging device, or a pathological feature indicative of a disease.

Claims (64)

1. A method comprising:

acquiring, by a processor, a digital image generated by a retinal camera during a calibration session that occurs outside of a diagnostic session involving a patient,

wherein the digital image does not include the patient who is to be imaged during the diagnostic session, and therefore is based on light reflected by an ambient environment through a lens of the retinal camera;

applying, by the processor, a machine learning model to the digital image to produce an output that identifies a digital feature in the digital image that is representative of an artifact with no pathological value,

wherein the digital feature corresponds to a segmented region of pixels in the digital image;

classifying, by the processor, the segmented region of pixels corresponding to the digital feature as being representative of (i) an artifact that is caused by an optical aberration or (ii) an artifact that is caused by a physical element adhered to the lens of the retinal camera; and

performing, by the processor, a remediation action based on an outcome of said classifying.

2. The method of claim 1 , wherein the digital image is generated by the retinal camera in conjunction with infrared radiation emitted by the retinal camera.

3. The method of claim 1 , wherein the digital image is one of a series of digital images generated by the retinal camera in conjunction with light emitted by a series of light sources, and wherein each digital image in the series of digital images is generated in conjunction with light emitted by a corresponding light source of the series of light sources.

4. The method of claim 1 , further comprising:

storing, by the processor, data regarding the digital feature in a structure formatted in accordance with a medical image standard,

wherein the data specifies a location and a classification of the segmented region of pixels.

5. The method of claim 1 , wherein said performing the remediation action comprises:

causing display of a notification that identifies the segmented region of contiguous pixels determined to represent an artifact.

6. The method of claim 5 , wherein said performing the remediation action comprises:

enabling an individual to identify a filter to be applied to the digital image so as to remove the artifact.

7. The method of claim 5 , wherein the notification is posted to

an interface that is accessible to a medical professional responsible for analyzing digital images generated during the diagnostic session, or

an interface that is accessible to an operator responsible for operating the retinal camera during the diagnostic session.

8. The method of claim 1 , further comprising:

determining, by the processor, that the digital feature is representative of an artifact caused by a physical element that is adhered to a lens through which light is collected by the retinal camera; and

wherein said performing the remediation action comprises:

transmitting an instruction to a cleaning system that cleans the lens of the retinal camera responsive to receiving the instruction.

9. The method of claim 1 , wherein the digital feature is one of multiple digital features identified in the digital image by the machine learning model.

10. A non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:

acquiring a series of digital images generated by a retinal camera in rapid succession;

applying a detection model to the series of digital images to produce a series of outputs,

wherein each output in the series of outputs corresponds to a digital image in the series of digital images, and

wherein each output identifies digital features in the corresponding digital image that may be representative of artifacts;

comparing the series of outputs produced by the detection model such that each digital feature is classified as being representative of either an artifact or a pathological feature; and

generating a notification that specifies whether any digital features have been classified as being representative of artifacts, wherein

the notification includes

an instruction to clean a lens of the retinal camera to remove a physical element adhered to the lens of the retinal camera, or

an instruction to employ a filter to mitigate an effect of an optical aberration from the image.

11. The non-transitory computer-readable medium of claim 10 , wherein each digital feature corresponds to a segmented region of contiguous pixels.

12. The non-transitory computer-readable medium of claim 11 , wherein said comparing comprises:

estimating a degree of consistency between segmented regions of pixels across different digital images.

13. The non-transitory computer-readable medium of claim 12 , wherein said estimating is performed by a machine learning model that considers differences in color, shadow, or contrast between the segmented regions of pixels.

14. The non-transitory computer-readable medium of claim 12 , wherein said estimating comprises:

for each segmented region of pixels,

defining a positional arrangement with respect to a known physiological feature, and

determining a count of digital images in the series of digital images that include a segmented region of pixels with a similar positional arrangement with respect to the known physiological feature.

15. The non-transitory computer-readable medium of claim 14 , wherein the degree of consistency is based on the count of digital images, and wherein a higher degree of consistency corresponds to a lower likelihood that a given segmented region of pixels is representative of an artifact.

16. The non-transitory computer-readable medium of claim 14 , wherein the degree of consistency is based on the count of digital images, and wherein a higher degree of consistency corresponds to a higher likelihood that a given segmented region of pixels is representative of an artifact.

17. The non-transitory computer-readable medium of claim 11 , wherein said comparing comprises:

for each segmented region of pixels,

defining a positional arrangement with respect to a pixel boundary, and

determining a count of digital images in the series of digital images that include a segmented region of pixels with a similar positional arrangement with respect to the pixel boundary.

18. The non-transitory computer-readable medium of claim 10 , wherein the notification includes a visualization component in which each segmented region of pixels that has been classified as being representative of an artifact is visually highlighted.

19. The non-transitory computer-readable medium of claim 10 , wherein the notification is viewable on a display of the retinal camera.

20. A computer-implemented method comprising:

acquiring first pixel data associated with a first digital image of a fundus generated by a retinal camera;

acquiring second pixel data associated with a second digital image of the fundus generated by the retinal camera;

applying a detection model to the first pixel data to identify a first digital feature in the first digital image that is unexpected given a known physiology of the fundus,

wherein the detection model is trained on images labeled as being representative of different types of artifacts, the different types of artifacts including artifacts caused by optical aberrations and artifacts caused by physical elements;

applying the detection model to the second pixel data to identify a second digital feature in the second digital image that is unexpected given the known physiology of the fundus;

classifying the first and second digital features identified in the first and second digital images as being representative of either physiological features or artifacts;

determining, based on said classifying, that a digital feature was classified as an artifact, the digital feature being the first digital feature or the second digital feature; and

storing an indication of the digital feature in a data structure.

21. The computer-implemented method of claim 20 , wherein the data structure is indicative of a profile that is associated with the retinal camera.

22. The computer-implemented method of claim 20 , wherein the first and second digital images are generated by the retinal camera during a diagnostic session in which the fundus is imaged.

23. The computer-implemented method of claim 20 , further comprising:

determining, based on said classifying, that another digital feature was classified as an artifact, the other digital feature being whichever of the first and second digital features is not the digital feature; and

storing an indication of the other digital feature in another data structure.

Assignments (2)
CHANGE OF NAME Recorded Apr 21, 2026
From: VERILY LIFE SCIENCES LLC
To: VERILY HEALTH INC.
Reel/Frame 075477/0981 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2021
From: KAVUSI, SAM; VIRMANI, SUNNY; GLIK, ELIEZER; WHITEHOUSE, KIRA
To: VERILY LIFE SCIENCES LLC
Reel/Frame 056257/0245 →
Continuity (1)
Provisional Application 63025802 · May 15, 2020
Cited By (1)
US 12,390,107