IP Library › Granted Patent US 10,664,567
Granted Patent B2
US 10,664,567 · App. 15/109,700 · Granted May 26, 2020

Extraction of information from an image and inclusion thereof in a clinical report

Inventors: Yinhui Deng (Shanghai, CN); Qi Zhong Lin (Shanghai, CN); Lilla Boroczky (Mount Kisco, NY); Michael Chun-Chieh Lee (Lexington, MA); Ying Wu (Shanghai, CN)
Assignee: Koninklijke Philips N.V.
G06F19/321G06F17/241G06K9/00449G06K9/00456G06K9/2063G06K9/3266G06K9/4604G16H15/00G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,664,567
App. No.
15/109,700
Granted
May 26, 2020
Kind
B2
Abstract

A method includes obtaining, in electronic format, an image ( 102 ) including a medical image display region ( 104 ) and an information display region ( 106 ). The at least one of the medical image display region or the information display region includes graphical indicia representing at least one of an annotation ( 110, 112, 114, 116 ) or alphanumeric information ( 118, 120, 122, 124, 126, 128, 130, 132, 134, 136, 138, 140 ). The method further includes evaluating pixels of the image to identify pixels including the graphical indicia representing an annotation or alphanumeric information of interest in the image. The method further includes extracting the annotation or alphanumeric information of interest from the identified graphical indicia from the image. The method further includes inserting the extracted annotation or alphanumeric information of interest in an electronically formatted clinical report for the image.

Claims (33)

1. A method, comprising:

obtaining, in electronic format, an image including a medical image display region and an information display region, wherein at least one of the medical image display region or the information display region includes an alphanumeric graphical indicia representing an alphanumeric information, and the medical image display region includes an annotation graphical indicia representing a non-textual annotation within the medical image display region;

evaluating pixels of the image to identify a first group of pixels including the alphanumeric graphical indicia representing the alphanumeric information of interest in the image and to identify a second group of pixels including the annotation graphical indicia representing the non-textual annotation of interest in the image;

determining first and second extraction groups of pixels based on first and second predetermined extraction regions, wherein the first extraction group of pixels includes the pixels identified as including the alphanumeric graphical indicia and at least one of: a predetermined number of pixels preceding the pixels identified as including the alphanumeric graphical indicia and a predetermined number of pixels succeeding the pixels identified as including the alphanumeric graphical indicia, and wherein the second extraction group of pixels includes the pixels identified as including the annotation graphical indicia and at least one of: a predetermined number of pixels preceding the pixels identified as including the annotation graphical indicia and a predetermined number of pixels succeeding the pixels identified as including the annotation graphical indicia;

extracting the first extraction group of pixels to extract the alphanumeric information, and extracting, without modifying the medical image display region, the second extraction group of pixels to extract the non-textual annotation; and

inserting the extracted alphanumeric information and extracted non-textual annotation in an electronically formatted clinical report corresponding to the image, wherein inserting the extracted alphanumeric information and extracted non-textual annotation comprises inserting the alphanumeric information into a first predetermined location within the electronically formatted clinical report corresponding to the image, and inserting the extracted non-textual annotation into a second predetermined location within the electronically formatted clinical report corresponding to the image, the first and second predetermined locations corresponding to the image and defined by a set of predetermined report rules.

2. The method of claim 1 , further comprising:

determining a group of search pixels based on a predetermined search region; and

evaluating the group of search pixels to identify the pixels including the graphical indicia.

3. The method of claim 1 , further comprising:

searching the entire image for the graphical indicia.

4. The method of claim 1 , further comprising:

determining a search region of the image to search, wherein the search region is a sub-region of the image; and

searching only the search region of the image for the graphical indicia.

5. The method of claim 4 , wherein the sub-region includes one of only the medical image display region or only the information display region.

6. The method of claim 1 , further comprising:

employing character recognition to identify the pixels including the graphical indicia.

7. The method of claim 1 , wherein the identified pixels of the non-textual annotation graphical indicia represent at least one of a numerical distance measurement between at least two user-identified locations in the image, a measurement unit of the measurement, or a measurement type of the measurement.

8. The method of claim 1 , wherein the predetermined location corresponds to a user-identified location.

9. The method of claim 1 , wherein the extraction group of pixels includes only the pixels identified as including indicia.

10. A computing system, comprising:

a non-transitory computer readable storage medium with instructions including an information extraction module and a report generating/editing module; and a processor, which, in response to executing the instructions:

obtains, in electronic format, an image including a medical image display region and an information display region, wherein at least one of the medical image display region and the information display region includes an alphanumeric graphical indicia representing an alphanumeric information, and the medical image display region includes an annotation graphical indicia representing a non-textual annotation within the medical image display region

evaluates pixels of the image to identify a first group of pixels including the alphanumeric graphical indicia, and to identify a second group of pixels including the annotation graphical indicia representing the non-textual annotation of interest in the image;

determines first and second extraction groups of pixels based on first and second predetermined extraction regions, wherein the first extraction group of pixels includes the pixels identified as including the alphanumeric graphical indicia and at least one of: a predetermined number of pixels preceding the pixels identified as including the alphanumeric graphical indicia or succeeding the pixels identified as including the alphanumeric graphical indicia, and wherein the second extraction group of pixels includes the pixels identified as including the annotation graphical indicia and at least one of: a predetermined number of pixels preceding the pixels identified as including the annotation graphical indicia and a predetermined number of pixels succeeding the pixels identified as including the annotation graphical indicia; and

extracts the first extraction group of pixels to extract the alphanumeric information, and extracts, without modifying the medical image display region, the second extraction group of pixels to extract the non-textual annotation; and

inserts the extracted alphanumeric information and extracted non-textual annotation in an electronically formatted clinical report corresponding to the image, wherein inserting the extracted alphanumeric information and extracted non-textual annotation comprises inserting the alphanumeric information into a first predetermined location within the electronic clinical report corresponding to the image, and inserts the extracted non-textual annotation into a second predetermined location within the electronically formatted clinical report corresponding to the image, the first and second predetermined locations defined by a set of predetermined report rules.

11. A non-transitory computer readable storage medium encoded with computer readable instructions, which, when executed by a processor, cause the processor to:

obtain, in electronic format, an image including a medical image display region and an information display region, wherein at least one of the medical image display region and the information display region includes an alphanumeric graphical indicia representing alphanumeric information, and the medical image display region includes an annotation graphical indicia representing a non-textual annotation within the medical image display region;

evaluate pixels of the image to identify a first group of pixels including the alphanumeric graphical indicia representing the alphanumeric information of interest in the image and to identify a second group of pixels including the annotation graphical indicia representing the non-textual annotation of interest in the image;

determine first and second extraction groups of pixels based on first and second predetermined extraction regions, wherein the first extraction group of pixels includes the pixels identified as including the alphanumeric graphical indicia and at least one of: a predetermined number of pixels preceding the pixels identified as including the alphanumeric graphical indicia and a predetermined number of pixels succeeding the pixels identified as including the alphanumeric graphical indicia, and wherein the second extraction group of pixels includes the pixels identified as including the annotation graphical indicia and at least one of: a predetermined number of pixels preceding the pixels identified as including the annotation graphical indicia and a predetermined number of pixels succeeding the pixels identified as including the annotation graphical indicia;

extract the first extraction group of pixels to extract the alphanumeric information in response to identifying the graphical indicia, and extract, without modifying the medical image display region, the second extraction group of pixels to extract the non-textual annotation; and

insert the extracted alphanumeric information and extracted non-textual annotation in an electronic clinical report corresponding to the image, wherein inserting the extracted alphanumeric information and extracted non-textual annotation comprises inserting the alphanumeric information into a predetermined location within the electronic clinical report corresponding to the image, and comprises inserting the extracted non-textual annotation into a second predetermined location within the electronically formatted clinical report corresponding to the image, the first and second predetermined locations defined by a set of predetermined report rules.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 5, 2016
From: DENG, YINHUI; LEE, MICHAEL CHUN-CHIEH; LIN, QI ZHONG; WU, YING
To: KONINKLIJKE PHILIPS N.V.
Reel/Frame 039073/0631 →
Continuity (2)
Provisional Application 61931735 · Jan 27, 2014
Related Publication 20160328517A1 · Nov 10, 2016