IP Library Granted Patent US 10,970,578
Granted Patent B2
US 10,970,578 · App. 16/270,432 · Granted Apr 6, 2021

System and method for extracting information from a non-planar surface

Inventors: Liam Moore (Carrigaline, IE); Ana Vinogradova (Cork, IE); Volodymyr Korolyov (Cork, IE)
Assignee: JOHNSON CONTROLS FIRE PROTECTION LP
G06K9/325G06F40/186G06K9/228G06K9/6202G06T3/4038H04N7/181H04N7/188G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,970,578
App. No.
16/270,432
Granted
Apr 6, 2021
Kind
B2
Abstract

A system receives images capturing a non-planar surface from different respective angles, where each of the images captures a section of an area on the non-planar surface, the area including textual or graphical information. The system stitches together the images to obtain a stitched image that captures an entirety of the area on the non-planar surface. The system processes the stitched image to extract the textual or graphical information from the area on the non-planar surface. The system then associates the extracted textual or graphical information with one or more information fields in a report template, and generates a report by populating the one or more information fields of the report template with corresponding associated information within the extracted textual or graphical information.

Claims (82)

1. A method of extracting data from an image, comprising:

receiving images that capture a non-planar surface from different respective angles, wherein each of the images captures a section of an area on the non-planar surface, the area including textual or graphical information;

identifying low image quality sections corresponding to curved edges of the non-planar surface in the images;

cropping the images to remove the low image quality sections;

stitching together the images to obtain a stitched image that captures an entirety of the area on the non-planar surface;

processing the stitched image to extract the textual or graphical information from the area on the non-planar surface;

associating the textual or graphical information with one or more information fields in a report template; and

generating a report by populating the one or more information fields of the report template with corresponding associated information within the textual or graphical information.

2. The method of claim 1 , further comprising:

wherein receiving of the images includes receiving a sequence of images; and

down-sampling the sequence of images to obtain the images.

3. The method of claim 1 , wherein the stitching comprises:

detecting one or more features in the images;

matching the one or more features across at least two adjacent ones of the images; and

combining the images by aligning the one or more features to obtain the stitched image.

4. The method of claim 1 , wherein the stitching comprises one or more of:

correcting rotation mismatches across the images;

performing color normalization on the images; or

removing artifacts from the stitched image.

5. The method of claim 1 , wherein the processing of the stitched image to extract the textual or graphical information comprises:

binarizing the stitched image to obtain a black and white stitched image; and

performing optical character recognition “OCR” on the black and white stitched image to extract textual information from the area on the non-planar surface.

6. The method of claim 1 , wherein the processing of the stitched image to extract the textual or graphical information comprises:

performing object recognition or pattern recognition on the stitched image to extract graphical information from the area on the non-planar surface.

7. The method of claim 1 , wherein the non-planar surface comprises a curved surface of a cylindrical vessel, wherein the area corresponds to a rectangular label attached on the curved surface of the cylindrical vessel, wherein the report comprises an inspection report of the cylindrical vessel.

8. The method of claim 1 , wherein the receiving of the images further comprises obtaining by a camera of a portable computing device via an application executed on the portable computing device.

9. The method of claim 8 , further comprising:

determining to start the application based on a user input received through a user interface of the portable computing device; and

wherein the obtaining of the images further comprises obtaining by the application via accessing the camera in the portable computing device.

10. The method of claim 9 , further comprising:

determining to stop the application based on detecting an edge of the area on the non-planar surface.

11. The method of claim 1 , wherein the receiving of the images further comprises obtaining by security cameras installed in a corresponding area where an object comprising the non-planar surface is located.

12. The method of claim 11 , further comprising:

adjusting a position of at least one of the security cameras to achieve a field of view that includes the non-planar surface.

13. The method of claim 11 , further comprising:

obtaining a first image by one of the security cameras;

changing a setting of the one of the security cameras; and

obtaining a second image by the one of the security cameras, wherein the first image and the second image comprise different sections of the area on the non-planar surface.

14. The method of claim 11 , further comprising:

determining a trigger to obtain the images; and

initiating the security cameras to obtain the images in response to the trigger.

15. The method of claim 14 , wherein the trigger comprises a periodic inspection schedule or a user input received via a user interface of a central computing device in communication with the security cameras.

16. An apparatus comprising:

a processor; and

a memory storing instructions that when executed by the processor cause the processor to:

receive images capturing a non-planar surface from different respective angles, wherein each of the images captures a section of an area on the non-planar surface, the area including textual or graphical information;

identify low image quality sections corresponding to curved edges of the non-planar surface in each of the images;

crop each of the images to remove the low image quality sections;

stitch together the images to obtain a stitched image that captures an entirety of the area on the non-planar surface;

process the stitched image to extract the textual or graphical information from the area on the non-planar surface;

associate the textual or graphical information with one or more information fields in a report template; and

generate a report by populating the one or more information fields of the report template with corresponding associated information within the textual or graphical information.

17. A method of extracting data from an image, comprising:

receiving images that capture a non-planar surface from different respective angles, wherein each of the images captures a section of an area on the non-planar surface, the area including textual or graphical information;

identifying low image quality sections corresponding to curved edges of the non-planar surface in each of the images;

cropping each of the images to remove the low image quality sections;

stitching together the images to obtain a stitched image that captures an entirety of the area on the non-planar surface;

processing the stitched image to extract the textual or graphical information from the area on the non-planar surface; and

sending the textual or graphical information to a second device.

18. The method of claim 17 , wherein the second device comprises a cloud processing device or a central computing device.

19. A first device comprising:

a processor; and

a memory storing instructions that when executed by the processor cause the processor to:

receive images that capture a non-planar surface from different respective angles, wherein each of the images captures a section of an area on the non-planar surface, the area including textual or graphical information;

identify low image quality sections corresponding to curved edges of the non-planar surface in each of the images;

crop each of the images to remove the low image quality sections;

stitch together the images to obtain a stitched image that captures an entirety of the area on the non-planar surface;

process the stitched image to extract the textual or graphical information from the area on the non-planar surface; and

send the textual or graphical information to a second device.

20. A method of extracting data from an image, comprising:

receiving, by a first device, a stitched image from a second device, the stitched image capturing an entirety of an area on a non-planar surface, the stitched image being based on cropped images obtained by cropping images that capture the non-planar surface from different respective angles, wherein each of the images captures a section of the area on the non-planar surface, the area including textual or graphical information, wherein cropping the images crops low image quality sections corresponding to curved edges of the non-planar surface in each of the images;

processing the stitched image to extract the textual or graphical information from the area on the non-planar surface;

associating the textual or graphical information with one or more information fields in a report template; and

generating a report by populating the one or more information fields of the report template with corresponding associated information within the textual or graphical information.

21. The method of claim 20 , wherein the first device comprises a cloud processing device or a central computing device, wherein the second device comprises a portable computing device.

22. A first device comprising:

a processor; and

a memory storing instructions that when executed by the processor cause the processor to:

receive a stitched image from a second device, the stitched image capturing an entirety of an area on a non-planar surface, the stitched image being based on cropped images obtained by cropping images that capture the non-planar surface from different respective angles, wherein each of the images captures a section of the area on the non-planar surface, the area including textual or graphical information, wherein cropping the images crops low image quality sections corresponding to curved edges of the non-planar surface in each of the images;

process the stitched image to extract the textual or graphical information from the area on the non-planar surface;

associate the textual or graphical information with one or more information fields in a report template; and

generate a report by populating the one or more information fields of the report template with corresponding associated information within the textual or graphical information.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2024
From: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
To: TYCO FIRE & SECURITY GMBH
Reel/Frame 066740/0208 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2021
From: JOHNSON CONTROLS FIRE PROTECTION LP
To: JOHNSON CONTROLS US HOLDINGS LLC
Reel/Frame 058599/0339 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2021
From: JOHNSON CONTROLS US HOLDINGS LLC
To: JOHNSON CONTROLS INC
Reel/Frame 058599/0922 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2021
From: JOHNSON CONTROLS INC
To: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
Reel/Frame 058600/0047 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2019
From: MOORE, LIAM; VINOGRADOVA, ANA; KOROLYOV, VOLODYMYR
To: JOHNSON CONTROLS FIRE PROTECTION LP
Reel/Frame 050635/0579 →
Cited By (2)
US 12,509,328 US 12,670,580