IP Library › Granted Patent US 11,373,388
Granted Patent B2
US 11,373,388 · App. 16/040,344 · Granted Jun 28, 2022

Persistent feature based image rotation and candidate region of interest

Inventor: Ryan J. Simpson (Vienna, VA)
Assignee: United States Postal Service
G06V10/25B07C3/20G06K7/1404G06V10/242G06V30/1478G06V30/40G06V30/413G06V30/424
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,373,388
App. No.
16/040,344
Granted
Jun 28, 2022
Kind
B2
Abstract

Embodiments of a system and method for sorting and delivering articles in a processing facility based on image data are described. Image processing results such as rotation notation information may be included in or with an image to facilitate downstream processing such as when the routing information cannot be extracted from the image using an unattended system and the image is passed to an attended image processing system. The rotation notation information may be used to dynamically adjust the image before presenting the image via the attended image processing system.

Claims (170)

1. A computer-implemented method comprising:

under control of one or more processors,

obtaining an image of a mail piece on a conveyor, the image showing address information affixed thereto;

receiving, from an optical character recognition (OCR) system, an OCR result comprising (i) an electronic text record of characters of the address information shown in the image, and (ii) rotation information for the image including a rotation angle, the rotation information identifying a number of degrees the image should be rotated to show characters of the address information from left to right with the characters in a standardized orientation;

determining that the electronic text record does not include a set of characters of the address information;

in response to determining that the electronic text record does not include the set of characters of the address information, adding the rotation angle to an image file of the image of the mail piece to generate an enhanced image;

transmitting the enhanced image for additional processing in a computing device; and

rotating, at the computing device, the enhanced image by the rotation angle prior to additional processing by the computing device,

wherein the rotation angle is dynamically generated based on a relationship between a maximum pixel density and an average pixel density for a row or a column, and

wherein the relationship is expressed as Equation (1) below:

rotation

⁢

=

30

*

density

w

-

avg

-

density

w

-

max

density

w

-

avg

Equation

⁢

⁢

(

1

)

where density w-max is the maximum density for a row of the image; and

density w-avg is the average density for the rows of the image.

2. The computer-implemented method of claim 1 , further comprising:

generating a grey-scale image from the image; and

providing the grey-scale image to the OCR system.

3. The computer-implemented method of claim 1 , wherein the address information is printed on a label affixed to the mail piece, and wherein the method further comprises:

identifying a type for the label based on an identifier for the label shown in the image;

receiving a layout template for the type, the layout template identifying a region for the address information; and

identifying the rotation information based at least in part on an orientation of the region shown in the image and an orientation of the characters shown within the region in the image.

4. The computer-implemented method of claim 3 , wherein the identifier for the type comprises at least one of a graphic or a scannable code identifying the type for the label.

5. The computer-implemented method of claim 1 , wherein determining that the electronic text record does not include the set of characters comprises evaluating the set of characters using a regular expression, and wherein a result of the evaluating indicates that the electronic text record does not include the set of characters.

6. The computer-implemented method of claim 1 , further comprising:

identifying a processing orientation for the mail piece based at least in part on a type for the mail piece; and

causing activation of the conveyor to rotate the mail piece from an orientation shown in the image to the processing orientation, wherein the activation is based at least in part on a difference between the processing orientation and the rotation information.

7. The computer-implemented method of claim 1 , wherein the image file comprises a rotation notation tagged field including the rotation angle.

8. The computer-implemented method of claim 1 , wherein the additional processing includes causing display of the enhanced image on a display device after the image is rotated by the rotation angle.

9. The computer-implemented method of claim 1 , wherein the rotation angle is adjusted in proportion to the difference between the average pixel density and the maximum pixel density for the row or the column.

10. A computer-implemented method comprising:

under control of one or more processors,

receiving an image showing an item and address information affixed thereto;

generating an electronic text record of the address information based on an optical character recognition (OCR) processing of at least a portion of the image showing the address information;

detecting orientation information for the image based on pixel information included in the image, the orientation information including a rotation angle;

determining that additional processing of the image is needed to identify routing information for the item based at least in part on the electronic text record;

in response to determining that the additional processing of the image is needed, adding the rotation angle to an image file of the image to generate an enhanced image;

transmitting the enhanced image for additional processing in a computing device; and

rotating, at the computing device, the enhanced image by the rotation angle prior to the additional processing by the computing device,

wherein the rotation angle is dynamically generated based on a relationship between a maximum pixel density and an average pixel density for a row or a column and

wherein the relationship is expressed as Equation (1) below:

rotation

⁢

=

30

*

density

w

-

avg

-

density

w

-

max

density

w

-

avg

Equation

⁢

⁢

(

1

)

where density w-max is the maximum density for a row of the image; and

density w-avg is the average density for the rows of the image.

11. The computer-implemented method of claim 10 , further comprising:

generating a grey-scale image from the image;

providing the grey-scale image to an OCR system; and

receiving the electronic text record from the OCR system.

12. The computer-implemented method of claim 10 , wherein the address information is printed on a label affixed to the item, and wherein the method further comprises:

identifying a type for the label based on an identifier for the label shown in the image;

receiving a layout template for the type, the layout template identifying a region for the address information; and

identifying the orientation information based at least in part on an orientation of the region shown in the image and an orientation of the characters shown within the region in the image.

13. The computer-implemented method of claim 12 , wherein the identifier for the type comprises at least one of a graphic or a scannable code identifying the type for the label.

14. The computer-implemented method of claim 10 , wherein determining that the electronic text record does not include the set of characters comprises evaluating the set of characters using a regular expression, and wherein a result of the evaluating indicates that the electronic text record does not include the set of characters.

15. The computer-implemented method of claim 10 , further comprising:

identifying a processing orientation for the item based at least in part on a type for the item; and

causing activation of item processing equipment to rotate the item from an orientation shown in the image to the processing orientation, wherein the activation is based at least in part on a difference between the processing orientation and the orientation information.

16. The computer-implemented method of claim 10 , wherein the image file comprises a rotation notation tagged field including the rotation angle.

17. The computer-implemented method of claim 10 , wherein the additional processing includes causing display of the enhanced image on a display device after the image is rotated by the rotation angle.

18. The computer-implemented method of claim 10 , further comprising:

generating a first binary image based on the image;

generating a second binary image based on the image and a second rotation angle, wherein the item is shown at a different orientation in the second binary image than the item is shown in the first binary image;

generating a pixel density difference metric for individual rows of pixels included in the first binary image;

generating a pixel density difference metric for individual columns of pixels included in the first binary image;

identifying a first row of the first binary image associated with a first pixel density difference metric, wherein the first pixel density difference metric for the first row indicates a level of density higher than other rows of the first binary image;

identifying a first column of the first binary image associated with a second pixel density difference metric, wherein the second pixel density difference metric for the first column indicates a level of density higher than other columns of the first binary image;

generating a pixel density difference metric for individual rows of pixels included in the second binary image;

generating a pixel density difference metric for individual columns of pixels included in the second binary image;

identifying a second row of the second binary image associated with a third pixel density difference metric, wherein the third pixel density difference metric for the second row indicates a level of density higher than: (a) other rows of the second binary image, and (b) the first pixel density difference metric; and

identifying a second column of the second binary image associated with a fourth pixel density difference metric, wherein the fourth pixel density difference metric for the second column indicates a level of density higher than: (a) other columns of the second binary image, and (b) the second pixel density difference metric.

19. An item processing system comprising:

a scanning device configured to acquire an image of an item at a first processing device;

an unattended image processing server configured to:

generate an electronic text record of address information shown in the image based on an optical character recognition (OCR) processing of at least a portion of the image, the OCR processing configured to detect orientation information of the image, the orientation information including a rotation angle

determine that additional processing of the image is needed to identify routing information for the item based at least in part on the electronic text record,

in response to determining that the additional processing of the image is needed, add the rotation angle to an image file of the image to generate an enhanced image, and

transmit the enhanced image to an attended image processing workstation; and the attended image processing workstation configured to:

receive the enhanced image,

identify a display orientation for the enhanced image,

rotate the enhanced image based on a difference between the display orientation and the rotation angle,

display the enhanced image as adjusted on a display device,

receive an address information input for the enhanced image, and

transmit the address information input to the unattended image processing server,

wherein the unattended image processing server is further configured to transmit a processing equipment control message to a second processing device to route the item to a location associated with the address information, and

wherein the unattended image processing server is configured to dynamically generate the rotation angle based on a relationship between a maximum pixel density and an average pixel density for a row or a column, and

wherein the relationship is expressed as Equation (1) below:

rotation

⁢

=

30

*

density

w

-

avg

-

density

w

-

max

density

w

-

avg

Equation

⁢

⁢

(

1

)

where density w-max is the maximum density for a row of the image; and

density w-avg is the average density for the rows of the image.

20. The system of claim 19 , wherein the unattended image processing server is further configured to:

receive the image showing the item and address information affixed thereto; and

detect the orientation information for the image based on a location where the address information is affixed to the item.

21. The system of claim 19 , wherein the unattended image processing server is further configured to:

generate a first binary image based on the image;

generate a second binary image based on the image and a second rotation angle, wherein the item is shown at a different orientation in the second binary image than the item is shown in the first binary image;

generate a pixel density difference metric for individual rows of pixels included in the first binary image;

generate a pixel density difference metric for individual columns of pixels included in the first binary image;

identify a first row of the first binary image associated with a first pixel density difference metric, wherein the first pixel density difference metric for the first row indicates a level of density higher than other rows of the first binary image;

identify a first column of the first binary image associated with a second pixel density difference metric, wherein the second pixel density difference metric for the first column indicates a level of density higher than other columns of the first binary image;

generate a pixel density difference metric for individual rows of pixels included in the second binary image;

generate a pixel density difference metric for individual columns of pixels included in the second binary image;

identify a second row of the second binary image associated with a third pixel density difference metric, wherein the third pixel density difference metric for the second row indicates a level of density higher than: (a) other rows of the second binary image, and (b) the first pixel density difference metric; and

identify a second column of the second binary image associated with a fourth pixel density difference metric, wherein the fourth pixel density difference metric for the second column indicates a level of density higher than: (a) other columns of the second binary image, and (b) the second pixel density difference metric.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 7, 2018
From: SIMPSON, RYAN J.
To: UNITED STATES POSTAL SERVICE
Reel/Frame 046812/0264 →
Continuity (2)
Provisional Application 62536309 · Jul 24, 2017
Related Publication 20190026587A1 · Jan 24, 2019