IP Library › Granted Patent US 11,481,567
Granted Patent B2
US 11,481,567 · App. 16/722,000 · Granted Oct 25, 2022

Methods and systems of package processing in a material handling environment

Inventors: Franck Laffargue (Toulouse, FR); Jean-Luc Courtemanche (Cadeilhan, FR); Robert Michael Hussey (Waxhaw, NC)
Assignee: HAND HELD PRODUCTS, INC.
G06K7/10722G06K7/1417
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,481,567
App. No.
16/722,000
Granted
Oct 25, 2022
Kind
B2
Abstract

A method for processing the packages in a material handling environment is disclosed. The method includes receiving an image of a field of view of an image capturing device. The image comprising at least one decodable indicia having a predefined shape, the at least one decodable indicia attached to a package. The method further includes identifying one or more decodable indicia image coordinates, wherein each of the one or more decodable indicia image coordinates define a corner of the at least one decodable indicia using a 2-dimensional (2D) coordinate system. The method further includes transforming the one or more decodable indicia image coordinates to one or more decodable indicia spatial coordinates in the 3D coordinate system. Further, the method includes comparing the one or more decodable indicia spatial coordinates with one or more 3D region coordinates to determine whether the package is within a 3D region.

Claims (65)

1. A system comprising:

a computing device, the computing device comprising a processor and a memory including computer program code, the memory and the computer program code are configured, when executed on the processor, to:

access an image captured by an image capturing device, wherein the image comprises at least one decodable indicia associated with a package;

identify one or more decodable indicia image coordinates, wherein each of the one or more decodable indicia image coordinates define the at least one decodable indicia in the image using a 2-dimensional (2D) coordinate system;

identify one or more 3D region coordinates, wherein each of the one or more 3D region coordinates define a 3D region using a 3D coordinate system, and wherein the 3D region is adapted to be filled with a plurality of packages;

transform the one or more decodable indicia image coordinates to one or more decodable indicia spatial coordinates in the 3D coordinate system; and

compare the one or more decodable indicia spatial coordinates with the one or more 3D region coordinates to determine whether at least a portion of the package protrudes out from the 3D region.

2. The system of claim 1 , wherein, to transform the one or more decodable indicia image coordinates, the memory and the computer program code are further configured, when executed on the processor, to:

determine interim values of an X-coordinate and a Y-coordinate of the at least one decodable indicia in the 3D coordinate system;

determine an actual value of a Z-coordinate of the at least one decodable indicia based on a corrective factor and an interim value of the Z-coordinate; and

determine an actual value of the X coordinate and the Y coordinate of the at least one decodable indicia based on the actual value of the Z coordinate, wherein each of the one or more decodable indicia spatial coordinates comprise the X-coordinate, the Y-coordinate, and the Z coordinate.

3. The system of claim 2 , wherein the memory and the computer program code are further configured, when executed on the processor, to:

determine an interim value of a Z coordinate;

determine a length of the at least one decodable indicia based on the interim values of Z-coordinate, the X-coordinate, and the Y-coordinate; and

determine the corrective factor based on the determined length of the at least one decodable indicia and a predefined length of the at least one decodable indicia.

4. The system of claim 3 , wherein the memory and the computer program code are further configured, when executed on the processor, to:

determine whether the actual value of the Z coordinate of the at least one decodable indicia exceeds a threshold Z coordinate value; and

cause the actual value of the Z-coordinate for each of the one or more decodable indicia spatial coordinates to be the same.

5. The system of claim 1 , wherein the captured image further comprises one or more second decodable indicia, wherein the one or more second decodable indicia defines a boundary of the 3D region or comprises encoded information that is representative of the one or more 3D region coordinates.

6. The system of claim 1 , wherein each of the one or more decodable indicia image coordinates define a corner of the at least one decodable indicia; and wherein each of the one or more 3D region coordinates define a corner of the 3D region.

7. The system of claim 1 , wherein the 3D region comprises a base region and one or more virtual wall regions, wherein the one or more virtual wall regions are perpendicular to and extend from the base region towards the image capturing device, wherein the base region has a first set of 3D region coordinates and the one or more virtual wall regions have a second set of 3D region coordinates, and wherein the one or more 3D region coordinates comprise the first set of 3D region coordinates and the second set of 3D region coordinates.

8. The system of claim 7 , wherein the base region is a pallet and wherein the first set of 3D region coordinates of the one or more 3D region coordinates define one or more corners of the pallet.

9. The system of claim 7 , wherein the memory and the computer program code are further configured, when executed on the processor, to:

define a package plane based on the one or more decodable indicia spatial coordinates; and

determine one or more intersection coordinates in the 3D coordinate system, wherein each of the one or more intersection coordinates define a 3D point where the package plane intersects with the one or more virtual wall regions.

10. The system of claim 9 , wherein the memory and the computer program code are further configured, when executed on the processor, to:

identify one or more package coordinates in the 3D coordinate system, wherein each of the one or more package coordinates define a corner of the package;

determine whether the package protrudes beyond the one or more virtual wall regions based on a comparison of the one or more package coordinates to the one or more intersection coordinates, and

generate a notification in an instance in which the package protrudes beyond the one or more virtual wall regions.

11. The system of claim 1 , wherein the memory and the computer program code are further configured, when executed on the processor, to:

determine whether the package is inside the 3D region;

in an instance in which the package is inside the 3D region, determine that the package is within the 3D region and decode the at least one decodable indicia; and

in an instance in which the package is outside the 3D region, determine that the package is not within the 3D region.

12. A method comprising:

receiving an image of a field of view of an image capturing device, the image comprising at least one decodable indicia having a predefined shape, the at least one decodable indicia is associated with a package;

identifying one or more decodable indicia image coordinates, wherein each of the one or more decodable indicia image coordinates define the at least one decodable indicia using a 2-dimensional (2D) coordinate system;

identifying one or more 3D region coordinates, wherein each of the one or more 3D region coordinates define a 3D region using a 3D coordinate system, and wherein the 3D region is adapted to be filled with a plurality of packages;

transforming the one or more decodable indicia image coordinates to one or more decodable indicia spatial coordinates in the 3D coordinate system; and

comparing the one or more decodable indicia spatial coordinates with the one or more 3D region coordinates to determine whether at least a portion of the package protrudes out from the 3D region.

13. The method of claim 12 , wherein transforming the one or more decodable indicia image coordinates comprises:

determining interim values of an X-coordinate and a Y-coordinate of the at least one decodable indicia in the 3D coordinate system;

determining an actual value of a Z-coordinate of the at least one decodable indicia based on a corrective factor and an interim value of the Z-coordinate; and

determining an actual value of the X coordinate and the Y coordinate of the at least one decodable indicia based on the actual value of the Z coordinate, wherein each of the one or more decodable indicia spatial coordinates comprise the X-coordinate, Y-coordinate, and Z coordinate.

14. The method of claim 13 further comprising:

determining an interim value of a Z coordinate;

determining a length of the at least one decodable indicia based on the interim values of Z-coordinate, the X-coordinate, and the Y-coordinate; and

determining the corrective factor based on the determined length of the at least one decodable indicia and a predefined length of the at least one decodable indicia.

15. The method of claim 14 further comprising

determining whether the actual value of the Z coordinate of the at least one decodable indicia exceeds a threshold Z coordinate value; and

causing the actual value of the Z-coordinate for each of the one or more decodable indicia spatial coordinates to be the same.

16. The method of claim 12 , wherein the captured image further comprises one or more second decodable indicia, wherein the one or more second decodable indicia defines a boundary of the 3D region or comprises encoded information that is representative of the one or more 3D region coordinates.

17. The method of claim 12 , wherein the 3D region comprises a base region and one or more virtual wall regions, wherein the one or more virtual wall regions are perpendicular to and extend from the base region towards the image capturing device, wherein the base region has a first set of 3D region coordinates and the one or more virtual wall regions have a second set of 3D region coordinates, and wherein the one or more 3D region coordinates comprise the first set of 3D region coordinates and the second set of 3D region coordinates.

18. The method of claim 17 , wherein the base region is a pallet and wherein the first set of 3D region coordinates of the one or more 3D region coordinates define one or more corners of the pallet.

19. The method of claim 17 further comprising:

defining a package plane based on the one or more decodable indicia spatial coordinates; and

determining one or more intersection coordinates in the 3D coordinate system, wherein each of the one or more intersection coordinates define a 3D point where the package plane intersects with the one or more virtual wall regions.

20. An image capturing device comprising:

an image sensor configured to receiving light signal from a field of view and generate an image based on the received light signal;

a processor communicatively coupled to the image sensor, the processor configured to:

identify at least one decodable indicia in the generated image, the at least one decodable indicia having a predefined shape, the at least one decodable indicia attached to a package;

determine one or more decodable indicia image coordinates, wherein each of the one or more decodable indicia image coordinates define a corner of the at least one decodable indicia using a 2-dimensional (2D) coordinate system;

define a 3-dimensional (3D) region within the field of view based at least on the captured image, wherein the 3D region is adapted to be filled with a plurality of packages;

identify one or more 3D region coordinates, wherein each of the one or more 3D region coordinates define a corner of the 3D region using a 3D coordinate system;

transform the one or more decodable indicia image coordinates to one or more decodable indicia spatial coordinates in the 3D coordinate system; and

compare the one or more decodable indicia spatial coordinates with the one or more 3D region coordinates to determine whether at least a portion of the package protrudes out from the 3D region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2019
From: LAFFARGUE, FRANCK; COURTEMANCHE, JEAN-LUC; HUSSEY, ROBERT MICHAEL
To: HAND HELD PRODUCTS, INC.
Reel/Frame 051340/0085 →
Continuity (2)
Provisional Application 62784031 · Dec 21, 2018
Related Publication 20200202088A1 · Jun 25, 2020
Cited By (1)
US 12,217,122