IP Library Granted Patent US 11,302,025
Granted Patent B2
US 11,302,025 · App. 16/696,533 · Granted Apr 12, 2022

Error mitigation for mobile dimensioning in stereo vision

Inventors: Raveen T. Thrimawithana (Pannipitiya, LK); Michael Wijayantha Medagama (Nawala, LK)
Assignee: Zebra Technologies Corporation
G06T7/62G06F3/04842G06T7/174G06T11/203G06T19/20G06T2210/12G06T2219/2016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,302,025
App. No.
16/696,533
Granted
Apr 12, 2022
Kind
B2
Abstract

The present disclosure provides techniques for dimensioning an object by capturing two three-dimensional images of the object from orthogonal perspectives. For instance, a first three-dimensional image of a first side of a target object, and a second three-dimensional image of an orthogonal side of the generally cuboid object may be captured using a three-dimensional camera. A first two-dimensional axis-aligned minimum bounding box may be generated based on the first three-dimensional image and a second two-dimensional axis-aligned minimum bounding box may be generated based on the second three-dimensional image. The dimensions of the object may be calculated using the width of the first axis-aligned two-dimensional minimum bounding box, the width of the second axis-aligned two-dimensional minimum bounding box, and the greater of the height of the first axis-aligned two-dimensional minimum bounding box and the height of the second axis-aligned two-dimensional minimum bounding box.

Claims (85)

1. A dimensioning method, comprising:

capturing, by a three-dimensional camera from a first perspective, a first three-dimensional image of an environment containing a target object to be dimensioned;

generating, by a processor, based on the first three-dimensional image, a first two-dimensional minimum bounding box containing a first side of the target object;

determining, by the processor, a first height and a first width of the first two-dimensional minimum bounding box containing the first side of the target object;

capturing, by the three-dimensional camera from a second perspective that is orthogonal to the first perspective, a second three-dimensional image of the environment containing the target object to be dimensioned;

generating, by the processor, based on the second three-dimensional image, a second two-dimensional minimum bounding box containing a second side of the target object;

determining, by the processor, a second height and a second width of the second two-dimensional minimum bounding box containing the second side of the target object; and

determining, by the processor, dimensions of the target object based on the first width, the second width, and the greater of the first height and the second height.

2. The dimensioning method of claim 1 , further comprising:

displaying, by a user interface, the first three-dimensional image and a graphical depiction of the first two-dimensional minimum bounding box;

receiving, by the user interface, an indication of a user selection of a second object contained in the first two-dimensional minimum bounding box;

generating, by the processor, based on the user selection, one or more of a corrected first height and a corrected first width for a corrected two-dimensional minimum bounding box containing the target object and excluding the second object; and

determining, by the processor, corrected dimensions of the target object based on the corrected first width, the second width, and the greater of the corrected first height and the second height.

3. The dimensioning method of claim 1 , further comprising:

displaying, by a user interface, the first three-dimensional image and a graphical depiction of the first two-dimensional minimum bounding box;

receiving, by the user interface, an indication of a user selection of a second object not contained in the first two-dimensional minimum bounding box;

generating, by the processor, based on the user selection, one or more of a corrected first height and a corrected first width for a corrected two-dimensional minimum bounding box containing the target object and including the second object; and

determining corrected dimensions of the target object based on the corrected first width, the second width, and the greater of the corrected first height and the second height.

4. The dimensioning method of claim 1 , further comprising:

displaying, by a user interface, the second three-dimensional image and a graphical depiction of the second two-dimensional minimum bounding box;

receiving, by the user interface, an indication of a user selection of a second object contained in the second two-dimensional minimum bounding box;

generating, by the processor, based on the user selection, one or more of a corrected first height and a corrected first width for a corrected second two-dimensional minimum bounding box containing the target object and excluding the second object; and

determining, by the processor, corrected dimensions of the target object based on the first width, the corrected second width, and the greater of the first height and the corrected second height.

5. The dimensioning method of claim 1 , further comprising:

displaying, by a user interface, the second three-dimensional image and a graphical depiction of the second two-dimensional minimum bounding box;

receiving, by the user interface, an indication of a user selection of a second object not contained in the second two-dimensional minimum bounding box;

generating, by the processor, based on the user selection, one or more of a corrected second height and a corrected second width for a corrected second two-dimensional minimum bounding box containing the target object and including the second object; and

determining corrected dimensions of the target object based on the first width, the corrected second width, and the greater of the first height and the corrected second height.

6. The dimensioning method of claim 1 , wherein the first and second two-dimensional minimum bounding boxes are two-dimensional axis-aligned minimum bounding boxes.

7. The dimensioning method of claim 1 , wherein generating the first two-dimensional minimum bounding box containing the target object comprises:

analyzing, by the processor, the first three-dimensional image to determine a plane of a floor in the first three-dimensional image;

rotating, by the processor, the first three-dimensional image such that the plane of the floor is aligned with an X-Z plane of a coordinate system;

rotating, by the processor, the first three-dimensional image such that an edge of the target object is aligned with an X axis of the coordinate system;

segmenting, by the processor, the first three-dimensional image such that a target object segment of the first three-dimensional image contains the first side of the target object and excludes the floor and the environment containing the target object to be dimensioned; and

generating, by the processor the first two-dimensional minimum bounding box containing the first side of the target object.

8. The dimensioning method of claim 1 , wherein generating the second two-dimensional minimum bounding box containing the target object comprises:

analyzing, by the processor, the second three-dimensional image to determine a plane of a floor in the second three-dimensional image;

rotating, by the processor, the second three-dimensional image such that the plane of the floor is aligned with an X-Z plane of a coordinate system;

rotating, by the processor, the second three-dimensional image such that an edge of the target object is aligned with an X axis of the coordinate system; and

segmenting, by the processor, the second three-dimensional image such that a target object segment of the second three-dimensional image contains the second side of the target object and excludes the floor and the environment containing the target object to be dimensioned; and

generating the second two-dimensional minimum bounding box containing the second side of the target object segment.

9. A mobile device, comprising:

a three-dimensional camera,

a memory storing computer executable instructions:

at least one processor configured to interface with the three-dimensional camera and the memory, and configured to execute the computer executable instructions to cause the mobile device to:

capture, by the three-dimensional camera from a first perspective, a first three-dimensional image of an environment containing a target object to be dimensioned;

generate, by the at least one processor, based on the first three-dimensional image, a first two-dimensional minimum bounding box containing a first side of the target object;

determine, by the at least one processor, a first height and a first width of the first two-dimensional minimum bounding box containing the first side of the target object;

capture, by the three-dimensional camera from a second perspective that is orthogonal to the first perspective, a second three-dimensional image of the environment containing the target object to be dimensioned;

generate, by the at least one processor, based on the second three-dimensional image, a second two-dimensional minimum bounding box containing a second side of the target object;

determine, by the at least one processor, a second height and a second width of the second two-dimensional minimum bounding box containing the second side of the target object; and

determine, by the at least one processor, dimensions of the target object based on the first width, the second width, and the greater of the first height and the second height.

10. The mobile device of claim 9 , further comprising a user interface, wherein the at least one processor is further configured to interface with the user interface, and wherein the computer executable instructions further cause the mobile device to:

display, by the user interface, the first three-dimensional image and a graphical depiction of the first two-dimensional minimum bounding box;

receive, by the user interface, an indication of a user selection of a second object contained in the first two-dimensional minimum bounding box;

generate, by the at least one processor, based on the user selection, one or more of a corrected first height and a corrected first width for a corrected two-dimensional minimum bounding box containing the target object and excluding the second object; and

determine, by the at least one processor, corrected dimensions of the target object based on the corrected first width, the second width, and the greater of the corrected first height and the second height.

11. The mobile device of claim 9 , further comprising a user interface, wherein the at least one processor is further configured to interface with the user interface, and wherein the computer executable instructions further cause the mobile device to:

display, by the user interface, the first three-dimensional image and a graphical depiction of the first two-dimensional minimum bounding box;

receive, by the user interface, an indication of a user selection of a second object not contained in the first two-dimensional minimum bounding box;

generate, by the at least one processor, based on the user selection, one or more of a corrected first height and a corrected first width for a corrected two-dimensional minimum bounding box containing the target object and including the second object; and

determine, by the at least one processor, corrected dimensions of the target object based on the corrected first width, the second width, and the greater of the corrected first height and the second height.

12. The mobile device of claim 9 , further comprising a user interface, wherein the at least one processor is further configured to interface with the user interface, and wherein the computer executable instructions further cause the mobile device to:

display, by the user interface, the second three-dimensional image and a graphical depiction of the second two-dimensional minimum bounding box;

receive, by the user interface, an indication of a user selection of a second object contained in the second two-dimensional minimum bounding box;

generate, by the at least one processor, based on the user selection, one or more of a corrected second height and a corrected second width for a corrected second two-dimensional minimum bounding box containing the target object and excluding the second object; and

determine, by the at least one processor, corrected dimensions of the target object based on the first width, the corrected second width, and the greater of the first height and the corrected second height.

13. The mobile device of claim 9 , further comprising a user interface, wherein the at least one processor is further configured to interface with the user interface, and wherein the computer executable instructions further cause the mobile device to:

display, by the user interface, the second three-dimensional image and a graphical depiction of the second two-dimensional minimum bounding box;

receive, by the user interface, an indication of a user selection of a second object not contained in the second two-dimensional minimum bounding box;

generate, by the at least one processor, based on the user selection, one or more of a corrected second height and a corrected second width for a corrected second two-dimensional minimum bounding box containing the target object and including the second object; and

determine, by the at least one processor, corrected dimensions of the target object based on the first width, the corrected second width, and the greater of the first height and the corrected second height.

14. The mobile device of claim 9 , wherein the first and second two-dimensional minimum bounding boxes are two-dimensional axis-aligned minimum bounding boxes.

15. The mobile device of claim 9 , wherein the computer executable instructions cause the mobile device to generate the first two-dimensional minimum bounding box containing the target object by:

analyze, by the at least one processor, the first three-dimensional image to determine a plane of a floor in the first three-dimensional image;

rotate, by the at least one processor, the first three-dimensional image such that the plane of the floor is aligned with an X-Z plane of a coordinate system;

rotate, by the at least one processor, the first three-dimensional image such that an edge of the target object is aligned with an X axis of the coordinate system;

segment, by the at least one processor, the first three-dimensional image such that a target object segment of the first three-dimensional image contains the first side of the target object and excludes the floor and the environment containing the target object to be dimensioned; and

generate, by the at least one processor, the first two-dimensional minimum bounding box containing the first side of the target object segment.

16. The mobile device of claim 9 , wherein the computer executable instructions cause the mobile device to generate the second two-dimensional minimum bounding box containing the target object by:

analyze, by the at least one processor, the second three-dimensional image to determine a plane of a floor in the first three-dimensional image;

rotate, by the at least one processor, the second three-dimensional image such that the plane of the floor is aligned with an X-Z plane of a coordinate system;

rotate, by the at least one processor, the second three-dimensional image such that an edge of the target object is aligned with an X axis of the coordinate system;

segment, by the at least one processor, the second three-dimensional image such that a target object segment of the second three-dimensional image contains the second side of the target object and excludes the floor and the environment containing the target object to be dimensioned; and

generate, by the at least one processor, a second two-dimensional minimum bounding box containing the second side of the target object segment.

Assignments (3)
RELEASE OF SECURITY INTEREST - 364 - DAY Recorded Mar 5, 2021
From: JPMORGAN CHASE BANK, N.A.
To: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
Reel/Frame 056036/0590 →
SECURITY INTEREST Recorded Sep 1, 2020
From: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 053841/0212 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2020
From: THRIMAWITHANA, RAVEEN T.; MEDAGAMA, MICHAEL WIJAYANTHA
To: ZEBRA TECHNOLOGIES CORPORATION
Reel/Frame 052923/0576 →