IP Library Granted Patent US 11,430,104
Granted Patent B2
US 11,430,104 · App. 16/795,243 · Granted Aug 30, 2022

Three-dimensional (3D) imaging systems and methods for detecting and dimensioning a vehicle storage area

Inventors: Justin F. Barish (Kings Park, NY); Adithya H. Krishnamurthy (Hicksville, NY)
Assignee: Zebra Technologies Corporation
G06T7/0004G06T7/593G06T7/62G06T15/00G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,430,104
App. No.
16/795,243
Granted
Aug 30, 2022
Kind
B2
Abstract

Three-dimensional (3D) imaging systems and methods are disclosed for detecting and dimensioning a vehicle storage area. A 3D-depth camera captures 3D image data comprising one or more 3D image datasets of the vehicle storage area during corresponding one or more image capture iterations. A 3D data analytics application (app) executing on one or more processors communicatively coupled to the 3D-depth camera and for each one or more image capture iterations, updates a number of planar regions detected within the one or more 3D image datasets. The 3D data analytics app further assigns a vehicle storage area type to the vehicle storage area based on the number of planar regions detected by the 3D analytics app over the one or more image capture iterations.

Claims (60)

1. A three-dimensional (3D) imaging system configured to detect and dimension a vehicle storage area, the 3D imaging system comprising:

a 3D-depth camera configured to capture 3D image data, the 3D-depth camera oriented in a direction to capture 3D image data of a vehicle storage area; and

a 3D data analytics application (app) executing on one or more processors communicatively coupled to the 3D-depth camera, the 3D data analytics app configured to:

capture one or more 3D image datasets of the vehicle storage area during corresponding one or more image capture iterations,

update, for each one or more image capture iterations, a number of planar regions detected within the one or more 3D image datasets,

assign a vehicle storage area type to the vehicle storage area based on the number of planar regions detected by the 3D analytics app over the one or more image capture iterations,

wherein the vehicle storage area type assigned is one of (1) a straight-rail trailer type when the number of planar regions detected equals one planar region, or (2) a drop-frame trailer type when the number of planar regions detected exceeds one planar region,

wherein:

each planar region is comprised of a dataset of 3D data points defining a planar surface, each planar surface having a planar thickness value;

the 3D data analytics app, upon each iteration of the one or more image capture iterations, increases a planar thickness threshold;

the 3D analytics app detects, during a particular iteration, a floor plane of the vehicle storage area when the planar thickness value is within the planar thickness threshold;

each planar surface is associated with a planar error angle value;

3D data analytics app, upon each iteration of the one or more image capture iterations, increases a planar error angle threshold; and

the 3D analytics app detects the floor plane, during the particular iteration, when the planar error angle value is within the planar thickness threshold.

2. The 3D imaging system of claim 1 , wherein the vehicle storage area type assigned is the drop-frame trailer type, and

wherein the 3D data analytics app is further configured to:

detect, based on the one or more 3D image datasets, at least two planar regions comprising a first planar region corresponding to a raised floor plane of the vehicle storage area and a second planar region corresponding to at least one lower floor plane of vehicle storage area,

dimension, based on the raised floor plane and the at least one lower floor plane, a cropped 3D storage area of the vehicle storage area, wherein the cropped 3D storage area is dimensioned by the 3D data analytics app cropping a lower 3D portion of the vehicle storage area located, within the 3D image data, between the at least one lower floor plane and the raised floor plane, and

determine an estimated fullness value of the vehicle storage area based on the cropped 3D storage area.

3. The 3D imaging system of claim 1 , wherein the one or more image capture iterations are limited to a predefined number of iterations.

4. The 3D imaging system of claim 1 further comprising a memory configured to store a trailer status parameter defining a loading state of the vehicle storage area.

5. The 3D imaging system of claim 4 , wherein the trailer status parameter indicates that a vehicle corresponding to the vehicle storage area is inbound and the loading state of the vehicle storage area is full.

6. The 3D imaging system of claim 4 , wherein the trailer status parameter indicates that a vehicle corresponding to the vehicle storage area is outbound and the loading state of the vehicle storage area is empty.

7. The 3D imaging system of claim 1 , further comprising a dashboard app, the dashboard app executing on a client device, and wherein the dashboard app indicates, on a display of the client device, the vehicle storage area type corresponding to the vehicle storage area.

8. The 3D imaging system of claim 7 , wherein the dashboard app is configured to display a graphical representation of the vehicle storage area, the graphical representation rendering an estimated fullness value on the display of the client device.

9. The 3D imaging system of claim 1 , wherein the 3D-depth camera and the one or more processors are housed in a mountable device.

10. The 3D imaging system of claim 1 , wherein the one or more processors are remote to the 3D-depth camera, the one or more processors comprising a server commutatively coupled to the 3D-depth camera through a computer network.

11. A three-dimensional (3D) imaging method for detecting and dimensioning a vehicle storage area, the 3D imaging method comprising:

capturing, by a 3D-depth camera oriented in a direction to capture 3D image data of a vehicle storage area, 3D image data comprising one or more 3D image datasets of the vehicle storage area during corresponding one or more image capture iterations;

updating, by a 3D data analytics application (app) executing on one or more processors communicatively coupled to the 3D-depth camera and for each one or more image capture iterations, a number of planar regions detected within the one or more 3D image datasets; and

assigning, by the 3D data analytics app, a vehicle storage area type to the vehicle storage area based on the number of planar regions detected by the 3D analytics app over the one or more image capture iterations,

wherein the vehicle storage area type assigned is one of (1) a straight-rail trailer type when the number of planar regions detected equals one planar region, or (2) a drop-frame trailer type when the number of planar regions detected exceeds one planar region,

wherein:

each planar region is comprised of a dataset of 3D data points defining a planar surface, each planar surface having a planar thickness value;

the 3D data analytics app, upon each iteration of the one or more image capture iterations, increases a planar thickness threshold;

the 3D analytics app detects, during a particular iteration, a floor plane of the vehicle storage area when the planar thickness value is within the planar thickness threshold;

each planar surface is associated with a planar error angle value;

3D data analytics app, upon each iteration of the one or more image capture iterations, increases a planar error angle threshold; and

the 3D analytics app detects the floor plane, during the particular iteration, when the planar error angle value is within the planar thickness threshold.

12. The 3D imaging method of claim 11 , wherein the vehicle storage area type assigned is the drop-frame trailer type, and wherein the 3D imaging method further comprises:

detecting, by the 3D data analytics app based on the one or more 3D image datasets, at least two planar regions comprising a first planar region corresponding to a raised floor plane of the vehicle storage area and a second planar region corresponding to at least one lower floor plane of vehicle storage area,

dimensioning, by the 3D data analytics app based on the raised floor plane and the at least one lower floor plane, a cropped 3D storage area of the vehicle storage area, wherein the cropped 3D storage area is dimensioned by the 3D data analytics app cropping a lower 3D portion of the vehicle storage area located, within the 3D image data, between the at least one lower floor plane and the raised floor plane, and

determining, by the 3D data analytics app, an estimated fullness value of the vehicle storage area based on the cropped 3D storage area.

13. A tangible, non-transitory computer-readable medium storing instructions for detecting and dimensioning a vehicle storage area, that when executed by one or more processors cause the one or more processors to:

capture, by a 3D-depth camera oriented in a direction to capture 3D image data of a vehicle storage area, 3D image data comprising one or more 3D image datasets of the vehicle storage area during corresponding one or more image capture iterations;

update, by a 3D data analytics application (app) for each one or more image capture iterations, a number of planar regions detected within the one or more 3D image datasets; and

assign, by the 3D data analytics app, a vehicle storage area type to the vehicle storage area based on the number of planar regions detected by the 3D analytics app over the one or more image capture iterations,

wherein the vehicle storage area type assigned is one of (1) a straight-rail trailer type when the number of planar regions detected equals one planar region, or (2) a drop-frame trailer type when the number of planar regions detected exceeds one planar region

wherein:

the vehicle storage area type assigned is the drop-frame trailer type;

the instructions are further configured, when executed by the one or more processors, to:

detect, based on the one or more 3D image datasets, at least two planar regions comprising a first planar region corresponding to a raised floor plane of the vehicle storage area and a second planar region corresponding to at least one lower floor plane of vehicle storage area,

dimension, based on the raised floor plane and the at least one lower floor plane, a cropped 3D storage area of the vehicle storage area, wherein the cropped 3D storage area is dimensioned by the 3D data analytics app cropping a lower 3D portion of the vehicle storage area located, within the 3D image data, between the at least one lower floor plane and the raised floor plane, and

determine an estimated fullness value of the vehicle storage area based on the cropped 3D storage area;

each planar region is comprised of a dataset of 3D data points defining a planar surface, each planar surface having a planar thickness value;

the 3D data analytics app, upon each iteration of the one or more image capture iterations, increases a planar thickness threshold;

the 3D analytics app detects, during a particular iteration, a floor plane of the vehicle storage area when the planar thickness value is within the planar thickness threshold;

each planar surface is associated with a planar error angle value;

3D data analytics app, upon each iteration of the one or more image capture iterations, increases a planar error angle threshold; and

the 3D analytics app detects the floor plane, during the particular iteration, when the planar error angle value is within the planar thickness threshold.

Assignments (4)
SECURITY INTEREST Recorded Apr 12, 2021
From: ZEBRA TECHNOLOGIES CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 056472/0063 →
RELEASE OF SECURITY INTEREST - 364 - DAY Recorded Mar 5, 2021
From: JPMORGAN CHASE BANK, N.A.
To: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
Reel/Frame 056036/0590 →
SECURITY INTEREST Recorded Sep 1, 2020
From: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 053841/0212 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2020
From: BARISH, JUSTIN F.; KRISHNAMURTHY, ADITHYA H.
To: ZEBRA TECHNOLOGIES CORPORATION
Reel/Frame 051978/0789 →