IP Library Granted Patent US 10,991,116
Granted Patent B1
US 10,991,116 · App. 16/664,428 · Granted Apr 27, 2021

Three-dimensional (3D) depth imaging systems and methods for automatically determining shipping container fullness based on imaging templates

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,991,116
App. No.
16/664,428
Granted
Apr 27, 2021
Kind
B1
Abstract

Three-dimensional (3D) depth imaging systems and methods are disclosed for automatically determining shipping container fullness based on imaging templates. A 3D-depth camera captures 3D image data of a shipping container located in a predefined search space during a shipping container loading session. A container fullness application (app) receives the 3D image data, and determines therefrom a 3D container point cloud representative of a shipping container. An imaging template that defines a 3D template point cloud corresponding to a shipping container type of the shipping container is loaded into memory. A fullness value of the shipping container is determined based on a 3D mapping that is generated from alignment of a 3D container front portion of the 3D container point cloud with a 3D template front portion of the 3D template point cloud.

Claims (33)

1. A three-dimensional (3D) depth imaging system configured to automatically determining shipping container fullness based on imaging templates, the 3D depth imaging system comprising:

a 3D-depth camera configured to capture 3D image data, the 3D-depth camera oriented in a direction to capture 3D image data of a shipping container located in a predefined search space during a shipping container loading session, the shipping container having a shipping container type; and

a container fullness application (app) configured to execute on one or more processors communicatively coupled to the 3D-depth camera, the container fullness app further configured to:

determine, based on 3D image data received from the 3D-depth camera, a 3D container point cloud representative of the shipping container,

load, into a memory communicatively coupled to the one or more processors, an imaging template that defines a 3D template point cloud corresponding to the shipping container type, the 3D template point cloud comprising a 3D template front portion of the shipping container type and a 3D template back portion of the shipping container type,

identify, within the 3D container point cloud, a 3D container front portion corresponding to a physical container front portion of the shipping container,

generate a 3D mapping of the 3D container point cloud and the 3D template point cloud, the 3D mapping comprising the 3D container front portion of the 3D container point cloud aligned with the 3D template front portion of the 3D template point cloud, and

determine, based on the 3D mapping, a fullness value of the shipping container.

2. The 3D depth imaging system of claim 1 , wherein the 3D container point cloud representative of the shipping container is a cropped portion of point cloud data of the 3D image data received from the 3D-depth camera.

3. The 3D depth imaging system of claim 1 , wherein the container fullness application determines the imaging template based on the 3D image data received from the 3D-depth camera, wherein a machine learning model, trained with 3D image data defining a plurality of shipping container types, inputs the 3D image data and outputs the shipping container type, and wherein the imaging template is loaded based on the shipping container type.

4. The 3D depth imaging system of claim 1 , wherein generating the 3D mapping comprises executing a closest point algorithm that generates a 3D match score based on distances between a first set of 3D points of the 3D container point cloud and a second set of 3D points of the 3D template point cloud.

5. The 3D depth imaging system of claim 1 , wherein the imaging template is a non-point cloud imaging template, and wherein in loading the imaging template includes converting, by the one or more processors, the non-point cloud template to the 3D template point cloud.

6. The 3D depth imaging system of claim 5 , wherein the non-point cloud template is a computer aided design (CAD) model.

7. The 3D depth imaging system of claim 1 , wherein the imaging template is preloaded into the memory.

8. The 3D depth imaging system of claim 1 , wherein the shipping container is a unit load device (ULD).

9. The 3D depth imaging system of claim 8 , wherein the shipping container type is a ULD type being one of: an AMJ type, an AAD type, an AKE type, an AYY type, a SAA type, an APE type, or an AQF type.

10. The 3D depth imaging system of claim 1 , wherein the imaging template comprises a template of a unit load device (ULD).

11. The 3D depth imaging system of claim 1 , wherein the 3D-depth camera and the one or more processors are housed in a mountable device.

12. The 3D depth imaging system of claim 1 , wherein the one or more processors are located at a server communicatively coupled to the 3D-depth camera via a digital network.

13. The 3D depth imaging system of claim 1 , wherein determining the fullness value comprises projecting the 3D mapping onto a two-dimensional (2D) grid corresponding to a storage area of the shipping container.

14. The 3D depth imaging system of claim 1 , further comprising a dashboard app, the dashboard app executing on a client device implementing a guided user interface (GUI), the GUI graphically indicating the fullness value of the shipping container.

15. A three-dimensional (3D) depth imaging method for automatically determining shipping container fullness based on imaging templates, the 3D depth imaging method comprising:

determining, based on 3D image data received from a 3D-depth camera, a 3D container point cloud representative of a shipping container, the shipping container located in a predefined search space during a shipping container loading session, the shipping container having a shipping container type;

loading, into a memory communicatively coupled to one or more processors, an imaging template that defines a 3D template point cloud corresponding to the shipping container type, the 3D template point cloud comprising a 3D template front portion of the shipping container type and a 3D template back portion of the shipping container type;

identifying, within the 3D container point cloud, a 3D container front portion corresponding to a physical container front portion of the shipping container;

generating a 3D mapping of the 3D container point cloud and the 3D template point cloud, the 3D mapping comprising the 3D container front portion of the 3D container point cloud aligned with the 3D template front portion of the 3D template point cloud; and

determining, based on the 3D mapping, a fullness value of the shipping container.

16. A tangible, non-transitory computer-readable medium storing instructions for automatically determining shipping container fullness based on imaging templates, that when executed by one or more processors of a computing device cause the one or more processors of the computing device to:

determine, based on 3D image data received from a 3D-depth camera, a 3D container point cloud representative of a shipping container, the shipping container located in a predefined search space during a shipping container loading session, the shipping container having a shipping container type;

load, into a memory communicatively coupled to one or more processors, an imaging template that defines a 3D template point cloud corresponding to the shipping container type, the 3D template point cloud comprising a 3D template front portion of the shipping container type and a 3D template back portion of the shipping container type;

identify, within the 3D container point cloud, a 3D container front portion corresponding to a physical container front portion of the shipping container;

generate a 3D mapping of the 3D container point cloud and the 3D template point cloud, the 3D mapping comprising the 3D container front portion of the 3D container point cloud aligned with the 3D template front portion of the 3D template point cloud; and

determine, based on the 3D mapping, a fullness value of the shipping container.

Assignments (3)
RELEASE OF SECURITY INTEREST - 364 - DAY Recorded Mar 5, 2021
From: JPMORGAN CHASE BANK, N.A.
To: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
Reel/Frame 056036/0590 →
SECURITY INTEREST Recorded Sep 1, 2020
From: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 053841/0212 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2020
From: BARISH, JUSTIN F.
To: ZEBRA TECHNOLOGIES CORPORATION
Reel/Frame 052757/0262 →