IP Library Granted Patent US 12,020,199
Granted Patent B2
US 12,020,199 · App. 17/303,237 · Granted Jun 25, 2024

Method and apparatus for tracking, damage detection and classification of a shipping object using 3D scanning

Inventors: Aamir Shahzad (Oslo, NO); Michael Heidingsfelder (Kiawah Island, SC); Eivind Kvedalen (Oslo, NO)
Assignee: Mettler-Toledo GmbH
G06Q10/0833G06F16/90335G06T7/0004G06T17/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,020,199
App. No.
17/303,237
Granted
Jun 25, 2024
Kind
B2
Abstract

A method, an apparatus and a computer program are provided for tracking a shipping object that has been selected from a stock of objects. The shipping object is subjected to a 3D scanning as it is being transported along a transport path from a first location to a second location distant from the first location.

Claims (41)

1. A method of tracking an object being shipped, the object moving along a transport path from a first location to a second location that is distant from the first location, the object selected from a stock of objects, the method comprising the steps of:

providing a database structure containing, for each object in the stock of objects, a content-based descriptor of a three-dimensional (“3D”) point cloud representation attributed to the object, each content-based descriptor derived by a predefined descriptor algorithm;

creating, for each of a plurality of locations along the transport path, a scanned 3D point cloud of at least a region of an outer surface of the object being shipped, by scanning the object being shipped with a three-dimensional (“3D”) scanning system at each of the locations, the three-dimensional scanning system;

deriving a scanned content-based descriptor for the object being shipped at each of the locations by applying the descriptor algorithm to each of the scanned 3D point clouds;

performing a similarity search, for each of the locations, by comparing the scanned content-based descriptor associated with the location to the content-based descriptors in the database structure, using a predefined similarity search algorithm; and

maintaining a record of a result of each performed similarity search for each of the plurality of locations along the transport path.

2. The method of claim 1 , wherein:

the object being shipped is scanned from a plurality of different directions by each 3D scanning system; and

creating each 3D point cloud comprises:

creating, for each of the plurality of scans, a first 3D point cloud;

extracting a feature, for each of the first 3D point clouds, using a predefined extraction algorithm, and

using the extracted features of the first 3D point clouds for registration of the scanned 3D point cloud of the object being shipped, using a predefined registration algorithm.

3. The method of claim 2 , wherein:

the similarity search algorithm computes a distance between the scanned content-based descriptor and the content-based descriptors in the database structure using a predefined metric.

4. The method of claim 1 , wherein the result of the similarity search comprises information about whether the object being shipped corresponds to one of the objects of the stock.

5. The method of claim 1 , wherein the result of the similarity search comprises information about whether the object being shipped is damaged.

6. The method of claim 1 , wherein the maintained records of the results of the similarity search comprise time and location of the 3D scanning system providing the 3D point cloud.

7. The method of claim 6 , wherein the maintained records of the results of the similarity search comprise information about whether the object being shipped is damaged.

8. The method of claim 1 , wherein the content-based descriptor is based on a geometric feature of the 3D point cloud.

9. The method of claim 1 , further comprising:

classifying the 3D point cloud representations of at least one of: the objects of the stock and the scanned 3D point clouds.

10. The method of claim 9 , wherein the artificial intelligence used is a convolutional neural network.

11. The method of claim 3 , wherein the maintained records of the results of the similarity search comprise time and location of the 3D scanning system providing the 3D point cloud.

12. The method of claim 3 , further comprising:

classifying the 3D point cloud representations of at least one of: the objects of the stock and the scanned 3D point clouds.

13. An apparatus for tracking an object being shipped, the object moving along a transport path from a first location to a second location that is distant from the first location, the object selected from a stock of objects, the apparatus comprising:

a database comprising a database structure in which each object in the stock of objects has a content-based descriptor of a 3D point cloud representation attributed thereto, the content-based descriptor being derived by a predefined descriptor algorithm;

a 3D scanning system, applicable to the object being shipped at each of a plurality of locations along the transport path, the 3D scanning system operative to create, for each of the plurality of locations, a scanned 3D point cloud of at least a region of an outer surface of the object being shipped;

a data processor, in communication with both the database and the 3D scanning system, configured to:

apply the descriptor algorithm to each of the scanned 3D point clouds to derive a scanned content-based descriptor associated with the respective location; and

perform, for each of the plurality of locations, a similarity search between the scanned content-based descriptor associated with the respective location and the content-based descriptors in the database structure, using a predefined similarity search algorithm, and

means for maintaining a record of the results of the similarity search for each of the plurality of locations along the transport path.

14. The apparatus of claim 13 , wherein the three-dimensional scanning system comprises a light detection and ranging (“LIDAR”) system.

15. The apparatus of claim 13 , further comprising:

a conveyor belt or a forklift that moved the object being shipped along the transport path.

16. A non-transitory computer-readable medium having stored thereon a computer program for tracking an object being shipped, selected from a stock of objects, along a transport path from a first location to a second location that is distant from the first location, the computer program comprising instructions that, when executed by a computer, cause the computer to carry out the following steps:

providing a database structure containing, for each object in the stock of objects, a content-based descriptor of a three-dimensional (“3D”) point cloud representation attributed to the object, each content-based descriptor derived by a predefined descriptor algorithm;

creating, for each of a plurality of locations along the transport path, a scanned 3D point cloud of at least a region of an outer surface of the object being shipped, by retrieving scanning data obtained by applying a 3D scanning system to the object being shipped at each of a plurality of locations along the transport path;

deriving a scanned content-based descriptor for the object being shipped at each of the locations by applying the descriptor algorithm to each of the scanned 3D point clouds;

performing a similarity search, for each of the locations, by comparing the scanned content-based descriptor associated with the location to the content-based descriptors in the database structure, using a predefined similarity search algorithm; and

maintaining a record of a result of each performed similarity search for each of the plurality of locations along the transport path.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2021
From: SHAHZAD, AAMIR; HEIDINGSFELDER, MICHAEL; KVEDALEN, EIVIND
To: METTLER-TOLEDO GMBH
Reel/Frame 058124/0290 →
Priority Claims (1)
EP 20176751 · May 27, 2020 · regional
Continuity (1)
Related Publication 20210374664A1 · Dec 2, 2021