IP Library Granted Patent US 11,709,046
Granted Patent B2
US 11,709,046 · App. 17/726,998 · Granted Jul 25, 2023

Systems and methods for volumetric sizing

Inventors: Francesco Peruch (Sunnyvale, CA); Giuliano Pasqualotto (Mountain View, CA); Giridhar Murali (Sunnyvale, CA); Ethan Cheng (Cupertino, CA); Kinh Tieu (Sunnyvale, CA)
Assignee: PACKSIZE LLC
G01B11/00G06T7/10G06T7/11G06T7/12G06T7/187G06T7/50G06T7/62G06V20/64G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,709,046
App. No.
17/726,998
Granted
Jul 25, 2023
Kind
B2
Abstract

A method for computing dimensions of an object in a scene includes: controlling, by a processor, a depth camera system to capture at least a frame of the scene, the frame including a color image and a depth image arranged in a plurality of pixels; detecting, by the processor, an object in the frame; determining, by the processor, a ground plane in the frame, the object resting on the ground plane; computing, by the processor, a rectangular outline bounding a projection of a plurality of pixels of the object onto the ground plane; computing, by the processor, a height of the object above the ground plane; and outputting, by the processor, computed dimensions of the object in accordance with a length and a width of the rectangular outline and the height.

Claims (64)

1. A computer system for volumetric sizing, comprising:

one or more processors; and

one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to:

control a camera system to capture at least a frame of a scene, the frame comprising an image arranged in a plurality of pixels;

detect an object in the frame;

identify a plurality of bottom pixels of the frame;

compute a partial plane from the bottom pixels of the frame;

extend the partial plane in the image to define a ground plane;

compute a rectangular outline bounding a projection of a plurality of pixels of the object onto the ground plane;

compute a height of the object above the ground plane; and

output computed dimensions of the object in accordance with a length and a width of the rectangular outline and the height.

2. The computer system as recited in claim 1 , wherein a width of a strip of the bottom pixels is calculated in accordance with noise characteristics of the camera system.

3. The computer system as recited in claim 1 , wherein the camera system comprises a depth camera.

4. The computer system as recited in claim 3 , wherein the camera system comprises a color camera.

5. The computer system as recited in claim 1 , wherein the executable instructions include instructions that are executable to configure the computer system to segment the object from the scene by:

identifying one or more initial pixels of the object; and

performing an iterative flood fill operation, starting with initial points of the object, each iteration of the flood fill operation comprising adding a plurality of neighboring pixels of the frame to the pixels of the object when distances between the neighboring pixels and the pixels of the object are within a threshold distance.

6. The computer system as recited in claim 5 , further comprising a display device coupled to the one or more processors,

wherein the executable instructions include instructions that are executable to configure the computer system to:

control the display device to display a view of a color image of the scene captured by the camera; and

overlay a reticle on the view in the display device, and wherein the one or more initial pixels of the object correspond to the pixels under the reticle.

7. The computer system as recited in claim 6 , further comprising a trigger,

wherein the one or more processors are configured to control the camera system to capture the frame in response to detecting an activation of the trigger, and

wherein the one or more initial pixels of the object correspond to the pixels under the reticle when the trigger is activated.

8. The computer system as recited in claim 1 , wherein the executable instructions include instructions that are executable to configure the computer system to segment the object from the scene by:

defining a graph, wherein each vertex of the graph corresponds to a vertical projection of the pixels of the scene onto the ground plane and wherein two vertices are connected by an edge if their distance is smaller than threshold;

detecting connected components of the vertical projection of the pixels; and

identifying a largest connected component of the graph as the pixels of the object.

9. The computer system as recited in claim 1 , further comprising a display device coupled to the processor, wherein the instructions configured to output the computed dimensions comprise instructions to display the computed dimensions on the display device.

10. A computer-implemented method, executed on one or more processors, for volumetric sizing, the method comprising:

controlling a camera system to capture at least a frame of a scene, the frame comprising an image arranged in a plurality of pixels;

detecting an object in the frame;

identifying a plurality of bottom pixels of the frame;

computing a partial plane from the bottom pixels of the frame;

extending the partial plane in the image to define a ground plane;

computing a rectangular outline bounding a projection of a plurality of pixels of the object onto the ground plane;

computing a height of the object above the ground plane; and

outputting computed dimensions of the object in accordance with a length and a width of the rectangular outline and the height.

11. The computer-implemented method as recited in claim 10 , wherein a width of a strip of the bottom pixels is calculated in accordance with noise characteristics of the camera system.

12. The computer-implemented method as recited in claim 10 , wherein the camera system comprises a depth camera.

13. The computer-implemented method as recited in claim 12 , wherein the camera system comprises a color camera.

14. The computer-implemented method as recited in claim 10 , wherein configuring the computer system to segment the object from the scene comprises:

identifying one or more initial pixels of the object; and

performing an iterative flood fill operation, starting with initial points of the object, each iteration of the flood fill operation comprising adding a plurality of neighboring pixels of the frame to the pixels of the object when distances between the neighboring pixels and the pixels of the object are within a threshold distance.

15. The computer-implemented method as recited in claim 14 , further comprising:

controlling a display device to display a view of a color image of the scene captured by the camera; and

overlaying a reticle on the view in the display device, and wherein the one or more initial pixels of the object correspond to the pixels under the reticle.

16. The computer-implemented method as recited in claim 15 , further comprising receiving a trigger,

wherein the one or more processors are configured to control the camera system to capture the frame in response to detecting an activation of the trigger, and

wherein the one or more initial pixels of the object correspond to the pixels under the reticle when the trigger is activated.

17. The computer-implemented method as recited in claim 10 , wherein configuring the computer system to segment the object from the scene comprises:

defining a graph, wherein each vertex of the graph corresponds to a vertical projection of the pixels of the scene onto the ground plane and wherein two vertices are connected by an edge if their distance is smaller than threshold;

detecting connected components of the vertical projection of the pixels; and

identifying a largest connected component of the graph as the pixels of the object.

18. The computer-implemented method as recited in claim 10 , further comprising outputting the computed dimensions comprise instructions to display the computed dimensions on a display device.

19. A non-transitory computer-readable media comprising one or more physical computer-readable storage media having stored thereon computer-executable instructions that, when executed at a processor, cause a computer system to perform a method for volumetric sizing, the method comprising:

controlling a camera system to capture at least a frame of a scene, the frame comprising an image arranged in a plurality of pixels;

detecting an object in the frame;

identifying a plurality of bottom pixels of the frame;

computing a partial plane from the bottom pixels of the frame;

extending the partial plane in the image to define a ground plane;

computing a rectangular outline bounding a projection of a plurality of pixels of the object onto the ground plane;

computing a height of the object above the ground plane; and

outputting computed dimensions of the object in accordance with a length and a width of the rectangular outline and the height.

Assignments (5)
SECURITY INTEREST Recorded May 15, 2025
From: PACKSIZE LLC
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 071282/0082 →
SECURITY INTEREST Recorded Aug 21, 2024
From: PACKSIZE LLC
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 068730/0393 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2023
From: PERUCH, FRANCESCO; PASQUALOTTO, GIULIANO; MURALI, GIRIDHAR; CHENG, ETHAN; TIEU, KINH
To: AQUIFI, INC.
Reel/Frame 063672/0413 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2023
From: AQUIFI, INC.
To: PACKSIZE INTERNATIONAL, LLC
Reel/Frame 063672/0624 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2023
From: PACKSIZE INTERNATIONAL, LLC
To: PACKSIZE LLC
Reel/Frame 063672/0712 →
Continuity (3)
Continuation 16240691 · Jan 4, 2019
Provisional Application 62613957 · Jan 5, 2018
Related Publication 20220327847A1 · Oct 13, 2022