IP Library › Granted Patent US 11,562,492
Granted Patent B2
US 11,562,492 · App. 17/095,589 · Granted Jan 24, 2023

Physical object boundary detection techniques and systems

Inventors: Patrick Ryan Nelson (San Carlos, CA); Jinrong Xie (San Jose, CA); Randall Chall Fry (Portland, OR)
Assignee: eBay Inc.
G06T7/13G06Q10/08345G06T7/70G06T19/006G06T2207/10028G06T2207/20024G06T2207/20092G06T2210/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,562,492
App. No.
17/095,589
Granted
Jan 24, 2023
Kind
B2
Abstract

Physical object boundary detection techniques and systems are described. In one example, an augmented reality module generates three dimensional point cloud data. This data describes depths at respective points within a physical environment that includes the physical object. A physical object boundary detection module is then employed to filter the point cloud data by removing points that correspond to a ground plane. The module then performs a nearest neighbor search to locate a subset of the points within the filtered point cloud data that correspond to the physical object. Based on this subset, the module projects the subset of points onto the ground plane to generate a two-dimensional boundary. The two-dimensional boundary is then extruded based on a height determined from a point having a maximum distance from the ground plane from the filtered cloud point data.

Claims (53)

1. A method implemented by at least one computing device, the method comprising:

receiving, by the at least one computing device, motion sensing data and digital images corresponding to a physical environment including a physical object;

generating, by the at least one computing device, point cloud data based on the motion sensing data and the digital images corresponding to the physical environment including the physical object;

generating, by the at least one computing device, a three-dimensional boundary of the physical object based on the point cloud data generated based on the motion sensing data and the digital images corresponding to the physical environment including the physical object;

generating, by the at least one computing device, shipping data based on the three-dimensional boundary; and

displaying, by the at least one computing device, the shipping data in a user interface.

2. The method as described in claim 1 , wherein the user interface includes a display of a digital image of a physical environment as part of a live camera feed that includes the physical object and corresponds to the motion sensing data and the digital images corresponding to the physical environment including the physical object.

3. The method as described in claim 1 , wherein the shipping data includes an amount of packing to be used in packaging the physical object.

4. The method as described in claim 1 , wherein the shipping data is an estimated cost of shipping the physical object.

5. The method as described in claim 1 , wherein the shipping data is a packaging size or shape to be used in packaging the physical object.

6. The method as described in claim 1 , further comprising:

receiving, by the at least one computing device via the user interface, a user input to initiate generation of digital content as part of an offer for sale of the physical object via a network by a service provider;

receiving, by the at least one computing device via the user interface, another user input to initiate generation of the shipping data for the physical object as part of generation of the digital content; and

generating, by the at least one computing device, the digital content as including the shipping data.

7. The method as described in claim 1 , wherein the displaying includes displaying the shipping data as augmented reality digital content.

8. The method as described in claim 1 , wherein the point cloud data is down sampled.

9. The method as described in claim 1 , wherein the shipping data describes an estimated shipping cost based on dimensions of the three-dimensional boundary.

10. The method as described in claim 1 , wherein the user interface includes digital images as a live camera feed, the displaying including displaying the shipping data as augmented reality digital content as disposed along with the physical object.

11. The method of claim 1 further comprising generating, by the at least one computing device, a listing of the physical object, the listing including the shipping data generated, and wherein the displaying, by the at least one computing device, of the shipping data in the user interface, is as part of the listing generated.

12. A method comprising:

receiving motion sensing data and digital images corresponding to a physical environment including a physical object;

generating point cloud data based on the motion sensing data and the digital images corresponding to the physical environment including the physical object;

generating a three-dimensional boundary of the physical object based on the point cloud data generated based on the motion sensing data and the digital images corresponding to the physical environment including the physical object;

generating shipping data based on the three-dimensional boundary; and

displaying the shipping data and the three-dimensional boundary as augmented reality digital content in a live camera feed of the physical object in a user interface by a display device.

13. The method as described in claim 12 , wherein the shipping data is an estimated cost of shipping the physical object.

14. The method as described in claim 12 , wherein the shipping data includes an amount of packing to be used in packaging the physical object.

15. The method as described in claim 12 , further comprising:

receiving, via the user interface, a user input to initiate generation of digital content as part of an offer of the physical object via a network by a service provider;

receiving, via the user interface, another user input to initiate generation of the shipping data for the physical object as part of generation of the digital content; and

generating the digital content as including the shipping data.

16. A computing device comprising:

an augmented reality module implemented at least partially in hardware to receive motion sensing data and digital images corresponding to a physical environment including a physical object;

a point cloud generation module implemented at least partially in hardware to generate point cloud data based on the motion sensing data and the digital images corresponding to the physical environment including the physical object;

a physical object boundary detection module implemented at least partially in hardware to generate a three-dimensional boundary of the physical object based on the point cloud data generated based on the motion sensing data and the digital images corresponding to the physical environment including the physical object; and

a shipping module implemented at least partially in hardware to:

generate shipping data based on the three-dimensional boundary; and

display the shipping data and the three-dimensional boundary as augmented reality digital content in a live camera feed of the physical object in a user interface by a display device.

17. The computing device as described in claim 16 , wherein the shipping data is an estimated cost of shipping the physical object.

18. The computing device as described in claim 16 , wherein the physical object boundary detection module is configured to generate the three-dimensional boundary by:

computing a plurality of two-dimensional boundaries, in which each two-dimensional boundary of the plurality of two-dimensional boundaries corresponds to a respective one of a plurality of perspectives captured by the data;

selecting a two-dimensional boundary from the plurality of two-dimensional boundaries; and

generating the three-dimensional boundary having dimensions based on the selected two-dimensional boundary and a height of the physical object from the data.

19. The computing device as described in claim 16 , wherein the physical object boundary detection module is configured to generate the three-dimensional boundary by:

filtering the point cloud data data by removing points from the point cloud data that correspond to a ground plane;

receiving, via the user interface, a user input specifying a location of the physical object;

locating a subset of points within the filtered point cloud data that correspond to the physical object, the locating performed using a proximity search based on the location of the user input;

generating a two-dimensional boundary based on the located subset; and

extruding the two-dimensional boundary into the three-dimensional boundary.

20. The computing device as described in claim 16 , wherein the physical object boundary detection module is configured to generate the three-dimensional boundary by:

computing a plurality of two-dimensional boundaries, in which each two-dimensional boundary corresponds to a respective one of a plurality of perspectives captured by the point cloud data;

selecting a two-dimensional boundary from the plurality of two-dimensional boundaries; and

generating the three-dimensional boundary having dimensions based on the selected boundary and a height of the physical object from the point cloud data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 11, 2020
From: NELSON, PATRICK RYAN; XIE, JINRONG; FRY, RANDALL CHALL
To: EBAY INC.
Reel/Frame 054341/0278 →
Continuity (2)
Continuation 15984140 · May 18, 2018
Related Publication 20210065375A1 · Mar 4, 2021
Cited By (1)
US 12,367,590