IP Library Granted Patent US 10,930,057
Granted Patent B2
US 10,930,057 · App. 16/540,166 · Granted Feb 23, 2021

Generating two-dimensional plan from three-dimensional image data

Inventors: Wren Dougherty (San Francisco, CA); Ho Joon Park (San Francisco, CA); Yat Choi (San Francisco, CA); David Aaron Staub (San Francisco, CA)
Assignee: Airbnb, Inc.
G06T15/20G06T2200/08G06T2210/04G06T2215/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,930,057
App. No.
16/540,166
Granted
Feb 23, 2021
Kind
B2
Abstract

Systems and methods are provided for accessing a three-dimensional (3D) image comprising a 3D mesh comprising a plurality of vertices and each vertex of the plurality of vertices having respective 3D coordinates within a space of the 3D mesh. The systems and methods further provide for generating a subset of the plurality of vertices comprising vertices that are within a specified height range and have a specified orientation. The systems and methods further provide for generating a two-dimensional (2D) grid corresponding to the 3D mesh, applying the subset of the plurality of vertices to the 2D grid, and rendering a 2D image of the space comprising an outer border corresponding to the size and shape of the 2D grid and indications of walls within the space based on the applied subset of the plurality of vertices.

Claims (69)

1. A method, comprising:

accessing, by a computing system, three-dimensional (3D) image data of a space generated from image data derived from a scan of the space via a camera of a computing device, the 3D image data comprising a 3D mesh of the space comprising a plurality of vertices and each vertex of the plurality of vertices having respective 3D coordinates within the space of the 3D mesh;

determining a largest Y coordinate value in the 3D mesh and a smallest Y coordinate value in the 3D mesh;

generating a height range comprising the difference between the largest Y coordinate value and the smallest Y coordinate value in the 3D mesh;

generating a minimum height boundary based on the smallest Y coordinate value in the 3D mesh, a specified minimum bound, and the generated height range;

generating a maximum height boundary based on the largest Y coordinate value in the 3D mesh, a specified maximum bound, and the generated height range;

generating a subset of the plurality of vertices comprising vertices that are within the generated minimum height boundary and maximum height boundary and have a vertical orientation;

generating an outer boundary for the 3D mesh using the maximum X and Y coordinates in each direction in the space of the 3D mesh;

generating a two-dimensional (2D) grid corresponding to the 3D mesh, the 2D grid having a same size and shape as the generated outer boundary of the 3D mesh;

applying the subset of the plurality of vertices to the 2D grid, each vertex of the subset of the plurality of vertices represented as a dot on the 2D grid based on an X and Y coordinate of the respective 3D coordinates; and

rendering a 2D image of the space comprising an outer border corresponding to the size and shape of the 2D grid and indications of walls within the space based on the applied subset of the plurality of vertices.

2. The method of claim 1 , further comprising:

discarding, by the computing system, vertices that are not within the generated minimum height boundary and maximum height boundary;

discarding, by the computing system, vertices that do not have a vertical orientation.

3. The method of claim 1 , wherein the indications of walls within the space are represented by a clustering of dots corresponding to each vertex of the subset of the plurality of vertices.

4. The method of claim 1 , wherein the indications of walls within the space are represented by lines corresponding to clustering of dots corresponding to each vertex of the subset of the plurality of vertices.

5. The method of claim 1 , further comprising:

causing display on a computing device of the rendered 2D image of the space.

6. The method of claim 1 , further comprising:

determining that a portion of the 2D image indicates a room; and

determining a room type for the portion of the 2D image based on objects detected in the 3D image data that are located in the portion of the 2D image.

7. The method of claim 6 , further comprising:

rendering images of one or more objects of the objects located in the portion of the 2D image, each rendered image placed in a location associated with coordinates corresponding to the respective object.

8. The method of claim 7 , further comprising:

causing display on a computing device of the rendered 2D image comprising the rendered images of the one or more objects.

9. The method of claim 1 , further comprising:

causing display of the rendered 2D image adjacent to a display a 3D representation of the space.

10. The method of claim 9 , wherein the display further comprises controls for rotating the rendered 2D image and the 3D representation of the space and wherein rotating the 2D image using the controls results in the 3D representation rotating in the same way and rotating the 3D representation results in the rendered 2D image rotating in the same way.

11. The method of claim 9 , wherein the display further comprises an option to edit the rendered 2D image, and further comprising:

receiving edits comprising lines representing walls or designated spaces within the space;

generating a 2D floor plan comprising the lines representing the walls or designated spaces within the space;

receiving a request to upload the generated 2D floor plan to a listing for a place in an online marketplace; and

adding the generated 2D floor plan to the listing for the place in an online marketplace.

12. The method of claim 11 , wherein the generated 2D floor plan is displayed concurrently with each image of the space, and a room of the generated 2D floor plan is highlighted when an image corresponding to the room is displayed.

13. A computing system comprising:

a memory that stores instructions; and

one or more processors configured by the instructions to perform operations comprising:

accessing three-dimensional (3D) image data of a space generated from image data derived from a scan of the space via a camera of a computing device, the 3D image data comprising a 3D mesh of the space comprising a plurality of vertices and each vertex of the plurality of vertices having respective 3D coordinates within the space of the 3D mesh;

determining a largest Y coordinate value in the 3D mesh and a smallest Y coordinate value in the 3D mesh;

generating a height range comprising the difference between the largest Y coordinate value and the smallest Y coordinate value in the 3D mesh;

generating a minimum height boundary based on the smallest Y coordinate value in the 3D mesh, a specified minimum bound, and the generated height range;

generating a maximum height boundary based on the largest Y coordinate value in the 3D mesh, a specified maximum bound, and the generated height range;

generating a subset of the plurality of vertices comprising vertices that are within the generated minimum height boundary and maximum height boundary and have a vertical orientation;

generating an outer boundary for the 3D mesh using the maximum X and Y coordinates in each direction in the space of the 3D mesh;

generating a two-dimensional (2D) grid corresponding to the 3D mesh, the 2D grid having a same size and shape as the generated outer boundary of the 3D mesh;

applying the subset of the plurality of vertices to the 2D grid, each vertex of the subset of the plurality of vertices represented as a dot on the 2D grid based on an X and Y coordinate of the respective 3D coordinates; and

rendering a 2D image of the space comprising an outer border corresponding to the size and shape of the 2D grid and indications of walls within the space based on the applied subset of the plurality of vertices.

14. The computing system of claim 13 , wherein the indications of walls within the space are represented by a clustering of dots corresponding to each vertex of the subset of the plurality of vertices.

15. The computing system of claim 13 , wherein the indications of walls within the space are represented by lines corresponding to a clustering of dots corresponding to each vertex of the subset of the plurality of vertices.

16. The computing system of claim 13 , the operations further comprising:

causing display on a computing device of the rendered 2D image of the space.

17. The computing system of claim 13 , the operations further comprising:

determining that a portion of the 2D image indicates a room; and

determining a room type for the portion of the 2D image based on objects detected in the 3D image data that are located in the portion of the 2D image.

18. The computing system of claim 17 , the operations further comprising:

rendering images of one or more objects of the objects located in the portion of the 2D image, each rendered image placed in a location associated with coordinates corresponding to the respective object.

19. The computing device of claim 18 , the operations further comprising:

causing display on a computing device of the rendered 2D image comprising the rendered images of the one or more objects.

20. A non-transitory computer-readable medium comprising instructions stored thereon that are executable by at least one processor to cause a computing device associated with a first data owner to perform operations comprising:

accessing three-dimensional (3D) image data of a space generated from image data derived from a scan of the space via a camera of a computing device, the 3D image data comprising a 3D mesh of the space comprising a plurality of vertices and each vertex of the plurality of vertices having respective 3D coordinates within the space of the 3D mesh;

determining a largest Y coordinate value in the 3D mesh and a smallest Y coordinate value in the 3D mesh;

generating a height range comprising the difference between the largest Y coordinate value and the smallest Y coordinate value in the 3D mesh;

generating a minimum height boundary based on the smallest Y coordinate value in the 3D mesh, a specified minimum bound, and the generated height range;

generating a maximum height boundary based on the largest Y coordinate value in the 3D mesh, a specified maximum bound, and the generated height range;

generating a subset of the plurality of vertices comprising vertices that are within the generated minimum height boundary and maximum height boundary and have a vertical orientation;

generating an outer boundary for the 3D mesh using the maximum X and Y coordinates in each direction in the space of the 3D mesh;

generating a two-dimensional (2D) grid corresponding to the 3D mesh, the 2D grid having a same size and shape as the generated outer boundary of the 3D mesh;

applying the subset of the plurality of vertices to the 2D grid, each vertex of the subset of the plurality of vertices represented as a dot on the 2D grid based on an X and Y coordinate of the respective 3D coordinates; and

rendering a 2D image of the space comprising an outer border corresponding to the size and shape of the 2D grid and indications of walls within the space based on the applied subset of the plurality of vertices.

Assignments (8)
RELEASE (REEL 054586 / FRAME 0033) Recorded Nov 1, 2022
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: AIRBNB, INC.
Reel/Frame 061825/0910 →
RELEASE OF SECURITY INTEREST IN PATENTS Recorded Apr 21, 2021
From: TOP IV TALENTS, LLC
To: AIRBNB, INC.
Reel/Frame 055997/0907 →
RELEASE OF SECURITY INTEREST Recorded Mar 8, 2021
From: CORTLAND CAPITAL MARKET SERVICES LLC
To: AIRBNB, INC.
Reel/Frame 055527/0531 →
SECURITY AGREEMENT Recorded Nov 19, 2020
From: AIRBNB, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 054586/0033 →
FIRST LIEN SECURITY AGREEMENT Recorded Apr 21, 2020
From: AIRBNB, INC.
To: CORTLAND CAPITAL MARKET SERVICES LLC
Reel/Frame 052456/0036 →
SECOND LIEN PATENT SECURITY AGREEMENT Recorded Apr 17, 2020
From: AIRBNB, INC.
To: TOP IV TALENTS, LLC, AS COLLATERAL AGENT
Reel/Frame 052433/0416 →
CORRECTIVE ASSIGNMENT TO CORRECT THE LAST INVENTOR'S NAME PREVIOUSLY RECORDED AT REEL: 50046 FRAME: 719. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 20, 2019
From: DOUGHERTY, WREN; PARK, HO JOON; CHOI, YAT; STAUB, DAVID AARON
To: AIRBNB, INC.
Reel/Frame 050107/0396 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2019
From: DOUGHERTY, WREN; PARK, HO JOON; CHOI, YAT; STAUB, DAVID AARONG
To: AIRBNB, INC.
Reel/Frame 050046/0719 →
Cited By (1)
US 12,693,728