IP Library Granted Patent US 12,243,168
Granted Patent B2
US 12,243,168 · App. 18/349,911 · Granted Mar 4, 2025

Processing and/or transmitting 3D data associated with a 3D model of an interior environment

Inventors: Matthew Tschudy Bell (Palo Alto, CA); David Alan Gausebeck (Mountain View, CA); Gregory William Coombe (Mountain View, CA); Daniel Ford (Mountain View, CA)
Assignee: Matterport, Inc.
G06T17/20G06T3/40G06T2210/04G06T2210/08G06T2210/36H04N21/2362
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,243,168
App. No.
18/349,911
Granted
Mar 4, 2025
Kind
B2
Abstract

Systems and techniques for processing and/or transmitting three-dimensional (3D) data are presented. A partitioning component receives captured 3D data associated with a 3D model of an interior environment and partitions the captured 3D data into at least one data chunk associated with at least a first level of detail and a second level of detail. A data component stores 3D data including at least the first level of detail and the second level of detail for the at least one data chunk. An output component transmits a portion of data from the at least one data chunk that is associated with the first level of detail or the second level of detail to a remote client device based on information associated with the first level of detail and the second level of detail.

Claims (39)

1. A system, comprising:

a memory storing computer executable components; and

a processor configured to execute the computer executable components stored in the memory:

a partitioning component that receives a captured first three-dimensional (3D) data and a second 3D data, the first 3D data being used within a 3D model of an interior environment during a first period of time and the captured second 3D data at a time after the first period of time, the partitioning component partitioning the captured first 3D data into at least a first region and a second region based on a first object identified in the 3D model of the interior environment, the partitioning component partitioning the captured second 3D data into at least a third region and a fourth region based on a second object identified in the 3D model of the interior environment; and

an output component that transmits, to a digital device over a network, one of the first region and the second region of the first 3D data, and subsequently transmits, to the digital device, an update to one of the first region and the second region using one of the third region and the fourth region based on position data and orientation data associated with a camera view within a rendering of the 3D model.

2. The system of claim 1 , further comprising a selection component that selects one of the first region and the second region stored in a data component, the selection of one of the first region and the second region being based on a transmission order, the transmission order being based on the position data and the orientation data associated with the camera view within the rendering of the 3D model.

3. The system of claim 2 , wherein the selection component is implemented on the digital device.

4. The system of claim 2 , wherein the selection component selects the third region or the fourth region for transmission, to the digital device, based a position of a viewpoint associated with the camera view within the rendering of the 3D model.

5. The system of claim 2 , wherein the selection component selects the third region or the fourth region for transmission, to the digital device, based on a view frustum associated with the camera view within the rendering of the 3D model.

6. The system of claim 2 , wherein the selection component selects the third region or the fourth region for transmission, to the digital device, based on a history of position data associated with the rendering of the 3D model on the digital device.

7. The system of claim 2 , wherein the selection component selects the third region or the fourth region for transmission, to the digital device, based on a visibility-based culling technique.

8. The system of claim 1 , wherein the output component transmits, to the digital device, first texture data for the at least one of the first region or the second region of the first 3D data.

9. The system of claim 1 , wherein the output component transmits, to the digital device, first mesh data for the at least one of the first region and the second region at a first resolution and subsequently transmits, to the digital device, second mesh data for an other of the first region and the second region at a second resolution in response to a determination that a viewpoint associated with the rendering of the first 3D data on the digital device satisfies a defined criterion.

10. The system of claim 1 , wherein the output component transmits the update after the output component transmits the first region and the second region to the digital device and after the system receives the position data and the orientation data from the digital device.

11. A method comprising:

employing a processor that facilitates execution of computer executable instructions stored on a non-transitory computer readable medium to implement operations, comprising:

partitioning captured a first three-dimensional (3D) data to be used within a 3D model of an interior environment during a first period of time and partitioning a second 3D data at a time after the first period of time, partitioning the captured first 3D data comprising partitioning the first 3D data into at least a first region and a second region based on a first object identified in a first 3D model of an interior environment, portioning the second 3D data comprising partitioning the second 3D data into at least a third region and a fourth region based on a second object identified in the interior environment;

storing the first region, second region, third region, and fourth region;

transmitting, to a digital device, the one of the first region and the second region; and

transmitting, to the digital device, an update to one of the first region and the second region of the first 3D data with one of the third region and the fourth region of the second 3D data based on position data and orientation data associated with a camera view within a rendering of the 3D model.

12. The method of claim 11 , further comprising:

selecting one of the first region and the second region, the selection of one of the first region or the second region of the first 3D model based on a transmission order, the transmission order being based on the position data and the orientation data associated with the camera view within the rendering of the 3D model.

13. The method of claim 11 , further comprising selecting the third region or the fourth region for transmission, to the digital device, based on a position of a viewpoint associated with the camera view within the rendering of the second 3D data.

14. The method of claim 11 , wherein the transmitting, to the digital device, the update to one of the first region and the second region of the first 3D model is based on a view frustum associated with the camera view within the rendering of the first 3D model.

15. The method of claim 11 , wherein the transmitting, to the digital device, the update to one of the first region and the second region of the first 3D model is based on a history of position data associated with the rendering of the first 3D model on the digital device.

16. The method of claim 11 , wherein the transmitting, to the digital device, the update to one of the first region and the second region of the first 3D model is based on a visibility-based culling technique.

17. The method of claim 11 , further comprising receiving the position data and the orientation data from the digital device, wherein transmitting the update occurs after transmitting one of the first region and the second region to the digital device and after receiving the position data and the orientation data from the digital device.

18. A hardware server, comprising:

a memory storing computer executable components; and

a processor configured to execute the computer executable components to:

partition captured a first three-dimensional (3D) data to be used with a 3D model of an interior environment into at least a first region and a second region, based on a first object identified in the interior environment, during a first period of time and partition a second 3D data into at least a third region and fourth region, based on a second object identified in the interior environment, at a time after the first period of time;

store the first region, second region, third region, and fourth region;

transmit, to a digital device, the one of the first region and the second region to be used in the first 3D model; and

transmit, to the digital device, an update to one of the first region and the second region of the first 3D model with one of the third region and the fourth region based on position data and orientation data associated with a camera view within a rendering of the 3D model.

19. The hardware server of claim 18 , wherein the processor is configured to transmit the third region or fourth region to the digital device based on a position of a viewpoint associated with the rendering of the 3D model on the digital device.

20. The hardware server of claim 19 , wherein the processor is further configured to execute the computer executable components to:

transmit, to the digital device, geometry data for the at least one of the first region and the second region.

21. The hardware server of claim 18 , wherein the processor is further configured to execute the computer executable components to:

transmit, to the digital device, texture data for the at least one of the first region and the second region.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2025
From: MATTERPORT, LLC
To: COSTAR REALTY INFORMATION, INC.
Reel/Frame 072938/0375 →
MERGER AND CHANGE OF NAME Recorded Sep 10, 2025
From: MATTERPORT, INC.; MATRIX MERGER SUB II LLC
To: MATTERPORT, LLC
Reel/Frame 072827/0337 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2023
From: BELL, MATTHEW TSCHUDY; GAUSEBECK, DAVID ALAN; COOMBE, GREGORY WILLIAM; FORD, DANIEL
To: MATTERPORT, INC.
Reel/Frame 064521/0535 →
Continuity (6)
Continuation 17404944 · Aug 17, 2021
Continuation 16814304 · Mar 10, 2020
Continuation 16007566 · Jun 13, 2018
Continuation 15187611 · Jun 20, 2016
Continuation 14213531 · Mar 14, 2014
Related Publication 20240193867A1 · Jun 13, 2024
References Cited (48)
US 6046744A · Hoppe · 2000 [cited by applicant]
US 6931531B1 · Takahashi · 2005 [cited by applicant]
US 8471849B1 · Hickman et al. · 2013 [cited by applicant]
US 9396586B2 · Bell et al. · 2016 [cited by applicant]
US 10026224B2 · Bell · 2018 [cited by examiner]
US 10586386B2 · Bell · 2020 [cited by examiner]
US 11094117B2 · Bell · 2021 [cited by examiner]
US 11741669B2 · Bell · 2023 [cited by examiner]
US 20040203891A1 · Cole et al. · 2004 [cited by applicant]
US 20050012742A1 · Royan et al. · 2005 [cited by applicant]
US 20060061566A1 · Verma et al. · 2006 [cited by applicant]
US 20080112610A1 · Israelsen et al. · 2008 [cited by applicant]
US 20120188233A1 · Shuster et al. · 2012 [cited by applicant]
US 20120188241A1 · Ahn et al. · 2012 [cited by applicant]
US 20130156266A1 · Horii · 2013 [cited by applicant]
US 20130321593A1 · Kirk et al. · 2013 [cited by applicant]
CN 102652308A · 2012 [cited by applicant]
CN 103229041A · 2013 [cited by applicant]
“Geometric primitive,” Wikipedia, http://en.wikipedia.org/wiki/Geometric-primitive, Last accessed May 8, 20142 pages. [cited by applicant]
“Primitive,” Second Life Wiki, http://wiki.secondlife.com/wiki/Primitive, Last accessed May 8, 20144 pages. [cited by applicant]
Björkman, M., et al., “Active 3D scene segmentation and detection of unknown objects,” IEEE International Conference on Robotics and Automation, 20107 pages. [cited by applicant]
Campbell, N., et al., “Automatic 3D Object Segmentation in Multiple Views using Volumetric Graph-Cuts,” Image and Vision Computing, Jan. 201010 pages. [cited by applicant]
Caselles, V., et al. “Minimal Surfaces Based Object Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Apr. 1997, pp. 394-398, vol. 19No. 4. [cited by applicant]
Cignoni, P., et al., “A comparison of mesh simplification algorithms,” Sep. 25, 199729 pages. [cited by applicant]
Communication pursuant to Article 94(3) EPC received for EP Patent Application Serial No. 15762454.5 dated Nov. 12, 2018, 6 pages. [cited by applicant]
Extended European Search Report received for EP Patent Application Serial No. 15762454.5 dated Oct. 10, 2017, 10 pages. [cited by applicant]
First Office Action received for Chinese Patent Application Serial No. 201580023254.8 dated Aug. 20, 2018, 32 pages. [cited by applicant]
Funkhouser, et al., Management of Large Amounts of Data in Interactive Building Walkthroughs, 1992, I3D '92 Proceedings of the 1992 symposium on Interactive 3D graphicspp. 11-21. [cited by applicant]
Haumont, D., et al., “Volumetric cell-and-portal generation,” Eurographics, 2003, 10 pages, vol. 22, No. 3, Blackwell Publishers. [cited by applicant]
International Search Report and Written Opinion for International Patent Application Serial No. PCT/US2015/020439, dated Jun. 25, 201512 pages. [cited by applicant]
Lee, H., et al., “Angle-Analyzer: A Triangle-Quad Mesh Codec,” Eurographics, 2002, 10 pages, vol. 21, No. 3Blackwell Publishers. [cited by applicant]
Merchán, P., et al., “3D Complex Scenes Segmentation from a Single Range Image Using Virtual Exploration,” Iberamia, 2002, 10 pages. [cited by applicant]
Mulloni, et al., Interactive Walkthrough of Large 3D Models of Buildings on Mobile Devices, 2007, Proceedings of the twelfth international conference on 3D web technology, ser. Web3D '07. New York, NY, USA: ACM, 2007pp.… [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 14/213,531, dated Nov. 24, 2015, 55 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 15/187,611 dated Oct. 5, 2017, 31 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/404,944 dated Jan. 3, 2023, 28 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 14/213,531 dated Mar. 18, 2016, 27 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 15/187,611 dated Mar. 14, 2018, 18 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/404,944 dated Apr. 10, 2023, 10 pages. [cited by applicant]
Pfister, H., et al., “Surfels: Surface Elements as Rendering Primitives,” SIGGRAPH, 2000, pp. 335-342, New OrleansLA. [cited by applicant]
Roh, et al., An object-based 3D walk-through model for interior construction progress monitoring, 2011, Automation in Construction, 20 (1)pp. 66-75. [cited by applicant]
Sanchez, Victor, and Avideh Zakhor. “Planar 3D modeling of building interiors from point cloud data.” 2012 19th IEEE International Conference on Image Processing. IEEE, 2012. (Year: 2012). [cited by applicant]
Second Office Action received for Chinese Patent Application Serial No. 201580023254.8 dated Apr. 11, 2019, 34 pages. [cited by applicant]
Smith, Jason, G. Petrova, and Schaefer, Scott, “Progressive encoding and compression of surfaces generated from point cloud data.” Computers & Graphics 36.5 (2012): 341-348. (Year: 2012). [cited by applicant]
Teler, et al., “Streaming of Complex 3D Scenes for Remote Walkthroughs,” 2001, Eurographics, vol. 20, No. 3, pp. 1-9. [cited by applicant]
Udayan et al., “Multi-level Information Representation Based LOD Streaming for Urban Navigation”, 2013 International Symposium on Ubiquitousvirtual Reality, IEEE, Jul. 10, 2013, pp. 30-33. [cited by applicant]
Udayan et al., “Semantic levels of 1-9 information hierarchy for urban street navigation”, 2014 International Conference on Big Data and Smart Computing (BIGCOMP), IEEE, Jan. 15, 2014, pp. 235-240. [cited by applicant]
Yang, “A Progressive View-Dependent Technique for Interactive 3-D Mesh Transmission”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 14, No. 11, Nov. 2004. [cited by applicant]