IP Library Granted Patent US 12,315,101
Granted Patent B2
US 12,315,101 · App. 18/312,935 · Granted May 27, 2025

Selecting exterior images of a structure based on capture positions of indoor images associated with the structure

Inventors: Gunnar Hovden (Los Gatos, CA); Scott Adams (Redwood City, CA)
Assignee: Matterport, Inc.
G06T3/153G01S19/45G06F18/2433G06N3/08G06T3/40G06T7/74G06T17/05G06V20/10G06V20/176G06T2207/10028G06T2207/10032
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,315,101
App. No.
18/312,935
Granted
May 27, 2025
Kind
B2
Abstract

Systems, computer-implemented methods, apparatus and/or computer program products are provided that facilitate improving the accuracy of global positioning system (GPS) coordinates of indoor photos. The disclosed subject matter further provides systems, computer-implemented methods, apparatus and/or computer program products that facilitate generating exterior photos of structures based on GPS coordinates of indoor photos.

Claims (36)

1. A system, comprising:

a memory that stores computer executable components; and

a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:

an identification component that identifies at least one exterior image of a building of a plurality of exterior images, wherein each of the plurality of exterior images is associated with external coordinates of an external capture position and a field of view extending from the external coordinates of that external capture position, the external capture position being a location where each of the plurality of exterior images were captured, and wherein the building is associated with interior scan location information corresponding to interior capture locations, relative to a global positioning coordinate system, of interior images captured inside the building; and

an exterior perspective component that determines the at least one exterior image of the building based on the field of view associated with the at least one exterior image comprising determining an optimal view of an exterior of the building based on a field of view that maximizes inclusion of the interior capture locations.

2. The system of claim 1 , wherein the computer executable components further comprise:

a cropping component that crops the at least one exterior image based on a portion to generate an image of the exterior of the building from the field of view that maximizes inclusion of the interior capture locations.

3. The system of claim 2 , wherein the computer executable components further comprise:

an image leveling component that rotates the image relative to a ground plane based on a gravity vector associated with the at least one exterior image.

4. The system of claim 1 , wherein the exterior perspective component determines a portion of the at least one exterior image of the building based on a second field of view that is less than the field of view of the at least one exterior image and maximizes the inclusion of the interior capture locations.

5. The system of claim 1 , wherein the exterior perspective component determines the field of view that maximizes inclusion of the interior capture locations based on a minimum field of view that maximizes the inclusion of the interior capture locations.

6. The system of claim 5 , wherein the at least one exterior image comprises building image data corresponding to an exterior of the building and environmental image data corresponding to an environment around the exterior of the building, and wherein the exterior perspective component determines the field of view that maximizes inclusion of the interior capture locations based on the minimum field of view and an optimal ratio of the building image data to the environmental image data.

7. The system of claim 6 , wherein the exterior perspective component determines the optimal ratio using a machine learning model.

8. The system of claim 1 , wherein the at least one exterior image comprises a panoramic image of an environment from the capture location with a field of view up to 360° horizontally and vertically and wherein the exterior perspective component determines a portion of the at least one exterior image of the building comprises the optimal view.

9. The system of claim 8 , wherein a size of a portion is based on a smaller field of view relative to the field of view that maximizes the inclusion of the interior capture locations.

10. The system of claim 8 , wherein a portion is based on an orientation determined for the field of view that maximizes the inclusion of the interior capture locations.

11. The system of claim 1 , wherein the computer executable components further comprise:

an image classification component that determines whether image data associated with the building corresponds to an exterior image of the building or an interior image of the building, and wherein the identification component identifies the at least one exterior image for processing by the exterior perspective component based on a determination that the at least one exterior image corresponds to the exterior image of the building.

12. The system of claim 11 , wherein the image classification component determines whether the image data associated with the building corresponds to the exterior image of the building or the interior image of the building using a machine learning classification model.

13. The system of claim 11 , wherein the image classification component determines whether the image data associated with the building corresponds to the exterior image of the building or the interior image of the building based on a quantity or quality of depth data associated with the image data.

14. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:

receiving a plurality of exterior images of a building, each of the plurality of exterior images is associated with external coordinates of an external capture position and a field of view extending from the external coordinates of that external capture position, the external capture position being a location where each of the plurality of exterior images were captured, wherein the building is associated with interior scan location information corresponding to interior capture locations, relative to a global positioning coordinate system, of interior images captured inside the building; and

identifying at least one exterior image of the building based on the field of view associated with the at least one exterior image, the identifying comprising determining an optimal view of the exterior of the building based on a field of view that maximizes inclusion of the interior capture locations.

15. The non-transitory machine-readable storage medium of claim 14 , the operations further comprising:

cropping the at least one exterior image based on a portion to generate an image of the exterior of the building from a perspective.

16. The non-transitory machine-readable storage medium of claim 14 , the operations further comprising:

rotating the image relative to a ground plane based on a gravity vector associated with the at least one exterior image.

17. The non-transitory machine-readable storage medium of claim 14 , the operations further comprising:

determining a portion of the at least one exterior image of the building based on a second field of view that is less than the field of view of the at least one exterior image and maximizes the inclusion of the interior capture locations.

18. The non-transitory machine-readable storage medium of claim 14 , the operations further comprising:

determining, by the system, an optimal field of view for a perspective, including determining a smaller field of view relative to the field of view that maximizes the inclusion of the interior capture locations.

19. The non-transitory machine-readable storage medium of claim 14 , the operations further comprising:

determining, by the system, an orientation for a perspective that maximizes the inclusion of the interior capture locations.

20. A method comprising:

receiving a plurality of exterior images of a building, each of the plurality of exterior images is associated with external coordinates of an external capture position and a field of view extending from the external coordinates of that external capture position, the external capture position being a location where each of the plurality of exterior images was captured, wherein the building is associated with interior scan location information corresponding to interior capture locations, relative to a global positioning coordinate system, of interior images captured inside the building; and

identifying at least one exterior image of the building based on the field of view associated with the at least one exterior image, the identifying comprising determining an optimal view of the exterior of the building based on a field of view that maximizes inclusion of the interior capture locations.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2025
From: MATTERPORT, LLC
To: COSTAR REALTY INFORMATION, INC.
Reel/Frame 072938/0425 →
MERGER AND CHANGE OF NAME Recorded Sep 10, 2025
From: MATTERPORT, INC.; MATRIX MERGER SUB II LLC
To: MATTERPORT, LLC
Reel/Frame 072827/0559 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 19, 2023
From: HOVDEN, GUNNAR; ADAMS, SCOTT
To: MATTERPORT, INC.
Reel/Frame 063987/0837 →
Continuity (4)
Continuation 17242285 · Apr 27, 2021
Continuation 16271408 · Feb 8, 2019
Provisional Application 62628428 · Feb 9, 2018
Related Publication 20230274385A1 · Aug 31, 2023
References Cited (27)
US 7451549B1 · Sodhi et al. · 2008 [cited by applicant]
US 8711206B2 · Newcombe et al. · 2014 [cited by applicant]
US 10074215B2 · Meier et al. · 2018 [cited by applicant]
US 20130202197A1 · Reeler et al. · 2013 [cited by applicant]
US 20140300775A1 · Fan et al. · 2014 [cited by applicant]
US 20150094089A1 · Moeglein et al. · 2015 [cited by applicant]
US 20150244935A1 · Meier et al. · 2015 [cited by applicant]
US 20170102467A1 · Nielsen et al. · 2017 [cited by applicant]
US 20180232948A1 · Hong et al. · 2018 [cited by applicant]
US 20190250283A1 · Larnaout et al. · 2019 [cited by applicant]
US 20190251352A1 · Hovden et al. · 2019 [cited by applicant]
US 20190306424A1 · Boyadzhiev et al. · 2019 [cited by applicant]
US 20190371056A1 · Wetzel et al. · 2019 [cited by applicant]
WO WO2009105603A1 · 2009 [cited by examiner]
WO WO2011069112A1 · 2011 [cited by examiner]
Non-Final Office Action for U.S. Appl. No. 16/271,383 dated Jan. 12, 2024, 35 pages. [cited by applicant]
D. Larnaout, V. Gay-Bellile, S. Bourgeois and M. Dhome, “Vehicle 6-DoF localization based on SLAM constrained by GPS and digital elevation model information,” 2013 IEEE International Conference on Image Processing, 2013… [cited by applicant]
Dilution of precision (navigation)—Wikipedia.pdf' from https://web.archive .org/web/20170930014825/https://en.wikipedia.org/wiki/Dilution_of_precision_(navigation) (Year: 2017). [cited by applicant]
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, P. Sayd, “Generic and real-time structure from motion using local bundle adjustment,” Image and Vision Computing, vol. 27, Issue 8, 2009, pp. 1178-1193, ISSN 0262-8856,… [cited by applicant]
Final Office Action for U.S. Appl. No. 16/271,383 dated May 23, 2023, 29 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/271,383 dated Nov. 14, 2022, 30 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/271,408 dated Sep. 16, 2020, 6 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 16/271,408 dated Dec. 23, 2020, 7 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/242,285 dated Feb. 6, 2023, 8 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/242,285 dated Oct. 14, 2022, 7 pages. [cited by applicant]
Final Office Action for U.S. Appl. No. 16/271,383 dated Aug. 30, 2024, 36 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 16/271,383 dated Mar. 26, 2025, 19 pages. [cited by applicant]