IP Library Granted Patent US 12,361,511
Granted Patent B2
US 12,361,511 · App. 16/271,383 · Granted Jul 15, 2025

Accuracy of GPS coordinates associated with image capture locations

Inventors: Dorra Larnaout (Mountain View, CA); Greg Coombe (Mountain View, CA)
Assignee: Matterport, Inc.
G06T3/153G01S19/45G06F18/2433G06N3/08G06T3/40G06T7/74G06T17/05G06V20/10G06V20/176G06T2207/10028G06T2207/10032
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,361,511
App. No.
16/271,383
Granted
Jul 15, 2025
Kind
B2
Abstract

Systems, computer-implemented methods, apparatus and/or computer program products are provided that facilitate improving the accuracy of global positioning system (GPS) coordinates of indoor photos. The disclosed subject matter further provides systems, computer-implemented methods, apparatus and/or computer program products that facilitate generating exterior photos of structures based on GPS coordinates of indoor photos.

Claims (43)

1. A system, comprising:

a processor; and

a memory containing executable instructions, the executable instructions being executable by the processor to:

determine aligned image capture positions of a camera at a first subset of capture locations with a local three-dimensional system, the first subset of capture locations being within an indoor environment and an outdoor environment;

receive location coordinates of the camera generated by a global positioning system module coupled to the camera in association with capture of images by the camera at a second subset of capture locations within the outdoor environment;

determine optimized location coordinates by adjusting the location coordinates of the camera generated by the global positioning system module based on the aligned image capture positions, resulting in the optimized location coordinates; and

assign at least one optimized location coordinate of the optimized location coordinates to related at least one capture location of the second subset of capture locations.

2. The system of claim 1 , wherein the executable instructions being further executable by the processor to:

select a subset location coordinates of the camera based on second location coordinates of the camera included in the subset meeting a defined precision criterion, wherein the location coordinates of the camera comprise the subset of second location coordinates of the camera selected from a set of first location coordinates of the camera that were generated by the global positioning system module in association with the capture of the images by the camera.

3. The system of claim 2 , wherein the defined precision criterion comprises a minimum signal-to-noise ratio of received satellite signals of the global positioning system associated with the second location coordinates of the camera.

4. The system of claim 2 , wherein the defined precision criterion comprises a maximum position dilution of precision value associated with the second location coordinates of the camera.

5. The system of claim 1 , wherein the aligned image capture positions were determined based on depth data associated with the images as captured at the related capture locations.

6. The system of claim 1 , wherein the location coordinates are in a global positioning coordinate system, and wherein the adjusting the location coordinates is a function of a transformation between the local three-dimensional system and the global positioning coordinate system.

7. The system of claim 1 , wherein the adjusting the location coordinates of the camera uses an optimization problem that optimizes an alignment between the aligned image capture positions of the first subset of capture locations and the location coordinates.

8. The system of claim 7 , wherein the optimization problem comprises a cost function that penalizes the location coordinates based on error limitations associated with the location coordinates, and wherein the error limitations are based on position dilution of precision values associated with the location coordinates.

9. The system of claim 1 , wherein the location coordinates of the camera are in a global positioning coordinate system, wherein the related capture locations are associated with orientation information indicating reference orientations of the related capture locations in the global positioning coordinate system, and determining the optimized location coordinates is based on the orientation information.

10. The system of claim 9 , wherein the adjusting the location coordinates of the camera using an optimization problem that optimizes an alignment between the aligned image capture positions of the related capture locations and respective location coordinates of the location coordinates of the camera, and the reference orientations associated with the related capture locations.

11. The system of claim 10 , wherein the optimization problem comprises a cost function that penalizes the location coordinates based on differences between the reference orientations and actual orientations of the location coordinates in the global positioning coordinate system.

12. The system of claim 9 , wherein the executable instructions being further executable by the processor to:

calibrate magnetometer data to transform the magnetometer data into orientation information, orientation information being generated based on magnetometer data generated by a magnetometer coupled to the camera.

13. The system of claim 12 , wherein the calibrate the magnetometer data based on at least one of: relative tilt of the magnetometer to a ground plane, hard iron effect, or soft iron effect.

14. A method comprising:

determining, by a system operable coupled to a processor, aligned image capture positions of a camera at a first subset of capture locations with a local three-dimensional system, the first subset of capture locations being within an indoor environment and an outdoor environment;

receiving, by the system, location coordinates of the camera generated by a global positioning system module coupled to the camera in association with capture of images by the camera at a second subset of capture locations within the outdoor environment;

determining optimized location coordinates by adjusting the location coordinates of the camera generated by the global positioning module based on the aligned image capture positions, resulting in the optimized location coordinates; and

assigning, by the system, at least one optimized location coordinate of the optimized location coordinates to related at least one capture location of the second subset of capture locations.

15. The method of claim 14 further comprising:

selecting, by the system, a subset of second location coordinates of the camera based on second location coordinates of the camera included in the subset meeting a defined precision criterion, wherein the location coordinates of the camera comprise the subset of second location coordinates of the camera selected from a set of first location coordinates of the camera that were generated by the global positioning system module in association with the capture of the images by the camera.

16. The method of claim 15 , wherein the defined precision criterion is selected from a group consisting of, a minimum signal-to-noise ratio of received satellite signals of the global positioning system associated with the second location coordinates of the camera, and a maximum position dilution of precision value associated with the second location coordinates of the camera.

17. The method of claim 14 , wherein the aligned image capture positions were determined based on depth data associated with the images as captured at the related capture locations.

18. The method of claim 14 , wherein the location coordinates of the camera are in a global positioning coordinate system, and wherein the adjusting comprises adjusting the location coordinates of the camera as a function of a transformation between the local three-dimensional system and the global positioning coordinate system.

19. The method of claim 14 , wherein the adjusting comprises adjusting the location coordinates of the camera using an optimization problem that optimizes an alignment between the aligned image capture positions of the related capture locations and location coordinates of the camera.

20. The method of claim 19 , wherein the optimization problem comprises a cost function that penalizes the location coordinates of the camera based on error limitations associated with the location coordinates, wherein the error limitations are based on position dilution of precision values associated with the location coordinates of the camera.

21. The method of claim 14 , wherein the location coordinates of the camera are in a global positioning coordinate system, wherein the related capture locations are associated with orientation information indicating reference orientations of the related capture locations in the global positioning coordinate system, and wherein determining the optimized location coordinates is based on the orientation information.

22. The method of claim 21 , wherein the adjusting comprises adjusting the location coordinates of the camera using an optimization problem that optimizes an alignment between the aligned image capture positions of the related capture locations and respective location coordinates of the location coordinates of the camera and the reference orientations associated with the related capture locations.

23. The method of claim 22 , wherein the optimization problem comprises a cost function that penalizes the location coordinates based on differences between the reference orientations and actual orientations of the location coordinates in the global positioning coordinate system.

24. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:

determining aligned image capture positions of a camera at a first subset of capture locations with a local three-dimensional system, the first subset of capture locations being within an indoor environment and an outdoor environment;

receiving location coordinates of the camera generated by a global positioning system module coupled to the camera in association with capture of images by the camera at a second subset of capture locations within the outdoor environment;

determining optimized location coordinates by adjusting the location coordinates of the camera generated by the global positioning module based on the aligned image capture positions, resulting in the optimized location coordinates; and

assigning at least one optimized location coordinate of the optimized location coordinates to related at least one capture location of the second subset of capture locations.

25. The non-transitory machine-readable storage medium of claim 24 , wherein the location coordinates of the camera are in a global positioning coordinate system, wherein the related capture locations are associated with orientation information indicating reference orientations of the related capture locations in the global positioning coordinate system, and wherein determining the optimized location coordinates is based on the orientation information.

26. The non-transitory machine-readable storage medium of claim 25 , wherein determining adjustments to the location coordinates of the camera uses an optimization problem that optimizes an alignment between the related capture locations and at least one location coordinates of the location coordinates of the camera based on the aligned image capture positions of the related capture locations and the reference orientations associated with the related capture locations.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2025
From: MATTERPORT, LLC
To: COSTAR REALTY INFORMATION, INC.
Reel/Frame 072938/0425 →
MERGER AND CHANGE OF NAME Recorded Sep 10, 2025
From: MATTERPORT, INC.; MATRIX MERGER SUB II LLC
To: MATTERPORT, LLC
Reel/Frame 072827/0559 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2019
From: LARNAOUT, DORRA; COOMBE, GREG
To: MATTERPORT, INC.
Reel/Frame 048282/0549 →
Continuity (2)
Provisional Application 62628428 · Feb 9, 2018
Related Publication 20190250283A1 · Aug 15, 2019
References Cited (22)
US 7451549B1 · Sodhi · 2008 [cited by examiner]
US 8711206B2 · Newcombe · 2014 [cited by examiner]
US 10074215B2 · Meier · 2018 [cited by examiner]
US 20130202197A1 · Reeler · 2013 [cited by applicant]
US 20140300775A1 · Fan · 2014 [cited by examiner]
US 20150094089A1 · Moeglein · 2015 [cited by examiner]
US 20150244935A1 · Meier · 2015 [cited by examiner]
US 20170102467A1 · Nielsen · 2017 [cited by examiner]
US 20180232948A1 · Hong · 2018 [cited by applicant]
US 20190306424A1 · Boyadzhiev · 2019 [cited by applicant]
US 20190371056A1 · Wetzel · 2019 [cited by applicant]
WO 2009105603A1 · 2009 [cited by applicant]
WO 2011069112A1 · 2011 [cited by applicant]
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, P. Sayd, “Generic and real-time structure from motion using local bundle adjustment,” Image and Vision Computing, vol. 27, Issue 8, 2009, pp. 1178-1193, ISSN 0262-8856,… [cited by examiner]
D. Larnaout, V. Gay-Bellile, S. Bourgeois and M. Dhome, “Vehicle 6-DoF localization based on SLAM constrained by GPS and digital elevation model information,” 2013 IEEE International Conference on Image Processing, 2013… [cited by examiner]
Dilution of precision (navigation)—Wikipedia.pdf from https://web.archive.org/web/20170930014825/https://en.wikipedia.org/wiki/Dilution_of_precision_(navigation) (Year: 2017). [cited by examiner]
Non-Final Office Action for U.S. Appl. No. 16/271,408 dated Sep. 16, 2020, 6 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 16/271,408 dated Dec. 23, 2020, 7 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/242,285 dated Feb. 6, 2023, 8 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/242,285 dated Oct. 14, 2022, 7 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 18/312,935 dated Aug. 29, 2024, 6 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 18/312,935 dated Feb. 12, 2025, 8 pages. [cited by applicant]