IP Library › Granted Patent US 9,740,962
Granted Patent B2
US 9,740,962 · App. 15/133,836 · Granted Aug 22, 2017

Apparatus and method for spatially referencing images

Inventors: Alexander Thomas Starns (Redwood City, CA); Arjun Raman (Mountain View, CA); Gadi Royz (San Francisco, CA)
Assignee: Google Inc.
G06K9/6215G01C11/02G01C11/025G06T7/593G06T7/68G06T17/00H04N5/23238H04N13/02G06T2207/10012G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,740,962
App. No.
15/133,836
Granted
Aug 22, 2017
Kind
B2
Abstract

Provided is a method of spatially referencing a plurality of images captured from a plurality of different locations within an indoor space by determining the location from which the plurality of images were captured. The method may include obtaining a plurality of distance-referenced panoramas of an indoor space. The distance-referenced panoramas may each include a plurality of distance-referenced images each captured from one position in the indoor space and at a different azimuth from the other distance-referenced images, a plurality of distance measurements, and orientation indicators each indicative of the azimuth of the corresponding one of the distance-referenced images. The method may further include determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the orientation indicators and associating in memory the determined locations with the plurality of distance-referenced images captured from the determined location.

Claims (47)

1. A method of determining the location from which images were captured, the method comprising:

obtaining, by one or more processors, a plurality of distance-referenced images, each distance-referenced image captured from a different position among a plurality of positions;

determining, by the one or more processors, locations of the plurality of positions in a shared coordinate system by calculating a difference between a first distance and a second distance, wherein the first distance is measured from a first position among the plurality of positions in an orientation, and the second distance is measured from a second position among the plurality of positions in a same orientation as the first distance; and

associating, by the one or more processors, the determined locations with corresponding images of the distance-referenced images in spatially-referenced images stored by a spatially-referenced image repository.

2. The method of claim 1 , comprising:

receiving, by the one or more processors, a request for a spatially-referenced image user interface;

retrieving, by the one or more processors, based on the request, from the spatially-referenced image repository a spatially-referenced image responsive to the request;

identifying, by the one or more processors, based on a position of the spatially-referenced image, another spatially-referenced image stored in the spatially-referenced image repository;

transmitting, by the one or more processors, the spatially-referenced image and a selectable icon by which a user requests the other spatially-referenced image.

3. The method of claim 1 , wherein the locations of the plurality of positions in the shared coordinate system are relative to a point of reference, the point of reference being either absolute or relative.

4. The method of claim 1 , wherein obtaining the plurality of distance-referenced images includes placing a distance-referencing camera at each of the plurality of positions and orienting the distance-referencing camera at the same orientation during capture of each distance-referenced image.

5. The method of claim 4 , further comprising taking an orientation measurement of the distance-referencing camera at each of the plurality of positions to confirm the same orientation at each position.

6. The method of claim 1 , wherein the associating the determined locations with corresponding images of the distance-referenced images in the spatially-referenced images includes converting the plurality of distance-referenced images into corresponding spatially-referenced panoramas.

7. The method of claim 6 , further comprising preparing one or more of the spatially-referenced panoramas for presentation to a user.

8. The method of claim 1 , further comprising determining a distance between two perspectives from a pair of the spatially-referenced images.

9. The method of claim 1 , wherein the spatially-referenced images are spatially-referenced panoramas, and the method further comprises constructing a three-dimensional model of an indoor space based on a stereoscopic view of features from two of the spatially-referenced panoramas.

10. The method of claim 9 , further comprising texture mapping the two spatially-referenced panoramas onto the three-dimensional model of the indoor space.

11. A system for acquiring spatially-referenced images of indoor spaces, the camera comprising:

a camera having an optical axis;

a camera mount coupled to the camera and configured to align the optical axis with a plurality of generally radially symmetric directions, each of the radially symmetric directions approximately corresponding to an angular field of view of the camera;

a laser range finders coupled to the camera and oriented approximately parallel to the optical axis of the camera;

an orientation sensor coupled to the camera and configured to indicate which of the radially symmetric directions the camera is facing;

one or more processors communicatively connected to the laser range finders and the camera; and

non-transitory machine-readable memory communicatively connected to the one or more processors, the memory storing instructions that when executed by the one or more processors cause the one or more processors to:

obtain a plurality of distance-referenced images, each distance-referenced image captured from a different position among a plurality of positions;

determine locations of the plurality of positions in a shared coordinate system by calculating a difference between a first distance and a second distance, wherein the first distance is measured from a first position among the plurality of positions in an orientation, and the second distance is measured from a second position among the plurality of positions in a same orientation as the first distance; and

associate the determined locations with corresponding images of the distance-referenced images in spatially-referenced images stored by a spatially-referenced image repository.

12. The system of claim 11 , wherein the system further includes a tour server configured to:

receive a request for a spatially-referenced image user interface;

retrieve based on the request, from the spatially-referenced image repository a spatially-referenced image responsive to the request;

identify, based on a position of the spatially-referenced image, another spatially-referenced image stored in the spatially-referenced image repository; and

transmit the spatially-referenced image and a selectable icon by which a user requests the other spatially-referenced image.

13. The system of claim 11 , wherein the plurality of distance-referenced images are obtained by placing the camera at each of the plurality of positions and orienting the camera at the same orientation during capture of each distance-referenced image.

14. The system of claim 13 , wherein the orientation sensor is configured to take an orientation measurement of the camera at each of the plurality of positions to confirm the same orientation at each position.

15. The system of claim 11 , wherein the system further includes a tour server configured to prepare one or more of the spatially-referenced images for presentation to a user.

16. The system of claim 11 , wherein the system further includes a tour server configured to determine a distance between two perspectives from a pair of the spatially-referenced images.

17. The system of claim 11 , wherein the spatially-referenced images are spatially-referenced panoramas, and the system further includes a tour server configured to construct a three-dimensional model of an indoor space based on a stereoscopic view of features from two of the spatially-referenced panoramas.

18. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the one or more processors to perform a method of determining a location from which images were captured, the method comprising:

obtaining a plurality of distance-referenced images, each distance-referenced image captured from a different position among a plurality of positions;

determining locations of the plurality of positions in a shared coordinate system by calculating a difference between a first distance and a second distance, wherein the first distance is measured from a first position among the plurality of positions in an orientation, and the second distance is measured from a second position among the plurality of positions in a same orientation as the first distance; and

associating the determined locations with corresponding images of the distance-referenced images in spatially-referenced images stored by a spatially-referenced image repository.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the method further comprises:

receiving a request for a spatially-referenced image user interface;

retrieving based on the request, from the spatially-referenced image repository a spatially-referenced image responsive to the request;

identifying based on a position of the spatially-referenced image, another spatially-referenced image stored in the spatially-referenced image repository;

transmitting the spatially-referenced image and a selectable icon by which a user requests the other spatially-referenced image.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the spatially-referenced images are spatially-referenced panoramas, and the method further comprises constructing a three-dimensional model of an indoor space based on a stereoscopic view of features from two of the spatially-referenced panoramas.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044097/0658 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 21, 2016
From: STARNS, ALEXANDER T.; RAMAN, ARJUN; ROYZ, GADI
To: GOOGLE INC.
Reel/Frame 038340/0015 →
Continuity (3)
Continuation 13783878 · Mar 4, 2013
Provisional Application 61612518 · Mar 19, 2012
Related Publication 20160371846A1 · Dec 22, 2016