IP Library › Granted Patent US 10,262,231
Granted Patent B2
US 10,262,231 · App. 15/642,827 · Granted Apr 16, 2019

Apparatus and method for spatially referencing images

Inventors: Alexander Thomas Starns (Redwood City, CA); Arjun Raman (Mountain View, CA); Gadi Royz (San Francisco, CA)
Assignee: Google LLC
G06K9/6215G01C11/02G01C11/025G06T7/593G06T7/68G06T17/00H04N5/23238H04N13/20G06T2207/10012G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,262,231
App. No.
15/642,827
Granted
Apr 16, 2019
Kind
B2
Abstract

Provided is a method of spatially referencing a plurality of images captured from a plurality of different locations within an indoor space by determining the location from which the plurality of images was captured. The method may include obtaining a plurality of distance-referenced panoramas of an indoor space. The distance-referenced panoramas may each include a plurality of distance-referenced images each captured from one position in the indoor space and at a different azimuth from the other distance-referenced images, a plurality of distance measurements, and orientation indicators each indicative of the azimuth of the corresponding one of the distance-referenced images. The method may further include determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the orientation indicators and associating in memory the determined locations with the plurality of distance-referenced images captured from the determined location.

Claims (38)

1. A system for providing spatially-referenced panoramic images of indoor spaces in a virtual tour, the system comprising:

a spatial referencer including one or more processing devices configured to receive a plurality of distance-referenced panoramic images of an indoor space and to convert the plurality of distance-referenced panoramic images into a set of corresponding spatially-referenced panoramic images associated with the indoor space, each of the plurality of distance-referenced panoramic images being captured from a different position within the indoor space, and each of the plurality of distance-referenced panoramic images having one or more distance measurements and an orientation indicator associated therewith;

a spatially-referenced tour repository operatively coupled to the spatial referencer, the spatially-referenced tour repository being configured to store the set of spatially-referenced panoramic images associated with the indoor space in memory; and

a tour web server including one or more processing devices operatively coupled to the spatially-referenced tour repository, the tour web server being configured to receive a request from a client device in response to a user interaction event, to select one or more of the spatially-referenced panoramic images of the set in response to the request, and to transmit the selected spatially-referenced panoramic images to the client device for presentation in a virtual tour on a display interface thereof,

wherein selected ones of the spatially-referenced panoramic images of the set are texture-mapped onto a three-dimensional model of the indoor space in order to simulate views from additional positions from which panoramas were not captured when full 360-degree panoramas are not used.

2. The system of claim 1 , wherein the one or more processing devices of the tour web server are configured to generate one or more icons for overlay on a presently viewed panoramic image, each of the one or more icons depicting a selected panoramic location of the indoor space.

3. The system of claim 2 , wherein a size of each icon is selected based on a distance between a presently viewed position and a location of a corresponding panoramic image.

4. The system of claim 3 , wherein the size of each icon is proportional to the distance between the presently viewed position and the location of the corresponding panoramic image.

5. The system of claim 2 , wherein regions of the display interface in which the one or more icons are to be overlaid correspond to one or more spaces in the display interface that identify a request to view a corresponding panoramic image associated with a respective one of the icons.

6. The system of claim 1 , wherein the one or more processing devices of the tour web server are configured to select the one or more of the spatially-referenced panoramic images to enable virtual navigation through the indoor space and to view the indoor space from different perspectives while maintaining a sense of a relative location of positions from which the indoor space is to be viewed.

7. The system of claim 1 , wherein the one or more distance measurements associated with a given one of the plurality of distance-referenced panoramic images indicate a distance between a position of the given distance-referenced panoramic image and an object depicted within the given distance-referenced panoramic image.

8. The system of claim 1 , wherein the system is configured to construct the three-dimensional model of the indoor space based on a stereoscopic view of features from two different spatially-referenced panoramic images of the set.

9. The system of claim 1 , wherein the tour web server is further configured to determine a distance between two perspectives from a pair of the spatially-referenced panoramic images of the set.

10. A method for providing spatially-referenced panoramic images of indoor spaces in a virtual tour, the method comprising:

receiving, by one or more processing devices, a plurality of distance-referenced panoramic images of an indoor space;

converting, by the one or more processing devices, the plurality of distance-referenced panoramic images into a set of corresponding spatially-referenced panoramic images associated with the indoor space, each of the plurality of distance-referenced panoramic images being captured from a different position within the indoor space, and each of the plurality of distance-referenced panoramic images having one or more distance measurements and an orientation indicator associated therewith;

storing the set of spatially-referenced panoramic images associated with the indoor space in memory;

receiving a request from a client device in response to a user interaction event;

selecting, by the one or more processing devices, one or more of the spatially-referenced panoramic images of the set in response to the request; and

transmitting the selected spatially-referenced panoramic images to the client device for presentation in a virtual tour on a display interface thereof,

wherein selected ones of the spatially-referenced panoramic images of the set are texture-mapped onto a three-dimensional model of the indoor space in order to simulate views from additional positions from which panoramas were not captured when full 360-degree panoramas are not used.

11. The method of claim 10 , further comprising generating one or more icons for overlay on a presently viewed panoramic image, each of the one or more icons depicting a selected panoramic location of the indoor space.

12. The method of claim 11 , wherein a size of each icon is selected based on a distance between a presently viewed position and a location of a corresponding panoramic image.

13. The method of claim 12 , wherein the size of each icon is proportional to the distance between the presently viewed position and the location of the corresponding panoramic image.

14. The method of claim 10 , wherein the one or more distance measurements associated with a given one of the plurality of distance-referenced panoramic images indicate a distance between a position of the given distance-referenced panoramic image and an object depicted within the given distance-referenced panoramic image.

15. The method of claim 10 , further comprising constructing the three-dimensional model of the indoor space based on a stereoscopic view of features from two different spatially-referenced panoramic images of the set.

16. The method of claim 10 , further comprising determining a distance between two perspectives from a pair of the spatially-referenced panoramic images of the set.

17. A non-transitory computer-readable recording medium storing instructions thereon, the instructions, when executed by one or more processing devices, causing the one or more processing devices to perform a method for providing spatially-referenced panoramic images of indoor spaces in a virtual tour, the method comprising:

receiving a plurality of distance-referenced panoramic images of an indoor space;

converting the plurality of distance-referenced panoramic images into a set of corresponding spatially-referenced panoramic images associated with the indoor space, each of the plurality of distance-referenced panoramic images being captured from a different position within the indoor space, and each of the plurality of distance-referenced panoramic images having one or more distance measurements and an orientation indicator associated therewith;

storing the set of spatially-referenced panoramic images associated with the indoor space in memory;

receiving a request from a client device in response to a user interaction event;

selecting one or more of the spatially-referenced panoramic images of the set in response to the request; and

transmitting the selected spatially-referenced panoramic images to the client device for presentation in a virtual tour on a display interface thereof,

wherein selected ones of the spatially-referenced panoramic images of the set are texture-mapped onto a three-dimensional model of the indoor space in order to simulate views from additional positions from which panoramas were not captured when full 360-degree panoramas are not used.

18. The non-transitory computer-readable recording medium of claim 17 , wherein the method further comprises constructing the three-dimensional model of the indoor space based on a stereoscopic view of features from two different spatially-referenced panoramic images of the set.

19. The non-transitory computer-readable recording medium of claim 17 , wherein the method further comprises generating one or more icons for overlay on a presently viewed panoramic image, each of the one or more icons depicting a selected panoramic location of the indoor space.

20. The non-transitory computer-readable recording medium of claim 19 , wherein a size of each icon is selected based on a distance between a presently viewed position and a location of a corresponding panoramic image.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 7, 2017
From: STARNS, ALEXANDER T.; RAMAN, ARJUN; ROYZ, GADI
To: GOOGLE INC.
Reel/Frame 042931/0175 →
Continuity (4)
Continuation 15133836 · Apr 20, 2016
Continuation 13783878 · Mar 4, 2013
Provisional Application 61612518 · Mar 19, 2012
Related Publication 20170372172A1 · Dec 28, 2017