IP Library Granted Patent US 10,469,745
Granted Patent B1
US 10,469,745 · App. 16/007,519 · Granted Nov 5, 2019

Dynamic 3D panoramas

Inventors: Robert Anderson (Seattle, WA); Carlos Hernandez Esteban (Kirkland, WA)
Assignee: Google LLC
H04N5/23238H04N5/23296H04N5/232935
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,469,745
App. No.
16/007,519
Granted
Nov 5, 2019
Kind
B1
Abstract

The technology relates to transitioning between panoramic imagery and the imagery from which the panoramic imagery was generated. Individual images of a portion of a scene may be combined to create a panoramic image of the entire scene. The individual images may be associated with the portion of the panoramic image which they form. The full panoramic image may then be displayed. Based on user input, a portion of the panoramic image may be zoomed-in on, and the individual image associated with that portion of the panoramic image may be displayed.

Claims (69)

1. A method for presenting images including a combined image having a first portion and a second portion, comprising:

generating, with one or more processors, the combined image by combining an altered first image depicting the first portion of the combined image and an altered second image depicting the second portion of the combined image;

displaying, with the one or more processors, the combined image where the first portion of the combined image is based at least in part of the altered first image and where the second portion of the combined image is based at least in part on the altered second image;

receiving an instruction to zoom into the display of the combined image;

zooming, with the one or more processors, into a portion of the combined image; and

augmenting the displayed combined image by:

displaying, with the one or more processors, an unaltered first image when the zoomed-in portion of the combined image is the first portion, by replacing the first portion of the combined image with the unaltered first image; and

displaying, with the one or more processors, an unaltered second image when the zoomed-in portion of the combined image is the second portion, by replacing the second portion of the combined image with the oriented unaltered second image.

2. The method of claim 1 , further comprising aligning, with the one or more processors, the unaltered first and second images with the first and second portions of the combined image, respectively, such that locations of objects in the unaltered first and second images correspond with same locations of objects in the first and second portions of the combined image.

3. The method of claim 2 , further comprising:

receiving an instruction to pan between the unaltered first and second images; and

panning, with the one or more processors, between the unaltered first and second images,

wherein panning between the unaltered first and images causes locations of objects in a foreground of the unaltered first and second images to change more than locations of objects in a background of the unaltered first and second images to convey a sense of depth to viewing the combined image.

4. The method of claim 1 , wherein receiving the instruction to zoom into the display of the combined image includes receiving input, from a user, through at least one input device.

5. The method of claim 1 , further comprising:

receiving an instruction to pan between the display of the unaltered first image and the unaltered second image; and

panning, with the one or more processors, between the display of the unaltered first image and the unaltered second image,

wherein the altered first image and the altered second image are warped together using a sparse-flow field or the altered first image and the altered second image are blended together with homography.

6. The method of claim 1 , further comprising:

receiving an instruction to zoom out of the unaltered first image or the unaltered second image;

zooming, with the one or more processors, out of the unaltered first image or the unaltered second image; and

displaying, with the one or more processors, a corresponding portion of the combined image.

7. The method of claim 1 , further comprising:

displaying, with the one or more processors, the unaltered first image when the zoomed-in portion of the combined image is the first portion;

receiving an instruction to pan across the unaltered first image;

panning, with the one or more processors, across the unaltered first image to an overlap region with the unaltered second image; and

displaying, with the one or more processors, the unaltered second image by replacing the unaltered first image with the unaltered second image.

8. The method of claim 7 , wherein replacing the first portion of the combined image with the unaltered first image or replacing second first portion of the combined image with the unaltered second image includes, as zooming increases, increasing a transparency of the first or second portion of the combined image while increasing an opacity of the unaltered first or second image.

9. The method of claim 7 , wherein replacing the unaltered first image with the unaltered second image includes increasing a transparency of the unaltered first image while increasing an opacity of the unaltered second image.

10. The method of claim 1 , further comprising:

determining, with the one or more processors, tracking points including one or more features that appear in the unaltered first and unaltered second images;

determining, with the one or more processors, a flow of the tracking points between the unaltered first and unaltered second images; and

generating, with the one or more processors, the combined image by warping the altered first and altered second images together based on the determined flow, and/or transforming the altered first and altered second images together and blending the transformed altered first and altered second images.

11. A system for presenting images including a combined image having a first portion and a second portion, comprising:

one or more computing devices; and

memory storing instructions, the instructions executable by the one or more computing devices;

wherein the instructions comprise:

generating the combined image by combining an altered first image depicting the first portion of the combined image and an altered second image depicting the second portion of the combined image;

providing for display the combined image where the first portion of the combined image is based at least in part on the altered first image and where the second portion of the combined image is based at least in part on the altered second image;

receiving an instruction to zoom into the display of the combined image;

zooming into a portion of the combined image; and

augmenting the displayed combined image by:

providing for display the unaltered first image when a zoomed-in portion of the combined image is the first portion, by replacing the first portion of the combined image with the unaltered first image; and

providing for display the unaltered second image when the zoomed-in portion of the combined image is the second portion, by replacing the second portion of the combined image with the unaltered second image.

12. The system of claim 11 , wherein the instructions further comprise aligning the unaltered first and second images with the first and second portions of the combined image, respectively, such that locations of objects in the unaltered first and second images correspond with same locations of objects in the first and second portions of the combined image.

13. The system of claim 12 , wherein the instructions further comprise:

receiving an instruction to pan between the unaltered first and second images; and

panning between the unaltered first and second images,

wherein panning between the unaltered first and images causes locations of objects in a foreground of the unaltered first and second images to change more than locations of objects in a background of the unaltered first and second images to convey a sense of depth to viewing the combined image.

14. The system of claim 11 , wherein the system is a mobile device.

15. The system of claim 11 , wherein the instructions further comprise:

receiving an instruction to pan between the display of the unaltered first image and the unaltered second image; and

panning the display of the unaltered first image and the unaltered second image,

wherein the altered first image and the altered second image are warped together using a sparse-flow field or the altered first image and the altered second image are blended together with homography.

16. The system of claim 11 , wherein the instructions further comprise:

receiving an instruction to zoom out of the unaltered first image or the unaltered second image;

zooming out of the unaltered first image or the unaltered second image; and

displaying a corresponding portion of the combined image by replacing the unaltered first image or second image with the corresponding portion of the combined image.

17. The system of claim 11 , wherein the instructions further comprise:

displaying the unaltered first image when the zoomed-in portion of the combined image is the first portion;

receiving an instruction to pan across the unaltered first image;

panning across the unaltered first image to an overlap region with the unaltered second image; and

displaying the unaltered second image by replacing the unaltered first image with the unaltered second image.

18. The system of claim 17 , wherein replacing the first portion of the combined image with the unaltered first image or replacing second first portion of the combined image with the unaltered second image includes, as zooming increases, increasing a transparency of the first or second portion of the combined image while increasing an opacity of the unaltered first or second image.

19. The system of claim 17 , wherein replacing the unaltered first image with the unaltered second image includes increasing a transparency of the unaltered first image while increasing an opacity of the unaltered second image.

20. The system of claim 11 , wherein the instructions further comprise:

determining tracking points including one or more features that appear in the unaltered first and unaltered second images;

determining a flow of the tracking points between the unaltered first and unaltered second images; and

generating the combined image by warping the altered first and altered second images together based on the determined flow, and/or transforming the altered first and altered second images together and blending the transformed altered first and altered second images.

Assignments (2)
CHANGE OF NAME Recorded Oct 16, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 047240/0461 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2018
From: ANDERSON, ROBERT; ESTEBAN, CARLOS HERNANDEZ
To: GOOGLE INC.
Reel/Frame 047166/0251 →
Continuity (1)
Continuation 14609606 · Jan 30, 2015