IP Library Granted Patent US 10,242,280
Granted Patent B2
US 10,242,280 · App. 15/805,388 · Granted Mar 26, 2019

Determining regions of interest based on user interaction

Inventors: Kyle John Krafka (Athens, GA); Alan Sheridan (Apex, NC)
Assignee: Google LLC
G06K9/2081G06F3/04815G06K9/3208G06K9/3233G06K9/40G06T7/004G06T7/11G06T11/206G06T2207/20104
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,242,280
App. No.
15/805,388
Granted
Mar 26, 2019
Kind
B2
Abstract

A system and method provide for determining regions of interest within an image based on viewer interaction with the image. At least one image associated with a location is provided for display in a viewport, and pose data related to user interaction with the at least one image is identified. Weights are assigned to portions of the at least one image based on the pose data, the weights indicating at least a period of time the portion of the at least one image is generally at a center of the viewport. Based on the assigned weights, image regions of interest of the at least one image are determined.

Claims (37)

1. A computer-implemented method for analyzing geographical imagery, the method comprising:

selecting, by one or more computing devices, an image associated with a geographic location for presentation in a viewport of a client device display;

identifying, using the one or more computing devices, pose data related to user interaction with the image presented in the viewport;

assigning, using the one or more computing devices, weights to portions of the image based on the pose data, wherein the portions of the image are weighted according to a corresponding amount of user attention given to each respective portion of the image to assign more weight to a given portion of the image with more user attention, the weights indicating one or more factors selected from the group consisting of: a period of time a given portion of the image is at a central region of the viewport, locations of clicks in the viewport, and cursor movements in the viewport;

determining, using the one or more computing devices, one or more regions of interest of the image based on the assigned weights; and

setting, using the one or more computing devices, either a default image or a default view for the geographic location according to the one or more regions of interest, wherein responses to future requests regarding the geographic location provide either the default image or the default view.

2. The method of claim 1 , wherein assigning the weights includes increasing the weight to a given portion of the image the longer the given portion is at the central region.

3. The method of claim 1 , further comprising determining what part of the image is at the central region of the viewport by placing a 2D Gaussian distribution at a center point of the viewport and stretched the 2D Gaussian distribution to fill the viewport.

4. The method of claim 1 , wherein the weights assigned to portions of the image based the locations of clicks in the viewport and cursor movements in the viewport are less than the weights assigned to portions of the image based on the period of time the given portion of the image is at the central region of the viewport.

5. The method of claim 1 , wherein regions of the image that are viewed or interacted with more frequently are assigned higher weights than other portions of the image.

6. The method of claim 1 , wherein the user interaction includes at least one of zooming or panning the image.

7. The method of claim 6 , wherein assigning the weights to portions of the image based on the pose data includes giving more weight to a zoomed in localized area of the image.

8. The method of claim 1 , further comprising orienting the image in the viewport based on the determined one or more regions of interest.

9. The method of claim 1 , further comprising aggregating pose data of imagery from multiple client device displays.

10. The method of claim 1 , further comprising generating one or more thumbnail images for presentation with the image, each thumbnail image corresponding to a given one of the one or more regions of interest.

11. The method of claim 1 , further comprising determining supplemental content for presentation in the viewport based on the determined one or more regions of interest.

12. A system for analyzing geographical imagery, the system comprising:

a memory storing at least one image corresponding to a geographic location;

one or more processors in communication with the memory, the one or more processors configured to:

select an image associated with a geographic location for presentation in a viewport of a client device display;

identify pose data related to user interaction with the image presented in the viewport;

assign weights to portions of the image based on the pose data, wherein the portions of the image are weighted according to a corresponding amount of user attention given to each respective portion of the image to assign more weight to a given portion of the image with more user attention, the weights indicating one or more factors selected from the group consisting of: a period of time a given portion of the image is at a central region of the viewport, locations of clicks in the viewport, and cursor movements in the viewport;

determine one or more regions of interest of the image based on the assigned weights; and

set either a default image or a default view for the geographic location according to the one or more regions of interest, wherein responses to future requests regarding the geographic location provide either the default image or the default view.

13. The system of claim 12 , wherein the one or more processors are configured to assign the weights by increasing the weight to a given portion of the image the longer the given portion is at the central region.

14. The system of claim 12 , wherein regions of the image that are viewed or interacted with more frequently are assigned higher weights than other portions of the image.

15. The system of claim 12 , wherein the user interaction includes at least one of zooming or panning the image.

16. The system of claim 12 , wherein the one or more processors are configured to assign the weights to portions of the image based on the pose data by giving more weight to a zoomed in localized area of the image.

17. A non-transitory computer readable medium storing instruction executable by one or more processors to perform a method for analyzing geographical imagery, the method comprising:

selecting an image associated with a geographic location for presentation in a viewport of a client device display;

identifying pose data related to user interaction with the image presented in the viewport;

assigning weights to portions of the image based on the pose data, wherein the portions of the image are weighted according to a corresponding amount of user attention given to each respective portion of the image to assign more weight to a given portion of the image with more user attention, the weights indicating one or more factors selected from the group consisting of: a period of time a given portion of the image is at a central region of the viewport, locations of clicks in the viewport, and cursor movements in the viewport;

determining one or more regions of interest of the image based on the assigned weights; and

setting either a default image or a default view for the geographic location according to the one or more regions of interest, wherein responses to future requests regarding the geographic location provide either the default image or the default view.

18. The non-transitory computer readable medium of claim 17 , wherein the weights assigned to portions of the image based the locations of clicks in the viewport and cursor movements in the viewport are less than the weights assigned to portions of the image based on the period of time the given portion of the image is at the central region of the viewport.

19. The non-transitory computer readable medium of claim 17 , wherein regions of the image that are viewed or interacted with more frequently are assigned higher weights than other portions of the image.

20. The non-transitory computer readable medium of claim 17 , wherein the user interaction includes at least one of zooming or panning the image, and assigning the weights to portions of the image based on the pose data includes giving more weight to a zoomed in localized area of the image.

Assignments (2)
CHANGE OF NAME Recorded Nov 8, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044409/0497 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2017
From: KRAFKA, KYLE JOHN; SHERIDAN, ALAN
To: GOOGLE INC.
Reel/Frame 044055/0088 →
Continuity (3)
Continuation 15079519 · Mar 24, 2016
Provisional Application 62139135 · Mar 27, 2015
Related Publication 20180157926A1 · Jun 7, 2018