IP Library Granted Patent US 11,809,487
Granted Patent B2
US 11,809,487 · App. 17/320,648 · Granted Nov 7, 2023

Displaying objects based on a plurality of models

Inventor: Janne Matias Kontkanen (San Francisco, CA)
Assignee: Google LLC
G06F16/5838G06T15/00G06T15/04G06T15/20G06T15/205G06T15/503G06T17/00G06V20/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,809,487
App. No.
17/320,648
Granted
Nov 7, 2023
Kind
B2
Abstract

A system and method is provided for displaying surfaces of an object from a vantage point different from the vantage point from which imagery of the object was captured. In some aspects, imagery may be generated for display by combining visual characteristics from multiple source images and applying greater weight to the visual characteristics of some of the source images relative to the other source images. The weight may be based on the orientation of the surface relative to the location from which the image was captured and the location from which the object will be displayed.

Claims (55)

1. A method of providing an image for display, the method comprising:

accessing a first model of an object based on a first image of the object captured from a first vantage point;

accessing a second model of the object based on a second image of the object captured from a second vantage point;

receiving a request for a third image of the object from a requested vantage point different from the first vantage point and the second vantage point;

generating the third image from the first model and the requested vantage point, and from the second model and the requested vantage point; and

providing the third image for display.

2. The method of claim 1 , further comprising:

determining a first visual characteristic from the first model and the requested vantage point; and

determining a second visual characteristic from the second model and the requested vantage point,

wherein generating the third image includes generating the third image to have a third visual characteristic that is based on the first visual characteristic and the second visual characteristic.

3. The method of claim 2 , further comprising determining an orientation associated with the requested vantage point, wherein the first visual characteristic of the first model is based on the orientation, and the second visual characteristic of the second model is based on the orientation.

4. The method of claim 2 , wherein the third visual characteristic of the third image is based on a first orientation associated with the first model and a second orientation associated with the second model.

5. The method of claim 2 , wherein the third visual characteristic of the third image is based on first location information associated with the first model and second location information associated with the second model.

6. The method of claim 2 , wherein:

the first visual characteristic is based on a first ratio of a first texel count to a first pixel count generated by projecting image data captured from the first vantage point onto the first model; and

the second visual characteristic is based on a second ratio of a second texel count to a second pixel count generated by projecting image data captured from the second vantage point onto the second model.

7. The method of claim 2 , wherein:

the first visual characteristic is a first texel generated by projecting image data captured from the first vantage point onto the first model; and

the second visual characteristic is a second texel generated by projecting image data captured from the second vantage point onto the second model.

8. The method of claim 2 , wherein the first visual characteristic and second visual characteristics are associated with colors, and wherein the third visual characteristic of the third image is based on blending a first color associated with the first visual characteristic and blending a second color associated with the second visual characteristic.

9. The method of claim 2 , wherein the object has a surface defining a plane, the method further comprises selecting a point on the plane, and the first visual characteristic and the second visual characteristic are determined based on a respective angle between the plane and a line extending from the first vantage point and the second vantage point to the selected point on the plane.

10. A system comprising:

one or more computing devices; and

memory storing instructions, the instructions being executable by the one or more computing devices;

wherein the instructions comprise:

accessing a first model of an object based on a first image of the object captured from a first vantage point;

accessing a second model of the object based on a second image of the object captured from a second vantage point;

receiving a request for a third image of the object from a requested vantage point different from the first vantage point and the second vantage point;

generating the third image from the first model and the requested vantage point, and from the second model and the requested vantage point; and

providing the third image for display.

11. The system of claim 10 , further comprising:

determining a first visual characteristic from the first model and the requested vantage point; and

determining a second visual characteristic from the second model and the requested vantage point,

wherein generating the third image includes generating the third image to have a third visual characteristic that is based on the first visual characteristic and the second visual characteristic.

12. The system of claim 11 , further comprising determining an orientation associated with the requested vantage point, wherein the first visual characteristic of the first model is based on the orientation, and the second visual characteristic of the second model is based on the orientation.

13. The system of claim 11 , wherein the third visual characteristic of the third image is based on a first orientation associated with the first model and a second orientation associated with the second model.

14. The system of claim 11 , wherein the third visual characteristic of the third image is based on first location information associated with the first model and second location information associated with the second model.

15. The system of claim 11 , wherein the first visual characteristic and second visual characteristics are associated with colors, and wherein the third visual characteristic of the third image is based on blending a first color associated with the first visual characteristic and blending a second color associated with the second visual characteristic.

16. The system of claim 11 , wherein:

the first visual characteristic is a first texel generated by projecting image data captured from the first vantage point onto the first model; and

the second visual characteristic is a second texel generated by projecting image data captured from the second vantage point onto the second model.

17. A non-transitory computing-device readable storage medium on which computing-device readable instructions of a program are stored, the instructions, when executed by one or more computing devices, causing the one or more computing devices to perform a method, the method comprising:

accessing a first model of an object based on a first image of the object captured from a first vantage point;

accessing a second model of the object based on a second image of the object captured from a second vantage point;

receiving a request for a third image of the object from a requested vantage point different from the first vantage point and the second vantage point;

generating the third image from the first model and the requested vantage point, and from the second model and the requested vantage point; and

providing the third image for display.

18. The medium of claim 17 , further comprising:

determining a first visual characteristic from the first model and the requested vantage point; and

determining a second visual characteristic from the second model and the requested vantage point,

wherein generating the third image includes generating the third image to have a third visual characteristic that is based on the first visual characteristic and the second visual characteristic.

19. The medium of claim 18 , further comprising determining an orientation associated with the requested vantage point, wherein the first visual characteristic of the first model is based on the orientation, and the second visual characteristic of the second model is based on the orientation.

20. The medium of claim 18 , wherein:

the first visual characteristic is a first texel generated by projecting image data captured from the first vantage point onto the first model; and

the second visual characteristic is a second texel generated by projecting image data captured from the second vantage point onto the second model.

Assignments (2)
CHANGE OF NAME Recorded Jul 14, 2021
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 056874/0911 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2021
From: KONTKANEN, JANNE MATIAS
To: GOOGLE INC.
Reel/Frame 056838/0186 →
Continuity (4)
Continuation 16575941 · Sep 19, 2019
Continuation 15692548 · Aug 31, 2017
Continuation 14877368 · Oct 7, 2015
Related Publication 20210271706A1 · Sep 2, 2021
Cited By (1)
US 12,339,897