IP Library › Granted Patent US 10,679,404
Granted Patent B2
US 10,679,404 · App. 16/150,813 · Granted Jun 9, 2020

Shadows for inserted content

Inventor: Ivan Neulander (Los Angeles, CA)
Assignee: Google LLC
G06T15/60G02B27/0172G06T7/70G06T13/40G06T15/005G06T17/20G06T19/006G02B2027/0138G06T2210/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,679,404
App. No.
16/150,813
Granted
Jun 9, 2020
Kind
B2
Abstract

Systems and methods for generating shadows for inserted content are provided. The inserted content may include augmented reality content that is inserted into an image of a physical space. An example includes determining a location to insert content within an image. The content may include a polygonal mesh defined in part by a skeleton that has a plurality of joints. Examples may further include selecting a plurality of selected joints form the plurality of joints. Examples may also include generating a shadow polygon based on the content and determining shadow contributions values for the plurality of selected joints for pixels of the shadow polygon. Examples may also include combining the shadow contribution values from the selected joints to generate shadow magnitude values for the pixels, rendering the shadow polygon using the shadow magnitude values, and overlaying the inserted content on the rendered shadow polygon.

Claims (32)

1. A method comprising:

determining a location to insert content within an image, the content including a polygonal mesh defined in part by a skeleton that has a plurality of joints;

selecting a plurality of selected joints form the plurality of joints;

generating a shadow polygon;

determining shadow contributions values for the plurality of selected joints for pixels of the shadow polygon;

combining the shadow contribution values from the selected joints to generate shadow magnitude values for the pixels;

rendering the shadow polygon using the shadow magnitude values; and

overlaying the inserted content on the rendered shadow polygon.

2. The method of claim 1 , wherein the selecting the plurality of selected joints comprises selecting joints based on a data structure associated with the content.

3. The method of claim 1 , wherein the selecting the plurality of selected joints includes selecting joints based on distance to a surface plane that is identified based on the image.

4. The method of claim 3 , wherein the selecting the plurality of selected joints includes selecting a threshold percentage of the plurality of joints that are closest to the surface plane.

5. The method of claim 3 , further comprising assigning fade-out values to the plurality of selected joints based on distance to the surface plane.

6. The method of claim 1 , wherein the shadow contribution value for a specific joint from the plurality of selected joints for a specific pixel of the shadow polygon is based on a distance from the specific pixel to the specific joint.

7. The method of claim 6 , wherein the shadow contribution value for the specific joint for the specific pixel is based on an overhead angle between the specific pixel and the specific selected joint.

8. The method of claim 1 , wherein the combining the shadow contribution values includes selecting a maximum shadow contribution value.

9. The method of claim 1 , further comprising combining the rendered shadow polygon with an additional shadow polygon.

10. A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by at least one processor, are configured to cause a computing system to at least:

capture an image with a camera assembly;

identify a surface plane within the image;

determine a location to insert content, the content including a polygonal mesh defined in part by a skeleton that has a plurality of joints;

select a plurality of selected joints form the plurality of joints;

assign fade-out values to the plurality of selected joints;

generate a shadow polygon based on the content;

determine shadow contributions values for the plurality of selected joints for pixels of the shadow polygon;

combine the shadow contribution values from the selected joints to generate shadow magnitude values for the pixels;

normalize the shadow magnitude values to generate normalized shadow magnitude values;

apply a radial falloff to the normalized shadow magnitude values to generated adjusted shadow magnitude values;

render the shadow polygon using the adjusted shadow magnitude values; and

overlay the inserted content on the rendered shadow polygon.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the instructions that cause the system to select the plurality of selected joints include instructions that cause the system to select joints based on a data structure associated with the content.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the data structure includes a joint blacklist and the instructions that cause the system to select the plurality of selected joints include instructions that cause the system to select joints that are not in the joint blacklist.

13. The non-transitory computer-readable storage medium of claim 10 , wherein the selecting the plurality of selected joints includes selecting joints based on distance to a surface plane that is identified based on the image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2018
From: NEULANDER, IVAN
To: GOOGLE LLC
Reel/Frame 047128/0064 →
Continuity (3)
Provisional Application 62568115 · Oct 4, 2017
Provisional Application 62568083 · Oct 4, 2017
Related Publication 20190102935A1 · Apr 4, 2019