IP Library › Granted Patent US 11,436,808
Granted Patent B2
US 11,436,808 · App. 17/283,774 · Granted Sep 6, 2022

Selecting augmented reality objects for display based on contextual cues

Inventors: Diane Wang (Mountain View, CA); Paulo Coelho (San Jose, CA); Tarik Hany Abdel-Gawad (San Francisco, CA); Matthew Gilgenbach (Mountain View, CA); Jackson Lango (Mountain View, CA); Douglas Muir (Encino, CA); Mark Dochtermann (Sherman Oaks, CA); Suddhasattwa Bose (Culver City, CA); Ashley Pinnick (Pasadena, CA); Drew Skillman (Sherman Oaks, CA); Samantha Raja (San Francisco, CA); Steven Toh (Alameda, CA); Brian Collins (San Francisco, CA); Jay Steele (Santa Monica, CA)
Assignee: GOOGLE LLC
G06T19/006G06F3/04815G06T7/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,436,808
App. No.
17/283,774
Granted
Sep 6, 2022
Kind
B2
Abstract

Disclosed are various embodiments for selecting augmenting reality (AR) objects based on contextual cues associated with an image captured by a camera associated with electronic device. Contextual cues are obtained at an electronic device and AR objects are identified from a memory associated with the electronic device. The electronic device implements a processor employing image segmentation techniques to combine the identified AR objects with the captured image and render the combined image for display at a display associated with the electronic device.

Claims (37)

1. A method comprising:

obtaining one or more contextual cues at an electronic device;

identifying multiple augmented reality (AR) objects from a memory associated with the electronic device based on the one or more contextual cues;

ranking the multiple AR objects based on a confidence metric associated with each of the one or more contextual cues, and selecting at least one AR object of the multiple AR objects based at least in part on the ranking;

combining an image with the selected at least one AR object; and

displaying the combined image at a display associated with the electronic device.

2. The method of claim 1 , wherein the contextual cues comprise a person, a facial expression, a sound, a location, a time, a date, an activity, weather information, an item, or combinations thereof.

3. The method of claim 1 , further comprising generating the combined image using image segmentation.

4. The method of claim 1 , further comprising changing the identified at least one AR object based on a change in the one or more contextual cues.

5. The method of claim 1 , further comprising:

presenting a set of AR objects based on the ranking.

6. The method of claim 1 , further comprising identifying the one or more contextual cues from the image.

7. The method of claim 1 , further comprising receiving the one or more contextual cues from a sensor associated with the electronic device.

8. The method of claim 1 , wherein the selecting of the at least one AR object is further based on a user selection.

9. A system comprising:

a memory configured to store a plurality of augmented reality (AR) objects;

a processor configured to:

identify multiple augmented reality (AR) objects from the memory based on one or more contextual cues;

rank the multiple AR objects based on a confidence metric associated with each of the one or more contextual cues;

select at least one AR object of the multiple AR objects based at least in part on the ranking; and

combine an image from the memory with the selected at least one AR object; and

a display configured to display the combined image.

10. The system of claim 9 , wherein the processor is configured to change the at least one selected AR object in response to a change in the one or more contextual cues.

11. The system of claim 9 , the processor being further configured to:

present a set of identified AR objects based on the ranking.

12. The system of claim 9 , wherein the contextual cues comprise a person, a facial expression, a sound, a location, a time, a date, an activity, weather information, an item, or combinations thereof.

13. The system of claim 9 , wherein the processor is further configured to obtain the one or more contextual cues from a sensor.

14. The system of claim 9 , wherein the processor is further configured to employ image segmentation to combine the image with the selected at least one AR object.

15. The system of claim 9 , wherein the selected at least one AR object is further based on a user selection.

16. A non-transitory computer readable medium embodying a set of executable instructions, the set of executable instructions to manipulate a computer system to perform a portion of a process to fabricate at least part of a processor, the processor comprising:

a ranking component configured to identify multiple augmented reality (AR) objects from a memory associated with an electronic device based on a confidence metric associated with one or more contextual cues, and to rank the multiple AR objects based at least in part on the associated confidence metric;

a combining component configured to combine an image with a selected at least one augmented reality (AR) object, the selected at least one AR object being selected based at least in part on a rank assigned to the at least one AR object; and

a display configured to display the combined image.

17. The non-transitory computer readable medium of claim 16 , wherein the one or more contextual cues are based at least in part on one or more images captured by a camera associated with the electronic device, information generated by one or more sensors associated with the electronic device, information received via a network associated with the electronic device, or a combination thereof.

18. The non-transitory computer readable medium of claim 16 , wherein the processor is further configured to employ image segmentation to generate the combined image.

19. The non-transitory computer readable medium of claim 16 , wherein the processor is further configured to modify the at least one AR object.

20. The non-transitory computer readable medium of claim 16 , wherein the selected at least one AR object is modified based on a change of the one or more contextual cues.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2022
From: WANG, DIANE; COELHO, PAULO; ABDEL-GAWAD, TARIK; GILGENBACH, MATTHEW; LANGO, JACKSON; MUIR, DOUGLAS; DOCHTERMANN, MARK; JOU, BRENDAN; BOSE, SUDDHASATTWA
To: GOOGLE LLC
Reel/Frame 060257/0305 →
Continuity (2)
Provisional Application 62743038 · Oct 9, 2018
Related Publication 20210383609A1 · Dec 9, 2021
Cited By (3)
US 12,406,450 US 12,536,744 US 12,705,828