IP Library Granted Patent US 11,513,355
Granted Patent B2
US 11,513,355 · App. 16/810,323 · Granted Nov 29, 2022

Method and system for pervasive memory augmentation via a heads-up display

Inventors: Evangelos Niforatos (Kitchener, CA); Daniel Perry (Waterloo, CA)
Assignee: GOOGLE LLC
G02B27/0172G06F1/163G06F40/10G02B2027/0174
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,513,355
App. No.
16/810,323
Granted
Nov 29, 2022
Kind
B2
Abstract

A method of pervasive memory augmentation includes receiving a request to augment a memory via a heads-up display having a field of view and generating one or more processor-readable memory cues that are connected to the memory. A cued content is generated based on the one or more processor-readable memory cues. The cued content is presented in a display space in the field of view of the heads-up display on a spaced repetition schedule.

Claims (37)

1. A method of pervasive memory augmentation, the method comprising:

in response to a request to augment a memory received via a wearable heads-up display having a field of view, generating one or more processor-readable memory cues that are connected to the memory;

generating a cued content based on the one or more processor-readable memory cues;

presenting the cued content in a display space in the field of view of the wearable heads-up display on a spaced repetition schedule; and

pausing presentation of the cued content when notifications are not allowed including at least one of: when a user of the wearable heads-up display is engaged in a face-to-face conversation with another person; at designated times of day; and when the user has set a “do not disturb” mode.

2. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule comprises presenting the cued content at one of (i) equally spaced time intervals over a set time period, (ii) unequally spaced time intervals over a set time period, and (iii) progressively expanding time intervals over a set time period.

3. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule begins automatically following generation of the one or more processor-readable memory cues.

4. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule begins in response to a subsequent request to begin augmenting the memory.

5. The method of claim 1 , further comprising apart from presenting the cued content in the display space on the spaced repetition schedule, presenting the cued content in the display space in response to at least one of (a) a request to present the memory cues, (b) a reminder of an event linked to the memory, and (c) detecting that the wearable heads-up display is at a location linked to the memory.

6. The method of claim 1 , further comprising repeating presenting the cued content in the display space on the spaced repetition schedule in response to a request to replay augmentation of the memory.

7. The method of claim 1 , further comprising acquiring memory data that describes the memory prior to generating the one or more processor-readable memory cues that are connected to the memory.

8. The method of claim 7 , wherein acquiring memory data comprises receiving audio data captured in a portion of an environment of the wearable heads-up display by at least one microphone during a period in which the memory is made.

9. The method of claim 8 , wherein generating one or more processor-readable memory cues that are connected to the memory comprises:

causing conversion of the audio data to text data;

causing natural language processing of the text data to identify one or more topics and/or keywords in the text data; and

causing generation of the one or more processor-readable memory cues from the one or more topics and/or keywords.

10. The method of claim 9 , further comprising receiving the audio data from the at least one microphone by a processor of the wearable heads-up display and transmitting the audio data to a remote device, wherein a remote processor associated with the remote device generates the one or more processor-readable memory cues.

11. The method of claim 1 , further comprising receiving a location information of one or more digital assets connected to the memory and retrieving the one or more digital assets using the location information, wherein generating one or more processor-readable memory cues that are connected to the memory comprises generating the one or more processor-readable memory cues from the one or more digital assets.

12. The method of claim 11 , wherein receiving a location information of one or more digital assets connected to the memory comprises presenting a user interface in the display space to receive the location information.

13. The method of claim 11 , wherein retrieving the one or more digital assets using the location information comprises at least one of retrieving a digital asset from an application installed on a host device paired with the wearable heads-up display, retrieving a digital asset from an application hosted on at least one remote server, and retrieving a digital asset from a posting on a social network.

14. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule comprises modulating an output light of a display light source with at least a portion of the cued content and projecting the output light to the display space.

15. The method of claim 1 , wherein the memory indicated in the request to augment a memory is selected from a memory of a meeting, a memory of a past event, a memory of a future event, a memory of learning material, and a memory of sensitive information.

16. A method of pervasive memory augmentation, the method comprising:

in response to a request to augment a memory received via a wearable heads-up display having a field of view, generating one or more processor-readable memory cues that are connected to the memory;

generating a cued content based on the one or more processor-readable memory cues;

presenting the cued content in a display space in a field of view of the wearable heads-up display in response to at least one of (a) a request to present the memory cues, (b) a reminder of an event linked to the memory, and (c) detecting that the wearable heads-up display is at a location linked to the memory; and

pausing presentation of the cued content when notifications are not allowed including at least one of: when a user of the wearable heads-up display is engaged in a face-to-face conversation with another person; at designated times of day; and when the user has set a “do not disturb” mode.

17. A system for pervasive memory augmentation, the system comprising:

a wearable heads-up display having a field of view, the wearable heads-up display comprising a display light source, a processor communicatively coupled to the display light source, and a non-transitory processor-readable medium communicatively coupled to the processor, wherein the non-transitory processor-readable medium stores data and/or processor-executable instructions that, when executed by the processor, causes the wearable heads-up display to:

receive a request to augment a memory;

generate one or more processor-readable memory cues that are connected to the memory;

generate a cued content based on the one or more processor-readable memory cues;

present the cued content in a display space in the field of view of the wearable heads-up display on a spaced repetition schedule; and

pause presentation of the cued content when notifications are not allowed including at least one of: when a user of the wearable heads-up display is engaged in a face-to-face conversation with another person; at designated times of day; and when the user has set a “do not disturb” mode.

18. The system of claim 17 , further comprising at least one sensor to capture a sensory input in an environment of the wearable heads-up display.

19. The system of claim 17 , further comprising a support frame that in use is worn on a head of a user, wherein the display light source and processor are carried by the support frame.

20. The system of claim 17 , wherein the display light source is a scanning laser projector, a micro-display, or a digital light processing projector.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2020
From: NORTH INC.
To: GOOGLE LLC
Reel/Frame 054144/0892 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2020
From: NIFORATOS, EVANGELOS; PERRY, DANIEL
To: NORTH INC.
Reel/Frame 052030/0427 →
Continuity (2)
Provisional Application 62817666 · Mar 13, 2019
Related Publication 20200292823A1 · Sep 17, 2020
Cited By (4)
US 1,099,131 US 1,099,132 US 1,099,934 US 1,099,935