IP Library Granted Patent US 11,269,667
Granted Patent B2
US 11,269,667 · App. 16/931,088 · Granted Mar 8, 2022

Techniques to switch between different types of virtual assistance based on threshold being met

Inventors: Russell Speight VanBlon (Raleigh, NC); Girishpurushottam Hoogar (Apex, NC); John Weldon Nicholson (Cary, NC)
Assignee: Lenovo (Singapore) Pte. Ltd.
G06F9/453G06F3/013G06F3/167G06T19/006H04M1/72454G10L2015/226G10L2015/228G10L2025/786
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,269,667
App. No.
16/931,088
Granted
Mar 8, 2022
Kind
B2
Abstract

In one aspect, a device may include at least one processor, a display accessible to the at least one processor, and storage accessible to the at least one processor. The storage may include instructions executable by the at least one processor to receive, at the device, input from at least one sensor. The instructions may also be executable to determine whether a threshold is met based at least in part on the input from the at least one sensor. Responsive to the threshold not being met, the instructions may be executable to provide virtual assistance audibly. Responsive to the threshold being met, the instructions may be executable to present visual virtual assistance on the display using an avatar or other character. In various examples, the device may be a headset or other augmented reality device.

Claims (63)

1. A headset, comprising:

at least one processor;

a display accessible to the at least one processor, the display configured to present stereoscopic images; and

storage accessible to the at least one processor and comprising instructions executable by the at least one processor to:

initiate a calibration process for calibrating eye tracking to a particular user, the eye tracking being executable at least in part using the headset;

receive, at the headset, input from at least one sensor;

based at least in part on the input from the at least one sensor, determine whether a first threshold is met, the first threshold being a sound threshold or a light threshold;

based on initiation of the calibration process and responsive to the first threshold not being met, provide virtual assistance audibly that indicates that the particular user is to look at first and second graphical objects presented on the display in a particular sequence for calibrating the eye tracking;

based on initiation of the calibration process and responsive to the first threshold being met, present, on the display, visual virtual assistance that indicates that the particular user is to look at the first and second graphical objects presented on the display in the particular sequence for calibrating the eye tracking; and

based on the calibration process, subsequently execute the eye tracking as part of presentation of an augmented reality (AR) experience provided via the headset, presentation of the AR experience comprising presenting stereoscopic images on the display.

2. The headset of claim 1 , wherein the at least one sensor comprises a microphone, wherein the first threshold is a sound threshold, and wherein the instructions are executable to:

receive input from the microphone;

based at least in part on the input from the microphone, determine whether sound indicated in the input from the microphone is above the sound threshold;

responsive to the sound being below the sound threshold, provide the virtual assistance audibly; and

responsive to the sound being above the sound threshold, present, on the display, the visual virtual assistance.

3. The headset of claim 2 , wherein the sound threshold is a dynamic sound threshold that varies based on a volume level at which audio is set to be output, the dynamic sound threshold being identified from a relational database based on a current volume level at which audio is set to be output.

4. The headset of claim 2 , wherein the sound threshold is a dynamic sound threshold that varies based on a type of audio rendering device that is to be used to output audio associated with a function of the headset, the dynamic sound threshold being identified from a relational database based the type of audio rendering device that is to be used to output audio associated with the function of the headset.

5. The headset of claim 2 , wherein the sound threshold is a dynamic sound threshold that varies based on whether noise cancelling software will be used in conjunction with a function of the headset in order to cancel ambient sound detected via the headset.

6. The headset of claim 1 , wherein the at least one sensor comprises a light sensor, wherein the first threshold is a low light threshold, and wherein the instructions are executable to:

receive input from the light sensor;

based at least in part on the input from the light sensor, determine whether light indicated in the input from the light sensor is above the low light threshold;

responsive to the light indicated in the input being above the low light threshold, provide the virtual assistance audibly; and

responsive to the light indicated in the input being below the low light threshold, present, on the display, the visual virtual assistance.

7. The headset of claim 1 , wherein the headset is established at least in part by computerized glasses.

8. The headset of claim 1 , wherein the display is a transparent display.

9. The headset of claim 1 , wherein the visual virtual assistance comprises instructions provided via text presented on the display.

10. The headset of claim 1 , wherein the visual virtual assistance comprises animation of a third graphical object presented on the display.

11. The headset of claim 10 , wherein the third graphical object gestures in sequence to the first graphical object and then to the second graphical object to indicate that the particular user is to look at the first graphical object and then the second graphical object.

12. The headset of claim 11 , wherein the third graphical object moves on the display from the first graphical object to the second graphical object to indicate that the particular user is to look at the first graphical object and then the second graphical object.

13. The headset of claim 1 , wherein the instructions are executable to:

during the calibration process, present the first graphical object but not the second graphical object;

determine that the particular user has looked at the first graphical object;

based on determining that the particular user has looked at the first graphical object, present the second graphical object.

14. The headset of claim 13 , wherein the instructions are executable to:

based on determining that the particular user has looked at the first graphical object, remove the first graphical object from presentation on the display and present the second graphical object.

15. A method, comprising:

initiating a calibration process for calibrating eye tracking to a particular user, the eye tracking being executable at least in part using a device;

receiving input from at least one sensor;

based at least in part on the input from the at least one sensor, determining whether a first threshold is met, the first threshold being a sound threshold or a light threshold;

based on initiation of the calibration process and based on identifying in a first instance that the first threshold has not been met, providing virtual assistance audibly that indicates that the particular user is to look at first and second graphical objects presented on a display in a particular sequence for calibrating the eye tracking;

based on initiation of the calibration process and based on identifying in a second instance that the first threshold has been met, presenting, on the display, visual virtual assistance that indicates that the particular user is to look at the first and second graphical objects presented on the display in the particular sequence for calibrating the eye tracking; and

subsequently executing the eye tracking at the device according to the calibration process to control presentation of an augmented reality (AR) experience provided via the device.

16. The method of claim 15 , wherein the at least one sensor comprises a microphone, wherein the first threshold is a sound threshold, and wherein the method comprises:

receiving input from the microphone;

based at least in part on the input from the microphone, determining whether sound indicated in the input from the microphone is above the sound threshold;

responsive to the sound being below the sound threshold, providing the virtual assistance audibly; and

responsive to the sound being above the sound threshold, presenting, on the display, the visual virtual assistance.

17. The method of claim 16 , wherein the sound threshold varies based on a volume level at which audio is set to be output using the device, wherein the varying sound threshold to apply in a given instance is identified from a relational database based on a current volume level at which audio is set to be output.

18. The method of claim 15 , wherein the at least one sensor comprises a light sensor, wherein the first threshold is a low light threshold, and wherein the method comprises:

receiving input from the light sensor;

based at least in part on the input from the light sensor, determining whether light indicated in the input from the light sensor is above the low light threshold;

responsive to the light indicated in the input being above the low light threshold, providing the virtual assistance audibly; and

responsive to the light indicated in the input being below the low light threshold, presenting, on the display, the visual virtual assistance.

19. The method of claim 15 , comprising:

presenting the AR experience at least in part by presenting stereoscopic images on the display.

20. At least one computer readable storage medium (CRSM) that is not a transitory signal, the computer readable storage medium comprising instructions executable by at least one processor to:

initiate a calibration process for tailoring eye tracking settings to a particular user, the eye tracking settings being used as part of eye tracking executable at least in part using the at least one processor;

receive input from a sensor accessible to the at least one processor;

based at least in part on the input from the sensor, determine whether a first threshold is met, the first threshold being a sound threshold or a light threshold;

based on initiation of the calibration process and responsive to the first threshold not being met, provide one or more notifications audibly using the at least one processor that indicate that the particular user is to look at first and second graphical objects presented on the display in a particular sequence for tailoring the eye tracking settings;

based on initiation of the calibration process and responsive to the first threshold being met, present, on a display accessible to the at least one processor, one or more visual notifications that indicate that the particular user is to look at the first and second graphical objects presented on the display in the particular sequence for tailoring the eye tracking settings;

tailor the eye tracking settings based on the particular user looking at the first and second graphical objects in the particular sequence; and

execute eye tracking according to the eye tracking settings tailored to the particular user as part of presentation of an augmented reality (AR) experience provided via the at least one processor.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 20, 2025
From: LENOVO PC INTERNATIONAL LIMITED
To: LENOVO SWITZERLAND INTERNATIONAL GMBH
Reel/Frame 070269/0092 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 19, 2025
From: LENOVO (SINGAPORE) PTE LTD.
To: LENOVO PC INTERNATIONAL LIMITED
Reel/Frame 070269/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 16, 2020
From: VANBLON, RUSSELL SPEIGHT; HOOGAR, GIRISHPURUSHOTTAM; NICHOLSON, JOHN WELDON
To: LENOVO (SINGAPORE) PTE. LTD.
Reel/Frame 053232/0228 →
Continuity (1)
Related Publication 20220019450A1 · Jan 20, 2022