IP Library Granted Patent US 10,327,984
Granted Patent B2
US 10,327,984 · App. 15/340,058 · Granted Jun 25, 2019

Controlling ear stimulation in response to image analysis

Inventors: Eleanor V. Goodall (Seattle, WA); Roderick A. Hyde (Redmond, WA); Muriel Y. Ishikawa (Livermore, CA); Jordin T. Kare (San Jose, CA); Melanie K. Kitzan (North Bend, WA); Eric C. Leuthardt (St. Louis, MO); Mark A. Malamud (Seattle, WA); Stephen L. Malaska (Snoqualmie, WA); Nathan P. Myhrvold (Medina, WA); Brittany Scheid (St. Louis, MO); Katherine E. Sharadin (Redmond, WA); Elizabeth A. Sweeney (Seattle, WA); Clarence T. Tegreene (Mercer Island, WA); Charles Whitmer (North Bend, WA); Lowell L. Wood, Jr. (Bellevue, WA); Victoria Y. H. Wood (Livermore, CA)
Assignee: eQuility LLC
A61H23/02A61N1/0456A61N1/0472A61N1/36014A61N1/36036A61N1/37247A61N1/37258A61N2/002A61N2/006A61N5/0622A61N7/00A61N1/0484A61N1/36021A61N1/36025A61N2/02A61N2005/0647A61N2007/0026A63F13/212A63F13/424
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,327,984
App. No.
15/340,058
Granted
Jun 25, 2019
Kind
B2
Abstract

Systems and related methods for controlling an ear stimulation device with a personal computing device, based on analysis of an image of a user of the personal computing device captured with a camera associated with the personal computing device, are described. One or more parameter is determined from the processed image, including, but not limited, to user identity, emotion, physiological condition, or placement of a stimulator earpiece, and control of the ear stimulation device is based thereon. If the earpiece is located incorrectly, in an aspect, delivery of a stimulus is prevented. In another aspect, the user is notified of the need to reposition an incorrectly located earpiece.

Claims (81)

1. A method of controlling an ear stimulation device with a personal computing device, comprising:

capturing, with image capture circuitry on the personal computing device, via a user-facing camera associated with the personal computing device, an image of a user of the personal computing device;

processing the image, using image processing circuitry on the personal computing device, to determine at least one parameter; and

controlling, with neural stimulus control signal determination circuitry on the personal computing device, based at least in part on the at least one parameter, delivery of a stimulus to at least one nerve innervating a pinna of an ear of the user with the ear stimulation device,

wherein the stimulus includes as least one of a mechanical stimulus, electrical stimulus, magnetic stimulus, ultrasonic stimulus, optical stimulus, or chemical stimulus.

2. The method of claim 1 , wherein the image processing circuitry includes application software on the personal computing device.

3. The method of claim 1 , wherein the at least one parameter is an emotional parameter of the user.

4. The method of claim 1 , wherein the at least one parameter is a physiological parameter of the user.

5. The method of claim 1 , wherein the at least one parameter is a medical parameter of the user.

6. The method of claim 1 , wherein the at least one parameter is a biometric parameter indicative of an identity of the user.

7. The method of claim 1 , wherein the at least one parameter is a heart rate of the user.

8. The method of claim 1 , wherein the at least one parameter is an eye position or eye movement parameter of the user.

9. The method of claim 1 , wherein the at least one parameter is a distance of one or more portions of an earpiece of the ear stimulation device with respect to at least one anatomical feature of the ear of the user.

10. The method of claim 9 , including delivering, under control of notification circuitry on the personal computing device, a notification to the user informing the user of a need to adjust a position of the earpiece with respect to the ear of the user.

11. The method of claim 1 , wherein the personal computing device includes a phone, watch, wearable device, tablet computer, laptop computer, or desktop computer.

12. The method of method of claim 1 , including

processing the image, using the image processing circuitry, to determine

a presence of at least one earpiece of the ear stimulation device at the ear of the user, the ear selected from a right ear of the user and a left ear of the user; and

at least one attribute of the at least one earpiece indicative of usability of the at least one earpiece with the ear of the user at which the earpiece is located;

determining, using application software on the personal computing device, a target ear at which the at least one ear earpiece is usable, based on the at least one attribute of the at least one earpiece;

determining, using application software on the personal computing device, that the ear at which the at least one earpiece is located is not the target ear at which the at least one earpiece is usable; and

sending a control signal from the personal computing device to the ear stimulation device, under control of the neural stimulus control signal determination circuitry, to prevent delivery of a stimulus to the pinna of the ear at which the at least one earpiece is located via the at least one earpiece.

13. The method of claim 12 , including receiving, with handshake circuitry on the personal computing device, a handshake signal from ear stimulation device control circuitry associated with the ear stimulation device.

14. The method of claim 13 , including capturing the image of the user of the personal computing device responsive to receiving the handshake signal from the ear stimulation device control circuitry.

15. The method of claim 13 , including sending a handshake signal to the ear stimulation device control circuitry responsive to determining the presence of the at least one earpiece located at the ear of the user in the image.

16. The method of claim 12 , including delivering, under control of notification circuitry on the personal computing device, a notification to the user informing the user of a need to switch the at least one earpiece to the target ear if the ear at which the at least one earpiece is located is not the target ear at which the at least one earpiece is usable.

17. The method of claim 16 , wherein delivering the notification to the user includes delivering a text notification.

18. The method of claim 16 , wherein delivering the notification to the user includes delivering a visible notification.

19. The method of claim 16 , wherein delivering the notification to the user includes delivering an audio notification.

20. The method of claim 12 , wherein determining the at least one attribute of the at least one earpiece includes determining a shape of the at least one earpiece.

21. The method of claim 12 , wherein determining the at least one attribute of the at least one earpiece includes determining a color of the at least one earpiece.

22. The method of claim 12 , wherein determining the at least one attribute of the at least one earpiece includes determining a presence of an indicia on the at least one earpiece or an attachment to the at least one earpiece.

23. The method of claim 22 , wherein the attachment to the at least one earpiece includes a cable connected to the at least one earpiece.

24. An ear stimulation device control system, comprising:

a personal computing device;

a user-facing camera associated with the personal computing device;

image capture circuitry adapted to capture an image of a user of the personal computing device from the user-facing camera;

image processing circuitry configured to process the image to determine at least one parameter; and

neural stimulus control signal determination circuitry configured to control delivery of a stimulus to at least one nerve innervating a pinna of an ear of the user with an ear stimulation device, based at least in part on the at least one parameter,

wherein the stimulus includes as least one of a mechanical stimulus, electrical stimulus, magnetic stimulus, ultrasonic stimulus, optical stimulus, or chemical stimulus.

25. The system of claim 24 , wherein the at least one parameter is an emotional parameter of the user.

26. The system of claim 24 , wherein the at least one parameter is a physiological parameter of the user.

27. The system of claim 24 , wherein the at least one parameter is a biometric parameter indicative of an identity of the user.

28. The system of claim 24 , wherein the at least one parameter is a heart rate of the user.

29. The system of claim 24 , wherein the at least one parameter is an eye position or eye movement parameter of the user.

30. The system of claim 24 , wherein the at least one parameter is a distance of one or more portions of an earpiece of the ear stimulation device with respect to at least one anatomical feature of the ear of the user.

31. The system of claim 24 , wherein the personal computing device includes a phone, watch, wearable device, tablet computer, laptop computer, or desktop computer.

32. The system of claim 24 , wherein the image processing circuitry includes an earpiece location module configured to process the image to determine

a presence of at least one earpiece of the ear stimulation device at the ear of the user, the ear selected from a right ear of the user and a left ear of the user; and

at least one attribute of the at least one earpiece indicative of usability of the at least one earpiece with the ear of the user at which the at least one earpiece is located; and

wherein the neural stimulus control signal determination circuitry is configured to

determine a target ear at which the earpiece is usable, based on the at least one attribute of the at least one earpiece;

determine that the ear at which the at least one earpiece is located is not the target ear at which the earpiece is usable; and if the ear at which the at least one earpiece is located is not the ear at which the earpiece is usable; and

send a control signal from the personal computing device to the ear stimulation device to prevent delivery of the stimulus to the at least one nerve innervating the pinna of the ear of the user.

33. The system of claim 32 , including handshake circuitry adapted to receive a handshake signal from ear stimulation device control circuitry associated with the ear stimulation device.

34. The system of claim 33 , wherein the image capture circuitry is adapted to capture the image of the user of the personal computing device responsive to receiving the handshake signal from the ear stimulation device control circuitry.

35. The system of claim 33 , wherein the neural stimulus control signal determination circuitry is configured to send a handshake signal to the ear stimulation device control circuitry responsive to determining the presence of the at least one earpiece located at the ear of the user in the image.

36. The system of claim 32 , including

an output device; and

notification circuitry adapted to provide a notification via the output device instructing the user to switch the at least one earpiece to the other ear if the ear at which the at least one earpiece is located is not the target ear at which the earpiece is usable.

37. The system of claim 36 , wherein the output device is adapted to deliver a text notification to the user.

38. The system of claim 36 , wherein the output device is adapted to deliver a visible notification to the user.

39. The system of claim 36 , wherein the output device is adapted to deliver an audio notification to the user.

40. The system of claim 32 , wherein the at least one attribute of the at least one earpiece includes a shape of the at least one earpiece.

41. The system of claim 32 , wherein the at least one attribute of the at least one earpiece includes a color of the at least one earpiece.

42. The system of claim 32 , wherein the at least one attribute of the at least one earpiece includes a presence of an indicia on the at least one earpiece or an attachment to the at least one earpiece.

43. The system of claim 42 , wherein the attachment to the at least one earpiece includes a cable connected to the at least one earpiece.

44. An ear stimulation device control system, comprising:

a personal computing device;

a user-facing camera associated with the personal computing device;

image capture circuitry adapted to capture an image of a user of the personal computing device from the user-facing camera;

image processing circuitry configured to process the image to determine at least one parameter, wherein the image processing circuitry includes

an earpiece location module configured to process the image to determine

a presence of at least one earpiece of the ear stimulation device at the ear of the user, the ear selected from a right ear of the user and a left ear of the user; and

at least one attribute of the at least one earpiece indicative of usability of the at least one earpiece with the ear of the user at which the earpiece is located; and

neural stimulus control signal determination circuitry configured to control

delivery of a stimulus to at least one nerve innervating an ear of the user with an ear stimulation device, based at least in part on the at least one parameter,

wherein the neural stimulus control signal determination circuitry is configured to

determine a target ear at which the at least one earpiece is usable, based on the at least one attribute of the at least one earpiece;

determine that the ear at which the at least one earpiece is located is not the target ear at which the at least one earpiece is usable; and

send a control signal from the personal computing device to the ear stimulation device to prevent delivery of the stimulus to the a least one nerve innervating the ear of the user.

Assignments (6)
CHANGE OF NAME Recorded Apr 10, 2024
From: AURENAR, LLC
To: AURENAR, INC.
Reel/Frame 067070/0365 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 31, 2024
From: THE INVENTION SCIENCE FUND II, LLC
To: AURENAR, LLC
Reel/Frame 066314/0076 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2019
From: EQUILITY LLC
To: THE INVENTION SCIENCE FUND II, LLC
Reel/Frame 051329/0426 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2018
From: THE INVENTION SCIENCE FUND II, LLC
To: EQUILITY LLC
Reel/Frame 045869/0062 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2018
From: ELWHA LLC
To: THE INVENTION SCIENCE FUND II, LLC
Reel/Frame 045845/0196 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 19, 2017
From: GOODALL, ELEANOR V.; HYDE, RODERICK A.; ISHIKAWA, MURIEL Y.; KARE, JORDIN T.; KITZAN, MELANIE K.; LEUTHARDT, ERIC C.; MALAMUD, MARK A.; MALASKA, STEPHEN L.; MYHRVOLD, NATHAN P.; SCHEID, BRITTANY; SHARADIN, KATHERINE E.; SWEENEY, ELIZABETH A.; TEGREENE, CLARENCE T.; WHITMER, CHARLES; WOOD, LOWELL L., JR.; WOOD, VICTORIA Y. H.
To: ELWHA LLC
Reel/Frame 043039/0666 →
Continuity (8)
Continuation In Part 14670504 · Mar 27, 2015
Continuation In Part 14670537 · Mar 27, 2015
Continuation In Part 14670560 · Mar 27, 2015
Continuation In Part 14670582 · Mar 27, 2015
Continuation In Part 14670620 · Mar 27, 2015
Continuation In Part 14670656 · Mar 27, 2015
Continuation In Part 15291358 · Oct 12, 2016
Related Publication 20170113042A1 · Apr 27, 2017
Cited By (2)
US 12,357,811 US 12,409,316