IP Library › Granted Patent US 9,830,507
Granted Patent B2
US 9,830,507 · App. 13/073,820 · Granted Nov 28, 2017

Method and apparatus for detecting facial changes

Inventors: Akos Vetek (Helsinki, FI); Jukka Lekkala (Tampere, FI); Antti Vehkaoja (Tampere, FI); Jarmo Verho (Tampere, FI); Ville Rantanen (Ylojarvi, FI); Veikko Surakka (Tampere, FI); Toni Vanhala (Tampere, FI); Oleg Spakov (Tampere, FI)
Assignee: Nokia Technologies Oy
G06K9/00315
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,830,507
App. No.
13/073,820
Granted
Nov 28, 2017
Kind
B2
Abstract

In accordance with an example embodiment, there is an apparatus that has a headset, a contactless proximity sensor arranged to be supported by the headset in the proximity of a face of a user, and a sensor circuitry configured to cause output of a signal indicative of temporal variations in the distance between the contactless proximity sensor and the face of the user.

Claims (35)

1. An apparatus, comprising:

a headset;

a contactless proximity sensor arranged to be supported by a microphone arm of the headset in the proximity of a face of a user, the sensor comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user's mouth indicative of facial movements of the user's face while the electrode and the corner of the user's mouth are at a non-zero distance from one another, wherein the microphone arm of the headset comprises an arm including a microphone, the arm configured to extend over a cheek of the user to the proximity of the mouth of the user; and

a sensor circuitry configured to cause output of a signal indicative of a non-binary range of temporal variations in the distance between the capacitive electrode of the contactless proximity sensor and the corner of the user's mouth.

2. The apparatus of claim 1 , further comprising a wireless transmitter for the output of the signal indicative of temporal variations in the distance between the contactless proximity sensor and the corner of the user's mouth.

3. The apparatus of claim 1 , further comprising one or more further contactless proximity sensors for outputting with the sensor circuitry further signals indicative of temporal variations in the distance between respective contactless proximity sensor and the face of the user.

4. The apparatus of claim 3 , wherein the contactless proximity sensors are arranged to be supported by the headset in the proximity of different parts of the face of the user.

5. The apparatus of claim 1 , further comprising a wireless transmitter configured to wirelessly output the signal indicative of temporal variations in the distance between the contactless proximity sensor and the corner of the user's mouth.

6. A method, comprising:

associating facial detection criteria with facial movements;

receiving capacitance variation information based on operation of one or more capacitive sensors attached to a headset in the proximity of a face of a user, wherein the headset comprises a microphone arm including a microphone, the microphone arm configured to extend over a cheek of the user to the proximity of the mouth of the user, the one or more capacitive sensors each comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user's mouth indicative of facial movements of the user's face while the capacitive electrode and the corner of the user's mouth are at a non-zero distance from one another, and wherein the capacitance variation information comprises a signal indicative of a non-binary range of temporal variations in distance between the electrode and the corner of the user's mouth;

searching for the associated facial changes from the received capacitance variation information using the facial detection criteria; and

determining the facial change that is associated with the matched facial detection criteria in question on finding a sufficient correspondence between the facial detection criteria and the received capacitance variation information.

7. The method of claim 6 further comprising repeatedly providing determinations of facial changes to a data sink.

8. The method of claim 6 further comprising monitoring if either the determination of facial changes has become impossible or a termination signal is received and responsive to either event taking place, terminating the performing of the method.

9. An apparatus, comprising:

at least one processor; and

at least one memory including computer program code;

the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to perform at least the following:

associating facial detection criteria with facial movements;

receiving capacitance variation information based on operation of one or more capacitive sensors attached to a headset in the proximity of a face of a user, wherein the headset comprises a microphone arm including a microphone, the microphone arm configured to extend over a cheek of the user to the proximity of the mouth of the user, the one or more capacitive sensors each comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user's mouth indicative of facial movements of the user's face while the capacitive electrode and the corner of the user's mouth are at a non-zero distance from one another, and wherein the capacitance variation information comprises a signal indicative of a non-binary range of temporal variations in distance between the electrode and the corner of the user's mouth;

searching for the associated facial changes from the received capacitance variation information using the facial detection criteria; and

determining the facial change that is associated with the matched facial detection criteria in question on finding a sufficient correspondence between the facial detection criteria and the received capacitance variation information.

10. A computer program product comprising a non-transitory computer-readable medium having computer program code embodied therein for use with a computer, the computer program code comprising:

code for associating facial detection criteria with facial movements;

code for receiving capacitance variation information based on operation of one or more capacitive sensors attached to a headset in the proximity of a face of a user, wherein the headset comprises a microphone arm including a microphone, the microphone arm configured to extend over a cheek of the user to the proximity of the mouth of the user, the one or more capacitive sensors each comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user's mouth indicative of facial movements of the user's face while the capacitive electrode and the corner of the user's mouth are at a non-zero distance from one another, and wherein the capacitance variation information comprises a signal indicative of a non-binary range of temporal variations in distance between the electrode and the corner of the user's mouth;

code for searching for the associated facial changes from the received capacitance variation information using the facial detection criteria; and

code for determining the facial change that is associated with the matched facial detection criteria in question on finding a sufficient correspondence between the facial detection criteria and the received capacitance variation information.

11. The apparatus of claim 1 , wherein the detected capacitance between the electrode and the corner of the user's mouth is indicative of facial expressions of the user.

12. The method of claim 6 , wherein the detected capacitance between the electrode and the corner of the user's mouth is indicative of facial expressions of the user.

13. The apparatus of claim 9 , wherein the detected capacitance between the electrode and the corner of the user's mouth is indicative of facial expressions of the user.

14. The computer program product of claim 10 , wherein the detected capacitance between the electrode and the corner of the user's mouth is indicative of facial expressions of the user.

15. The apparatus of claim 1 , wherein the temporal variations in the distance between the capacitive electrode of the contactless proximity sensor and the corner of the user's mouth indicate distinctions between smiling, grimacing and talking.

16. The method of claim 6 , wherein determining the facial change that is associated with the matched facial detection criteria in question comprises determining distinctions between smiling, grimacing and talking.

17. The computer program product of claim 10 , wherein determining the facial change that is associated with the matched facial detection criteria in question comprises determining distinctions between smiling, grimacing and talking.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 16, 2015
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 035424/0652 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 20, 2011
From: VETEK, AKOS; LEKKALA, JUKKA; VEHKAOJA, ANTTI; VERHO, JARMO; RANTANEN, VILLE; SURAKKA, VEIKKO; VANHALA, TONI; SPAKOV, OLEG
To: NOKIA CORPORATION
Reel/Frame 026156/0903 →
Continuity (1)
Related Publication 20120254244A1 · Oct 4, 2012