IP Library Granted Patent US 10,432,687
Granted Patent B1
US 10,432,687 · App. 15/983,724 · Granted Oct 1, 2019

Biometric masking

Inventors: M. David Hanes (Lewisville, NC); Joseph Michael Clarke (Raleigh, NC); Gonzalo Salgueiro (Raleigh, NC); Charles Calvin Byers (Wheaton, IL)
Assignee: Cisco Technology, Inc.
H04L65/601G06K9/0061G06K9/00315G06T5/008G06T5/40G06T13/40H04L65/1089H04L65/1093H04L65/80H04N7/157G06T2207/10024G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,432,687
App. No.
15/983,724
Granted
Oct 1, 2019
Kind
B1
Abstract

Biometric masking includes testing video data and/or audio data from the video conference for biometric markers indicative of emotion or stress levels. If at least one of the biometric markers is detected with the testing, the video data and/or the audio data is edited to obfuscate the at least one of the biometric markers so that the video data and the audio data is transmitted to participants of the video conference with the at least one of the biometric markers obfuscated.

Claims (58)

1. An apparatus comprising:

a camera configured to:

capture sample video data during a setup period when a user becomes a participant of a video conference, wherein the sample video data is not transmitted to other participants of the video conference; and

capture video data of the participant during the video conference;

an input/output module to transmit data to and receive data from a video conference server hosting the video conference;

processing circuitry; and

a biometric detection and obfuscation application configured to derive baseline data from the sample video data, the biometric detection and obfuscation application comprising at least one of:

a pulse masker to be executed by the processing circuitry and operative to detect and obfuscate facial coloration of the participant in the video data that is indicative of a pulse rate of the participant, the facial coloration being detected based on a comparison of the video data to the baseline data;

an expression and respiration masker to be executed by the processing circuitry and operative to detect and obfuscate a micro-expression and/or a respiration rate for the participant in the video data, the micro-expression and/or the respiration rate being detected based on a comparison of the video data to the baseline data; and

a pupil masker to be executed by the processing circuitry and operative to detect and obfuscate a pupil size change and/or a pupil movement in the video data for the participant, the pupil size change and/or a pupil movement being detected based on a comparison of the video data to the baseline data.

2. The apparatus according to claim 1 further comprising:

a microphone configured to:

capture sample audio data during the setup period, wherein the sample audio data is not transmitted to the other participants of the video conference; and

capture audio data during the video conference; and

wherein the biometric detection and obfuscation application further derives the baseline data from the sample audio data and further comprises a voice masker to be executed by the processing circuitry and operative to detect and obfuscate voice stress markers in the audio data, the voice stress markers being detected based on a comparison of the audio data to the baseline data.

3. The apparatus according to claim 1 further comprising:

a circular buffer to store recent observations of the biometric detection and obfuscation application so that the biometric detection and obfuscation application can utilize the recent observations for detections and obfuscations.

4. The apparatus according to claim 1 , wherein the expression and respiration masker utilizes a reference example to obfuscate the micro-expression.

5. The apparatus according to claim 4 , wherein the expression and respiration masker is configured to imprint faces in the video data with the reference example.

6. The apparatus according to claim 1 wherein the expression and respiration masker is configured to imprint the video data with slight movements indicative of respiration to simulate an obfuscated respiration rate that is different from the respiration rate of the participant.

7. The apparatus according to claim 1 wherein the pulse masker is configured to edit the facial coloration to simulate an obfuscated pulse rate for the participant that is different from the pulse rate of the participant.

8. A method for obfuscating biometric markers in a video conference, the method implemented on a computing device and comprising:

capturing sample video data and/or sample audio data during a setup period when a user becomes a participant of a video conference, wherein the sample video data and the sample audio data is not transmitted to other participants of the video conference;

deriving baseline data from the sample video data and/or the sample audio data;

testing video data and/or audio data from during the video conference for biometric markers indicative of emotion or stress levels by comparing the video data and/or the audio data to the baseline data;

upon detecting at least one of the biometric markers with the testing, editing the video data and/or the audio data to obfuscate the at least one of the biometric markers; and

transmitting the video data and/or the audio data to the other participants of the video conference with the at least one of the biometric markers obfuscated.

9. The method according to claim 8 wherein:

the testing comprises detecting changes in facial color indicative of a pulse rate of the participant; and

the editing comprises at least one of: replacing the facial color with a neutral color from a previous video frame in the video data, or manipulating a histogram and color space for a current video frame associated with the changes.

10. The method according to claim 8 wherein:

the testing comprises detecting changes in facial color indicative of a pulse rate of the participant; and

the editing comprises replacing a current video frame associated with the changes with a previous video frame in which the changes were not detected.

11. The method according to claim 8 wherein:

the testing comprises detecting changes in facial color indicative of a pulse rate of the participant; and

the editing comprises replacing the facial color with a baseline color from the baseline data.

12. The method according to claim 8 wherein the biometric markers include at least one of pupil movement and pupil size.

13. The method according to claim 12 further comprising:

extracting pupil data from recent observations of pupils in the video data;

storing the pupil data in a circular buffer that is configured to store data from a predetermined period of time prior to the testing; and

using the pupil data to obfuscate at least one of the pupil movement and the pupil size.

14. The method according to claim 8 further comprising:

performing the testing and the editing iteratively until the biometric markers are not detected in the testing.

15. The method according to claim 8 wherein the testing further comprises:

testing for movements consistent with breathing;

tracking a breathing rate according to the testing for movements; and

detecting a biometric marker associated with respiration rate according to the tracking and the baseline data.

16. The method according to claim 8 wherein the editing comprises editing the video data to imprint non-detected biometric markers into the video data.

17. One or more non-transitory computer readable storage media encoded with software comprising computer executable instructions and when the software is executed operable to:

capture sample video data and/or sample audio data during a setup period when a user becomes a participant of a video conference, wherein the sample video data and the sample audio data is not transmitted to other participants of the video conference;

derive baseline data from the sample video data and/or the sample audio data;

test video data and/or audio data from during a video conference session for biometric markers indicative of emotion or stress levels by comparing the video data and/or the audio data to the baseline data;

upon detecting at least one of the biometric markers with the testing, edit the video data and/or the audio data to obfuscate the at least one of the biometric markers; and

transmit the video data and/or the audio data to the other participants of the video conference with the at least one of the biometric markers obfuscated.

18. The non-transitory computer readable storage media of claim 17 , wherein the biometric markers comprise at least one of: voice stress markers; a pulse rate; a micro-expression; a respiration rate; a pupil size; or a pupil movement.

19. The non-transitory computer readable storage media of claim 17 , wherein the instructions operable to edit are operable to edit the video data to imprint non-detected biometric markers into the video data.

20. The non-transitory computer readable storage media of claim 17 , further comprising instructions operable to:

store the video data from during the video conference session in a circular buffer that is configured to store data from a predetermined period of time.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2018
From: HANES, M. DAVID; CLARKE, JOSEPH MICHAEL; SALGUEIRO, GONZALO; BYERS, CHARLES CALVIN
To: CISCO TECHNOLOGY, INC.
Reel/Frame 045846/0743 →
Cited By (6)
US 12,277,949 US 12,328,199 US 12,348,327 US 12,388,930 US 12,418,628 US 12,619,393