IP Library Granted Patent US 10,950,047
Granted Patent B2
US 10,950,047 · App. 16/654,990 · Granted Mar 16, 2021

Techniques for anonymizing neuromuscular signal data

Inventors: Patrick Kaifosh (New York, NY); Tudor Giurgica-Tiron (Stanford, CA); Steven Kober (New York, NY); Adam Al-natsheh (New York, NY); Alexandre Barachant (Brooklyn, NY)
Assignee: Facebook Technologies, LLC
G06T19/00G06F3/014G06F3/015G06F3/016G06F3/017G06N20/00G06F2203/011
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,950,047
App. No.
16/654,990
Granted
Mar 16, 2021
Kind
B2
Abstract

Methods and apparatus for anonymizing neuromuscular signals used to generate a musculoskeletal representation. The method comprises recording, using a plurality of neuromuscular sensors arranged on one or more wearable devices, a plurality of neuromuscular signals from a user, providing as input to a trained statistical model, the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals; and generating, the musculoskeletal representation based, at least in part, on an output of the trained statistical model, wherein the musculoskeletal representation is an anonymized musculoskeletal representation from which at least one personal characteristic of the user has been removed.

Claims (57)

1. A computerized system for anonymizing neuromuscular signals used to generate a musculoskeletal representation, the computerized system comprising:

a plurality of neuromuscular sensors configured to continuously record a plurality neuromuscular signals from a user, wherein the plurality of neuromuscular sensors are arranged on one or more wearable devices; and

at least one computer processor programmed to:

provide as input to a statistical model, the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals, wherein the statistical model is trained to remove a first personal characteristic of the user from the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals; and

generate the musculoskeletal representation based, at least in part, on an output of the statistical model,

wherein the musculoskeletal representation is an anonymized musculoskeletal representation of the user from which the first personal characteristic of the user has been removed, and

the statistical model is further trained to predict estimates of positions associated with a plurality of rigid segments within the musculoskeletal representation.

2. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to process the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals to remove a second personal characteristic of the user.

3. The computerized system of claim 2 , wherein the second personal characteristic of the user is removed from the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals prior to providing the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals as an input to the statistical model.

4. The computerized system of claim 2 , wherein the at least one computer processor is further programmed to:

store the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals in a storage device,

wherein the second personal characteristic of the user is removed from the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals prior to storage of the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals in the storage device.

5. The computerized system of claim 2 , wherein the second personal characteristic of the user comprises at least one movement pattern associated with a neurological disorder or a non-pathological idiosyncrasy, and the at least one computer processor is further programmed to process the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals to:

identify the at least one movement pattern in the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals; and

remove the at least one movement pattern from the plurality of neuromuscular signals and/or the information based on the plurality of neuromuscular signals.

6. The computerized system of claim 1 , wherein the first personal characteristic of the user relates to at least one of muscle fatigue, muscle activity level, muscle frequency content, and muscle spiking pattern.

7. The computerized system of claim 1 , wherein the first personal characteristic of the user relates to a body mass index or a body fat percentage of the user.

8. The computerized system of claim 1 , wherein the first personal characteristic of the user comprises at least one movement pattern associated with a neurological disorder or a non-pathological idiosyncrasy.

9. The computerized system of claim 1 , wherein the statistical model is trained to remove the first personal characteristic of the user using an adversarial training approach.

10. The computerized system of claim 9 , wherein the adversarial training approach comprises using an adversarial network that is configured to concurrently remove the first personal characteristic of the user and predict estimates of position and force associated with at least one segment of the musculoskeletal representation.

11. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to process the output of the statistical model to remove a second personal characteristic of the user.

12. The computerized system of claim 11 , wherein processing the output of the statistical model comprises averaging the output of the statistical model over time to generate a smoothed output of the statistical model.

13. The computerized system of claim 11 , wherein processing the output of the statistical model comprises mapping the output of the statistical model to at least one state of a plurality of discrete states, wherein each state of the plurality of discrete states represents a generic movement pattern.

14. The computerized system of claim 1 , wherein the at least one computer processor is further programmed to:

render a visual representation based on the musculoskeletal representation, wherein a second personal characteristic of the user is removed during rendering of the visual representation.

15. A computerized system for anonymizing neuromuscular signals used to generate a musculoskeletal representation, the computerized system comprising:

a plurality of neuromuscular sensors configured to continuously record a plurality of neuromuscular signals from a user, wherein the plurality of neuromuscular sensors are arranged on one or more wearable devices; and

at least one computer processor programmed to:

provide as input to a statistical model, the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals;

generate, the musculoskeletal representation based, at least in part, on an output of the statistical model; and

render a visual representation of the user based on the musculoskeletal representation, wherein at least one personal characteristic of the user is removed during rendering of the visual representation,

wherein the visual representation includes estimates of positions associated with a plurality of rigid segments within the musculoskeletal representation.

16. The computerized system of claim 15 , wherein the at least one personal characteristic of the user is removed by altering a dimension of one or more fingers of a hand during rendering of the visual representation of the hand.

17. The computerized system of claim 15 , wherein the at least one personal characteristic of the user is removed by mapping the visual representation to at least one state of a plurality of discrete states, wherein each state of the plurality of discrete states represents a generic visual representation.

18. The computerized system of claim 15 , wherein the at least one computer processor is further programmed to:

receive user input indicating whether to render an anonymized visual representation.

19. The computerized system of claim 18 , wherein the at least one computer processor is further programmed to:

in response to a user input indicating that the anonymized visual representation is to be rendered, render the visual representation with the at least one personal characteristic removed from the visual representation.

20. A method for anonymizing neuromuscular signals used to generate a musculoskeletal representation, the method comprising:

recording, using a plurality of neuromuscular sensors arranged on one or more wearable devices, a plurality of neuromuscular signals from a user;

providing as input to a statistical model, the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals, wherein the statistical model is trained to remove at least one personal characteristic of the user from the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals; and

generating, the musculoskeletal representation based, at least in part, on an output of the statistical model,

wherein the musculoskeletal representation is an anonymized musculoskeletal representation of the user from which the at least one personal characteristic of the user has been removed, and

the statistical model is further trained to predict estimates of positions associated with a plurality of rigid segments within the musculoskeletal representation.

21. The method of claim 20 , wherein the statistical model is trained to remove a first personal characteristic of the user using an adversarial training approach.

22. The method of claim 21 , wherein the adversarial training approach comprises using an adversarial network that is configured to concurrently remove the at least one personal characteristic of the user and predict estimates of position and force associated with at least one segment of the musculoskeletal representation.

23. A method for anonymizing neuromuscular signals used to generate a musculoskeletal representation, the method comprising:

providing as input to a trained statistical model, a plurality of neuromuscular signals recorded from a user by a plurality of neuromuscular sensors and/or information based on the plurality of neuromuscular signals;

generating the musculoskeletal representation based, at least in part, on an output of the trained statistical model;

estimating positions associated with a plurality of rigid segments within the musculoskeletal representation; and

rendering a visual representation based on the musculoskeletal representation, wherein at least one personal characteristic of the user is removed during rendering of the visual representation.

24. The method of claim 23 , wherein the at least one personal characteristic of the user is removed by altering a dimension of one or more fingers of a hand during rendering of the visual representation of the hand.

25. The method of claim 23 , wherein the at least one personal characteristic of the user is removed by mapping the visual representation to at least one state of a plurality of discrete states, wherein each state of the plurality of discrete states represents a generic visual representation.

26. The method of claim 23 , wherein the method further comprises:

receiving user input indicating whether to render an anonymized visual representation.

27. The method of claim 26 , wherein the method further comprises:

in response to a user input indicating that the anonymized visual representation is to be rendered, rendering the visual representation with the at least one personal characteristic removed from the visual representation.

Assignments (7)
CHANGE OF NAME Recorded May 26, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060199/0876 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY DATA PREVIOUSLY RECORDED AT REEL: 051649 FRAME: 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Feb 10, 2020
From: CTRL-LABS CORPORATION
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 051867/0136 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 20, 2020
From: CTRL-LABS CORPORATION
To: FACEBOOK TECHNOLOGIES, INC.
Reel/Frame 051649/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2019
From: BARACHANT, ALEXANDRE
To: CTRL-LABS CORPORATION
Reel/Frame 051397/0558 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2019
From: KAIFOSH, PATRICK; KOBER, STEVEN; AL-NATSHEH, ADAM
To: CTRL-LABS CORPORATION
Reel/Frame 051273/0509 →
CHANGE OF NAME Recorded Dec 12, 2019
From: COGNESCENT CORPORATION
To: CTRL-LABS CORPORATION
Reel/Frame 051273/0993 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2019
From: GIURGICA-TIRON, TUDOR
To: COGNESCENT CORPORATION
Reel/Frame 051273/0808 →