IP Library Granted Patent US 11,755,121
Granted Patent B2
US 11,755,121 · App. 17/580,545 · Granted Sep 12, 2023

Gesture information processing method and apparatus, electronic device, and storage medium

Inventors: Xiaolin Hong (Shenzhen, CN); Qingqing Zheng (Shenzhen, CN); Xinmin Wang (Shenzhen, CN); Kai Ma (Shenzhen, CN); Yefeng Zheng (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G06F3/017G06N3/045G06T5/002G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,755,121
App. No.
17/580,545
Granted
Sep 12, 2023
Kind
B2
Abstract

This application provides a gesture information processing method and apparatus, an electronic device, and a storage medium. The method includes: determining an electromyography signal collection target object in a gesture information usage environment; dividing the electromyography signal sample through a sliding window having a fixed window value and a fixed stride into different electromyography signals of the target object, and denoising the different electromyography signals of the target object; recognizing the denoised different electromyography signals, and determining probabilities of gesture information represented by the different electromyography signals; and weighting the probabilities of the gesture information represented by the different electromyography signals, so as to determine gesture information matching the target object.

Claims (84)

1. A gesture information processing method performed by an electronic device, the method comprising:

determining an electromyography signal collection target object in a gesture information usage environment of a gesture recognition model, wherein the gesture recognition model is trained by:

processing a training sample set through the gesture recognition model based on initial parameters of the gesture recognition model;

determining update parameters corresponding to different neural networks of the gesture recognition model;

iteratively updating parameters of the gesture recognition model through the training sample set according to the update parameters corresponding to the different neural networks of the gesture recognition model, so as to recognize different gesture information through the gesture recognition model;

acquiring an electromyography signal sample matching the electromyography signal collection target object, and a corresponding gesture information label;

dividing the electromyography signal sample through a sliding window having a fixed window value and a fixed stride into different electromyography signals of the target object, and denoising the different electromyography signals of the target object;

recognizing the denoised different electromyography signals based on the gesture information label, and determining probabilities of gesture information represented by the different electromyography signals using the gesture recognition model; and

weighting the probabilities of the gesture information represented by the different electromyography signals to determine gesture information matching the target object.

2. The method of claim 1 , wherein the denoising the different electromyography signals comprises:

determining a dynamic noise threshold matching the gesture information usage environment of the gesture recognition model; and

denoising the different electromyography signals according to the dynamic noise threshold to form a dynamic noise electromyography signal matching the dynamic noise threshold.

3. The method of claim 1 , wherein the denoising the different electromyography signals comprises:

determining a fixed noise threshold corresponding to the gesture recognition model, and denoising the different electromyography signals according to the fixed noise threshold, so as to form a fixed noise electromyography signal matching the fixed noise threshold.

4. The method of claim 1 , wherein the method further comprises:

determining features of corresponding electromyography signals when the gesture information usage environment of the gesture recognition model is a powered skeleton;

determining output sizes of different networks in a first neural network model of the gesture recognition model and a quantity of corresponding convolution kernels according to the features of the electromyography signals; and

adjusting quantities of a temporal convolutional layer network, a spatial convolutional layer network, a fully connected layer network and a residual module network of the first neural network model according to features of the powered skeleton, so as to match initialization parameters of the gesture recognition model with the features of the powered skeleton.

5. The method of claim 1 , wherein the training sample set is a second training sample set and the method before processing the training sample set further comprises:

obtaining a first training sample set, wherein the first training sample set comprises at least one group of electromyography signal samples and a corresponding gesture information label;

denoising the first training sample set to form the corresponding second training sample set;

determining the initial parameters of the gesture recognition model.

6. The method of claim 5 , wherein the denoising the first training sample set to form a corresponding second training sample set comprises:

determining a dynamic noise threshold matching the gesture information usage environment of the gesture recognition model; and

denoising the first training sample set according to the dynamic noise threshold to form the second training sample set matching the dynamic noise threshold; or

determining a fixed noise threshold corresponding to the gesture recognition model, and denoising the first training sample set according to the fixed noise threshold to form the second training sample set matching the fixed noise threshold.

7. The method of claim 6 , wherein the method further comprises:

determining a corresponding training sample collection object according to the gesture information usage environment of the gesture recognition model;

acquiring an electromyography signal sample matching the training sample collection object, and a corresponding gesture information label; and

dividing the electromyography signal sample through the sliding window having the fixed window value and the fixed stride to form the first training sample set.

8. The method of claim 5 , wherein the determining the initial parameters of the gesture recognition model comprises:

determining output sizes of different networks of the gesture recognition model and a quantity of corresponding convolution kernels according to features of electromyography signals in the first training sample set; and

adjusting quantities of the different networks of the gesture recognition model including a temporal convolutional layer network, a spatial convolutional layer network, a fully connected layer network and a residual module network according to an application environment of the gesture recognition model, so as to match initialization parameters of the gesture recognition model with the application environment.

9. The method of claim 8 , wherein the method further comprises:

determining a classification loss and a contrastive loss corresponding to the gesture recognition model; and

determining a fusion loss matching the gesture recognition model according to the classification loss and the contrastive loss, wherein the fusion loss is configured to stop training the gesture recognition model when a corresponding convergence condition is reached.

10. The method of claim 9 , wherein the processing the second training sample set through the gesture recognition model based on the initial parameters of the gesture recognition model, and determining update parameters corresponding to different neural networks of the gesture recognition model comprises:

keeping the initial parameters of the gesture recognition model unchanged, and determining output results of the gesture recognition model;

substituting, in response to the output results of the gesture recognition model, different training samples in the second training sample set into the fusion loss matching the gesture recognition model; and

determining parameters corresponding to different networks in the gesture recognition model when the fusion loss meets the convergence condition as update parameters of the gesture recognition model.

11. The method of claim 9 , wherein the iteratively updating parameters of the gesture recognition model through the second training sample set according to the update parameters corresponding to the different neural networks of the gesture recognition model comprises:

iteratively updating parameters of the different neural networks of the gesture recognition model respectively based on the second training sample set and according to the update parameters corresponding to the different neural networks of the gesture recognition model, until losses corresponding to the different neural networks of the gesture recognition model meet the corresponding convergence condition.

12. An electronic device, comprising:

a memory configured to store executable instructions; and

a processor configured to run the executable instructions stored in the memory to perform a gesture information processing method including:

determining an electromyography signal collection target object in a gesture information usage environment of a gesture recognition model, wherein the gesture recognition model is trained by:

processing a training sample set through the gesture recognition model based on initial parameters of the gesture recognition model;

determining update parameters corresponding to different neural networks of the gesture recognition model;

iteratively updating parameters of the gesture recognition model through the training sample set according to the update parameters corresponding to the different neural networks of the gesture recognition model, so as to recognize different gesture information through the gesture recognition model;

acquiring an electromyography signal sample matching the electromyography signal collection target object, and a corresponding gesture information label;

dividing the electromyography signal sample through a sliding window having a fixed window value and a fixed stride into different electromyography signals of the target object, and denoising the different electromyography signals of the target object;

recognizing the denoised different electromyography signals based on the gesture information label, and determining probabilities of gesture information represented by the different electromyography signals using the gesture recognition model; and

weighting the probabilities of the gesture information represented by the different electromyography signals to determine gesture information matching the target object.

13. The electronic device of claim 12 , wherein the denoising the different electromyography signals comprises:

determining a dynamic noise threshold matching the gesture information usage environment of the gesture recognition model; and

denoising the different electromyography signals according to the dynamic noise threshold to form a dynamic noise electromyography signal matching the dynamic noise threshold.

14. The electronic device of claim 12 , wherein the denoising the different electromyography signals comprises:

determining a fixed noise threshold corresponding to the gesture recognition model, and denoising the different electromyography signals according to the fixed noise threshold, so as to form a fixed noise electromyography signal matching the fixed noise threshold.

15. The electronic device of claim 12 , wherein the information processing method further comprises:

determining features of corresponding electromyography signals when the gesture information usage environment of the gesture recognition model is a powered skeleton;

determining output sizes of different networks in a first neural network model of the gesture recognition model and a quantity of corresponding convolution kernels according to the features of the electromyography signals; and

adjusting quantities of a temporal convolutional layer network, a spatial convolutional layer network, a fully connected layer network and a residual module network of the first neural network model according to features of the powered skeleton, so as to match initialization parameters of the gesture recognition model with the features of the powered skeleton.

16. The electronic device of claim 12 , wherein the training sample set is a second training sample set and the information processing method before processing the training sample set further comprises:

obtaining a first training sample set, wherein the first training sample set comprises at least one group of electromyography signal samples and a corresponding gesture information label;

denoising the first training sample set to form the corresponding second training sample set;

determining initial parameters of the gesture recognition model.

17. A non-transitory computer-readable storage medium, storing executable instructions that, when executed by a processor of an electronic device, cause the electronic device to perform a gesture information processing method including:

determining an electromyography signal collection target object in a gesture information usage environment of a gesture recognition model, wherein the gesture recognition model is trained by:

processing a training sample set through the gesture recognition model based on initial parameters of the gesture recognition model;

determining update parameters corresponding to different neural networks of the gesture recognition model;

iteratively updating parameters of the gesture recognition model through the training sample set according to the update parameters corresponding to the different neural networks of the gesture recognition model, so as to recognize different gesture information through the gesture recognition model;

acquiring an electromyography signal sample matching the electromyography signal collection target object, and a corresponding gesture information label;

dividing the electromyography signal sample through a sliding window having a fixed window value and a fixed stride into different electromyography signals of the target object, and denoising the different electromyography signals of the target object;

recognizing the denoised different electromyography signals based on the gesture information label, and determining probabilities of gesture information represented by the different electromyography signals using the gesture recognition model; and

weighting the probabilities of the gesture information represented by the different electromyography signals to determine gesture information matching the target object.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the denoising the different electromyography signals comprises:

determining a dynamic noise threshold matching the gesture information usage environment of the gesture recognition model; and

denoising the different electromyography signals according to the dynamic noise threshold to form a dynamic noise electromyography signal matching the dynamic noise threshold.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the denoising the different electromyography signals comprises:

determining a fixed noise threshold corresponding to the gesture recognition model, and denoising the different electromyography signals according to the fixed noise threshold, so as to form a fixed noise electromyography signal matching the fixed noise threshold.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the training sample set is a second training sample set and the information processing method before processing the training sample set further comprises:

obtaining a first training sample set, wherein the first training sample set comprises at least one group of electromyography signal samples and a corresponding gesture information label;

denoising the first training sample set to form the corresponding second training sample set;

determining the initial parameters of the gesture recognition model.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 26, 2022
From: HONG, XIAOLIN; ZHENG, QINGQING; WANG, XINMIN; MA, KAI; ZHENG, YEFENG
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 060024/0452 →
Priority Claims (1)
CN 202010033904.X · Jan 13, 2020 · national
Continuity (2)
Continuation PCTCN2020130567 · Nov 20, 2020
Related Publication 20220147151A1 · May 12, 2022
Cited By (1)
US 12,422,935