Dynamic gesture recognition using mmWave radar

1. A method comprising:

triggering an electronic device to activate a gesture recognition mode in response to detecting that a proximity condition for activating the gesture recognition mode is satisfied and to deactivate the gesture recognition mode when the proximity condition is not satisfied;

obtaining radar data while the gesture recognition mode is activated, wherein the radar data includes time-velocity data (TVD);

after a determination from a binary classifier that the obtained radar data includes gesture activity, detecting an end of a gesture based on the TVD of the obtained radar data;

to classify the gesture, determining a gesture, from among a set of gestures, that corresponds to a portion of the TVD between a start and the end of the gesture; and

outputting an event indicator indicating that a user of the electronic device performed the gesture classified.

2. The method of claim 1 , further comprising:

activating the gesture recognition mode by activating a gating mechanism to determine whether the obtained radar data corresponds to non-gesture activities or gesture-related activities;

in response to determining that the radar data corresponds to gesture activities, activating an activity detection module (ADM) to detect the end of the gesture based on information extracted from the TVD; and

in response to determining that the radar data corresponds to non-gesture activities, deactivating or maintaining the ADM in a deactivated state.

3. The method of claim 2 , further comprising determining, using the activated gating mechanism, whether the obtained radar data corresponds to non-gesture activities or gesture-related activities based on at least one of:

a first peak detection;

a variation in range during the gesture;

average TVD;

a persistent increase in the TVD;

a burst detection;

an elevation angle; and

an azimuth angle.

4. The method of claim 1 , wherein:

detecting the end of the gesture further comprises detecting, using an activity detection module (ADM), the end of a gesture based on information extracted from the TVD, and

the information extracted from the TVD includes low-dimensional Doppler features.

5. The method of claim 1 , wherein:

Detecting the end of the gesture further comprises detecting, using an activity detection module (ADM), the end of a gesture based on information extracted from the TVD, and

the information extracted from the TVD includes burst information.

6. The method of claim 5 , wherein detecting the end of the gesture further comprises:

determining whether a burst is detected based on a level of movement within a current frame of radar transmission;

determining that the current frame is not the end of a gesture in response to a determination that the burst is detected within the current frame; and

determining that the current frame is the end of a gesture in response to a determination that the burst is not detected within a threshold number of consecutive frames including the current frame.

7. The method of claim 1 , further comprising:

classifying the gesture using a gesture classifier that includes a convolutional neural network-based (CNN-based) architecture that is shift-invariant,

wherein the CNN-based architecture includes one or more blur pool layers.

8. An electronic device comprising:

a transceiver; and

a processor operatively connected to the transceiver, the processor configured to:

trigger the electronic device to activate a gesture recognition mode in response to detecting that a proximity condition for activating the gesture recognition mode is satisfied and to deactivate the gesture recognition mode when the proximity condition is not satisfied;

obtain radar data while the gesture recognition mode is activated, wherein the radar data includes time-velocity data (TVD);

after a determination from a binary classifier that the obtained radar data includes gesture activity, detect an end of a gesture based on the TVD of the obtained radar data;

to classify the gesture, determine a gesture, from among a set of gesture, that corresponds to a portion of the TVD between a start and the end of the gesture; and

output an event indicator indicating that a user of the electronic device performed the gesture classified.

9. The electronic device of claim 8 , wherein:

to activate the gesture recognition mode, the processor is further configured to activate a gating mechanism to determine whether the obtained radar data corresponds to non-gesture activities or gesture-related activities;

the processor is further configured to:

in response to determining that the radar data corresponds to gesture activities, activate an activity detection module (ADM) to detect the end of the gesture based on information extracted from the TVD; and

in response to determining that the radar data corresponds to non-gesture activities, deactivate or maintain the ADM in a deactivated state.

10. The electronic device of claim 9 , the processor is further configured to determine, using the activated gating mechanism, whether the obtained radar data corresponds to non-gesture activities or gesture-related activities based on at least one of:

a first peak detection;

a variation in range during the gesture;

average TVD;

a persistent increase in the TVD;

a burst detection;

an elevation angle; and

an azimuth angle.

11. The electronic device of claim 8 , wherein:

to detect the end of the gesture, the processor is further configured to detect, using an activity detection module (ADM), the end of a gesture based on information extracted from the TVD, and

the information extracted from the TVD includes low-dimensional Doppler features.

12. The electronic device of claim 8 , wherein:

to detect the end of the gesture, the processor is further configured to detect, using an activity detection module (ADM), the end of a gesture based on information extracted from the TVD, and

the information extracted from the TVD includes burst information.

13. The electronic device of claim 12 , wherein to detect the end of the gesture, the processor is further configured to:

determine whether a burst is detected based on a level of movement within a current frame of radar transmission;

determine that the current frame is not the end of a gesture in response to a determination that the burst is detected within the current frame; and

determine that the current frame is the end of a gesture in response to a determination that the burst is not detected within a threshold number of consecutive frames including the current frame.

14. The electronic device of claim 8 , wherein the processor is further configured to:

classify the gesture using a gesture classifier that includes a convolutional neural network-based (CNN-based) architecture that is shift-invariant, and

wherein the CNN-based architecture includes one or more blur pool layers.

15. A non-transitory, computer readable medium embodying a computer program, the computer program comprising program code that, when executed by a processor of an electronic device, causes the electronic device to:

trigger a gesture recognition mode in response to detecting that a proximity condition for activating the gesture recognition mode is satisfied and to deactivate the gesture recognition mode when the proximity condition is not satisfied;

obtain radar data while the gesture recognition mode is activated, wherein the radar data includes time-velocity data (TVD);

after a determination from a binary classifier that the obtained radar data includes gesture activity, detect an end of a gesture based on the TVD of the obtained radar data;

to classify the gesture, determine a gesture, from among a set of gesture, that corresponds to a portion of the TVD between a start and the end of the gesture; and

output an event indicator indicating that a user of the electronic device performed the gesture classified.

16. The non-transitory computer readable medium of claim 15 , wherein:

the program code that, when executed, causes the electronic device to activate the gesture recognition mode further comprises program code that, when executed, causes the electronic device to activate a gating mechanism to determine whether the obtained radar data corresponds to non-gesture activities or gesture-related activities; and

the computer program further comprises program code that, when executed, causes the electronic device to:

in response to determining that the radar data corresponds to gesture activities, activate an activity detection module (ADM) to detect the end of the gesture based on information extracted from the TVD; and

in response to determining that the radar data corresponds to non-gesture activities, deactivate or maintain the ADM in a deactivated state.

17. The non-transitory computer readable medium of claim 15 , wherein:

the program code that, when executed, causes the electronic device to detect the end of the gesture further comprises program code that, when executed, causes the electronic device to detect, using an activity detection module (ADM), the end of a gesture based on information extracted from the TVD, and

the information extracted from the TVD includes low-dimensional Doppler features.

18. The non-transitory computer readable medium of claim 15 , wherein:

the program code that, when executed, causes the electronic device to detect the end of the gesture further comprises program code that, when executed, causes the electronic device to detect, using an activity detection module (ADM), the end of a gesture based on information extracted from the TVD, and

the information extracted from the TVD includes burst information.

19. The non-transitory computer readable medium of claim 18 , wherein the program code that, when executed, causes the electronic device to detect the end of the gesture further comprises program code that when executed causes the electronic device to:

determine whether a burst is detected based on a level of movement within a current frame of radar transmission;

determine that the current frame is not the end of a gesture in response to a determination that the burst is detected within the current frame; and

determine that the current frame is the end of a gesture in response to a determination that the burst is not detected within a threshold number of consecutive frames including the current frame.

20. The non-transitory computer readable medium of claim 15 , further comprising program code that, when executed, causes the electronic device to:

classify the gesture using a gesture classifier that includes a convolutional neural network-based (CNN-based) architecture that is shift-invariant, and

wherein the CNN-based architecture includes one or more blur pool layers.