IP Library › Granted Patent US 11,567,580
Granted Patent B2
US 11,567,580 · App. 17/158,794 · Granted Jan 31, 2023

Adaptive thresholding and noise reduction for radar data

Inventors: Khuong N. Nguyen (Frisco, TX); Wenxun Qiu (Allen, TX); Vutha Va (Plano, TX); Boon Loong Ng (Plano, TX)
Assignee: Samsung Electronics Co., Ltd.
G06F3/017G01S7/415G01S7/417G06N3/0445G06N3/0454
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,567,580
App. No.
17/158,794
Granted
Jan 31, 2023
Kind
B2
Abstract

An electronic device for gesture recognition, includes a processor operably connected to a transceiver. The transceiver is configured to transmit and receive signals for measuring range and speed. The processor is configured to transmit the signals, via the transceiver. in response to a determination that a triggering event occurred, the processor is configured to track movement of an object relative to the electronic device within a region of interest based on reflections of the signals received by the transceiver to identify range measurements and speed measurements associated with the object. The processor is also configured to identify features from the reflected signals, based on at least one of the range measurements and the speed measurements. The processor is further configured to identify a gesture based in part on the features from the reflected signals. Additionally, the processor is configured to perform an action indicated by the gesture.

Claims (107)

1. An electronic device for gesture recognition, the electronic device comprising:

a transceiver configured to transmit and receive signals for measuring range and speed; and

a processor operably connected to the transceiver and configured to:

transmit the signals, via the transceiver,

in response to a determination that a triggering event occurred, track movement of an object relative to the electronic device within a region of interest based on reflections of the signals received by the transceiver to identify range measurements and speed measurements associated with the object,

identify features from the reflected signals, based on at least one of the range measurements and the speed measurements,

perform an adaptive thresholding to remove a portion of the identified features,

identify a gesture based in part on a portion of the features that were not removed from the reflected signals, and

perform an action indicated by the gesture.

2. The electronic device of claim 1 , the processor is further configured to:

determine whether the triggering event occurred based on a detection of: (i) a gesture including at least one repetition in a feature domain, (ii) a physical touching of the electronic device, or (iii) a connection to another device; and

transmit, via the transceiver, signature signals to identify the region of interest associated with the movement of the object.

3. The electronic device of claim 1 , wherein to identify the gesture, the processor is configured to:

perform the adaptive thresholding to generate a clipped map;

normalize the clipped map; and

identify the gesture from the normalized map.

4. The electronic device of claim 3 , wherein to perform the adaptive thresholding, the processor is configured to:

identify an upper threshold and a lower threshold from the identified features in a time velocity diagram (TVD), wherein the TVD is represented as a one dimensional array and indicates a radial velocity between the transceiver and the object that is moving;

generate a threshold window based on the upper threshold and the lower threshold; and

remove the portion of the identified features based on the threshold window to generate the clipped map, wherein the clipped map corresponds to the region of interest of the identified features,

wherein the adaptive thresholding removes noise from the reflected signals.

5. The electronic device of claim 3 , wherein the processor is further configured to apply a low pass filter to remove noise from the normalized map.

6. The electronic device of claim 1 , wherein the processor is further configured to:

identify a compensation coefficient associated with a delay tap, the compensation coefficient represents variations in a channel impulse response (CIR);

modify the reflected signals to compensate for changes associated with the CIR based on the compensation coefficient; and

identify the features from the modified reflected signals.

7. The electronic device of claim 1 , wherein:

the reflected signals represent a data frame including a burst dimension, a channel dimension, a repetition dimension, and a tap dimension, and

to identify the features from the reflected signals, the processor is further configured to generate a range Doppler map (RDM) representing the reflected signals in two dimensions.

8. The electronic device of claim 7 , wherein to generate the RDM, the processor is configured to:

remove a zero doppler value from the burst dimension of the data frame;

apply a Fast Fourier Transform to the burst dimension;

convert the reflected signals included in the data frame to a decibel scale;

remove a first tap and a second tap from the tap dimension;

extract a portion of bursts along the burst dimension; and

average the data frame over the channel dimension to generate the RDM in two-dimensions,

wherein the RDM is based in part on the portion of bursts along the burst dimension and remaining taps after the first tap and the second tap are removed.

9. The electronic device of claim 1 , further comprising a neural network that includes a Gaussian pooling layer,

wherein the Gaussian Pooling layer is configured to blur and down-sample the reflected signals associated with the object, and

wherein the processor is further configured to identify the gesture using the neural network based on the blurred and down-sampled reflected signals.

10. The electronic device of claim 1 , further comprising convolutional recurrent neural network that includes a convolutional neural network and recurrent neural network,

wherein the convolutional neural network is configured to extract spatial relationships from a RDM representing the reflected signals in two dimensions,

wherein the recurrent neural network is configured to identify temporal relationships between the reflected signals in a time series, and

wherein the processor is further configured to identify the gesture, based on the extracted spatial relationships and the identified temporal relationships.

11. A method for gesture recognition, the method comprising:

transmitting signals, via a transceiver;

in response to a determination that a triggering event occurred, tracking movement of an object relative to an electronic device within a region of interest based on reflections of the signals received by the transceiver to identify range measurements and speed measurements associated with the object;

identifying features from the reflected signals, based on at least one of the range measurements and the speed measurements;

performing an adaptive thresholding to remove a portion of the identified features

identifying a gesture based in part on a portion of the features that were not removed from the reflected signals; and

performing an action indicated by the gesture.

12. The method of claim 11 , further comprising:

determine whether the triggering event occurred based on a detection of: (i) a gesture including at least one repetition in a feature domain, (ii) a physical touching of the electronic device, or (iii) a connection to another device; and

transmitting, via the transceiver, signature signals to identify the region of interest associated with the movement of the object.

13. The method of claim 11 , wherein identifying the gesture comprises:

performing the adaptive thresholding to generate a clipped map;

normalizing the clipped map; and

identifying the gesture from the normalized map.

14. The method of claim 13 , wherein performing the adaptive thresholding comprises:

identifying an upper threshold and a lower threshold from the identified features in a time velocity diagram (TVD), wherein the TVD is represented as a one dimensional array and indicates a radial velocity between the transceiver and the object that is moving;

generating a threshold window based on the upper threshold and the lower threshold; and

removing the portion of the identified features based on the threshold window to generate the clipped map, wherein the clipped map corresponds to the region of interest of the identified features,

wherein the adaptive thresholding removes noise from the reflected signals.

15. The method of claim 13 , further comprising applying a low pass filter to remove noise from the normalized map.

16. The method of claim 11 , further comprising:

identifying a compensation coefficient associated with a delay tap, the compensation coefficient represents variations in a channel impulse response (CIR);

modifying the reflected signals to compensate for changes associated with the CIR based on the compensation coefficient; and

identifying the features from the modified reflected signals.

17. The method of claim 11 , wherein:

the reflected signals represent a data frame including a burst dimension, a channel dimension, a repetition dimension, and a tap dimension, and

identifying the features from the reflected signals, comprises generating a range Doppler map (RDM) representing the reflected signals in two dimensions.

18. The method of claim 17 , wherein generating the RDM, comprises:

removing a zero doppler value from the burst dimension of the data frame;

applying a Fast Fourier Transform to the burst dimension;

converting the reflected signals included in the data frame to a decibel scale;

removing a first tap and a second tap from the tap dimension, wherein the first tap is an initial tap in the tap dimension and the second tap is a last tap in the tap dimension;

extracting a portion of bursts along the burst dimension; and

averaging the data frame over the channel dimension to generate the RDM in two-dimensions,

wherein the RDM is based in part on the portion of bursts along the burst dimension and remaining taps after the first tap and the second tap are removed.

19. The method of claim 11 , further comprising:

blurring and down-sampling the reflected signals associated with the object using a neural network that includes a Gaussian pooling layer, and

identifying the gesture using the neural network based on the blurred and down-sampled reflected signals.

20. The method of claim 11 , further comprising:

extracting spatial relationships from a RDM representing the reflected signals in two dimensions, using a convolutional neural network;

identifying temporal relationships between the reflected signals in a time series using a recurrent neural network; and

identifying the gesture, based on the extracted spatial relationships and the identified temporal relationships.

21. A non-transitory computer readable medium embodying a computer program, the computer program comprising computer readable program code that, when executed by a processor of an electronic device, causes the processor to:

transmit signals, via a transceiver;

in response to a determination that a triggering event occurred, track movement of an object relative to the electronic device within a region of interest based on reflections of the signals received by the transceiver to identify range measurements and speed measurements associated with the object;

identify features from the reflected signals, based on at least one of the range measurements and the speed measurements;

perform an adaptive thresholding to remove a portion of the identified features;

identify a gesture based in part on a portion of the features that were not removed from the reflected signals; and

perform an action indicated by the gesture.

22. The non-transitory computer readable medium of claim 21 , wherein the computer readable program code, when executed by the processor, further causes the processor to:

perform the adaptive thresholding to generate a clipped map;

normalize the clipped map; and

identify the gesture from the normalized map.

23. The non-transitory computer readable medium of claim 22 , wherein the computer readable program code, when executed by the processor, further causes the processor to:

identify an upper threshold and a lower threshold from the identified features in a time velocity diagram (TVD), wherein the TVD is represented as a one dimensional array and indicates a radial velocity between the transceiver and the object that is moving;

generate a threshold window based on the upper threshold and the lower threshold; and

remove the portion of the identified features based on the threshold window to generate the clipped map, wherein the clipped map corresponds to the region of interest of the identified features,

wherein the adaptive thresholding removes noise from the reflected signals.

24. The non-transitory computer readable medium of claim 21 , wherein the computer readable program code, when executed by the processor, further causes the processor to:

identify a compensation coefficient associated with a delay tap, the compensation coefficient represents variations in a channel impulse response (CIR);

modify the reflected signals to compensate for changes associated with the CIR based on the compensation coefficient; and

identify the features from the modified reflected signals.

25. The non-transitory computer readable medium of claim 21 , wherein the computer readable program code, when executed by the processor, further causes the processor to generate a range Doppler map (RDM) representing the reflected signals in two dimensions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2021
From: NGUYEN, KHUONG N.; QIU, WENXUN; VA, VUTHA; NG, BOON LOONG
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 055037/0810 →
Continuity (3)
Provisional Application 62967159 · Jan 29, 2020
Provisional Application 63064653 · Aug 12, 2020
Related Publication 20210232228A1 · Jul 29, 2021
Cited By (3)
US 12,474,782 US 12,529,777 US 12,560,702