Granted Patent B2

US 11,448,727 · App. 17/492,599 · Granted Sep 20, 2022

Inventors: Muhammed Zahid Ozturk (Beltsville, MD); Chenshu Wu (Hong Kong, CN); Beibei Wang (Clarksville, MD); K. J. Ray Liu (Potomac, MD); Oscar Chi-Lim Au (San Jose, CA)

Assignee: ORIGIN WIRELESS, INC.

G01S7/415G01S7/354G01S7/356G01S13/584H04W4/023

Abstract

Claims (109)

1. A system for human recognition, comprising:

a transmitter configured to transmit a first wireless signal through a wireless channel of a venue;

a receiver configured to receive a second wireless signal through the wireless channel, wherein the second wireless signal comprises a reflection of the first wireless signal by at least one object in the venue; and

a processor configured for:

obtaining a time series of channel information (CI) of the wireless channel based on the second wireless signal,

determining a presence of a person moving in the venue based on the time series of CI (TSCI),

extracting at least one gait feature of the person from the TSCI, and

recognizing an identity of the person based on the at least one gait feature, wherein recognizing the identity of the person comprises classifying the person as one of known users or an unknown identity, based on a deep learning model and the at least one gait feature of the person,

wherein the deep learning model is pre-trained to register each of the known users based on a fixed and same size of normalized spectrogram for each user.

2. The system of claim 1 , wherein:

each CI comprises a channel impulse response (CIR) calculated based on frequency modulated carrier wave (FMCW) waveforms;

the TSCI is a function of a time frame and a distance range from the receiver; and

the first wireless signal is carried on a millimeter wave.

3. The system of claim 2 , wherein determining the presence of the person comprises:

calculating a time-range variance signal based on variances of the TSCI for each time frame; and

detecting the presence of the person based on at least one of:

extracting peaks on the time-range variance signal,

extracting outliers on the time-range variance signal, or

comparing, for each distance range from the receiver, the time-range variance signal with a threshold that varies based on the distance range.

4. The system of claim 3 , wherein determining the presence of the person comprises:

calculating a radar spectrogram based on a short-time Fourier transform of the TSCI for each time frame; and

applying a constant false alarm rate (CFAR) detection rule on the radar spectrogram to detect the presence of the person.

5. The system of claim 4 , wherein the processor is further configured for:

estimating a moving trace of the person based on a cost function and a threshold, wherein the cost function is calculated based on at least one of:

a measurement of discrepancy of the time-range variance signal between every two consecutive time frames,

a measurement of discrepancy of the radar spectrogram between every two consecutive time frames,

peak heights in the radar spectrogram,

peak heights in the time-range variance signal, or

a variance of the radar spectrogram.

6. The system of claim 5 , wherein the cost function is dependent on the distance from the receiver.

7. The system of claim 6 , wherein the processor is further configured for:

estimating, for each time frame, a speed of the person based on peaks in the radar spectrogram and a Doppler shift; and

estimating the moving trace of the person based on a cost function calculated based on a measurement of discrepancy of the speed between every two consecutive time frames.

8. The system of claim 6 , wherein the processor is further configured for:

estimating, for each time frame, a speed of the person based on the moving trace of the person;

comparing, for each time frame, the estimated speed of the person with two thresholds to generate comparison results; and

extracting a subset of time frames during which the person is moving towards or away from the receiver, based on the comparison results at each time frame.

9. The system of claim 8 , wherein the processor is further configured for:

extracting a subset of locations of the person at the subset of time frames from the moving trace of the person; and

calculating a normalized spectrogram based on the subset of locations and the subset of time frames, wherein the normalized spectrogram is a function of: a frequency, a time frame and a distance range from the receiver.

10. The system of claim 9 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

extracting steps of the person based on the normalized spectrogram; and

determining an average moving periodicity of the person.

11. The system of claim 10 , wherein the average moving periodicity is determined based on:

calculating an energy function of the normalized spectrogram; and

extracting peak locations of the energy function to determine the average moving periodicity.

12. The system of claim 10 , wherein the average moving periodicity is determined based on:

calculating a Fourier transform of an energy function of the normalized spectrogram; and

extracting peak locations of the Fourier transform to determine the average moving periodicity.

13. The system of claim 10 , wherein the average moving periodicity is determined based on:

calculating an autocorrelation function of an energy function of the normalized spectrogram; and

extracting a location of the first peak of the autocorrelation function to determine the average moving periodicity.

14. The system of claim 10 , wherein the average moving periodicity is determined based on:

calculating an autocorrelation function of the normalized spectrogram; and

determining the average moving periodicity based on the autocorrelation function.

15. The system of claim 10 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

removing, from the extracted steps of the person, each step having a gait duration below a duration threshold and/or each step having a stride length below a length threshold.

16. The system of claim 15 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

calculating a mean or median of parameters of the extracted steps, wherein the parameters include at least one of: the gait duration, the speed or the stride length; and

removing each step that has any of the parameters lower than 75% of the mean or median.

17. The system of claim 16 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

removing, from the extracted steps of the person, first one or more steps after the person starts moving; and

removing, from the extracted steps of the person, last one or more steps before the person stops moving.

18. The system of claim 17 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

determining a moving direction of the person based on the moving trace of the person; and

removing, from the normalized spectrogram, each frequency component that is not part of the moving body of the person based on the moving direction of the person.

19. The system of claim 18 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

reducing dimensionality of distance range components of the normalized spectrogram by downsampling the normalized spectrogram with respect to the distance range from the receiver.

20. The system of claim 19 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

removing, from the normalized spectrogram, each distance range component that corresponds to a distance range less than a threshold from the receiver, when the person is walking away from the receiver.

21. The system of claim 20 , wherein extracting the at least one gait feature of the person from the TSCI comprises:

extracting, from the moving trace of the person, the at least one gait feature of the person based on left components of the normalized spectrogram after reducing dimensionality.

22. The system of claim 21 , wherein the at least one gait feature includes an average and/or variance of at least one of: stride length, gait duration or speed of the person.

23. The system of claim 1 , wherein the processor is further configured for:

registering the person as a new user by training the deep learning model based on the left components of the normalized spectrogram with reduced dimensionality, when the person is classified as an unknown identity.

24. The system of claim 23 , wherein the normalized spectrogram with reduced dimensionality has a same dimensionality as that of each trained user.

25. A wireless device of a human recognition system, comprising:

a processor;

a memory communicatively coupled to the processor; and

a receiver communicatively coupled to the processor, wherein:

an additional wireless device of the human recognition system is configured to transmit a first wireless signal through a wireless channel of a venue,

the receiver is configured to receive a second wireless signal through the wireless channel,

the second wireless signal comprises a reflection of the first wireless signal by at least one object in the venue, and

the processor is configured for:

obtaining a time series of channel information (CI) of the wireless channel based on the second wireless signal,

determining a presence of a person moving in the venue based on the time series of CI (TSCI),

extracting at least one gait feature of the person from the TSCI, and

recognizing an identity of the person based on the at least one gait feature, wherein recognizing the identity of the person comprises classifying the person as one of known users or an unknown identity, based on a deep learning model and the at least one gait feature of the person,

wherein the deep learning model is pre-trained to register each of the known users based on a fixed and same size of normalized spectrogram for each user.

26. The wireless device of claim 25 , wherein:

each CI comprises a channel impulse response (CIR);

the TSCI is a function of a time frame and a distance range from the receiver; and

the first wireless signal is carried on a millimeter wave.

27. The wireless device of claim 26 , wherein determining the presence of the person comprises:

calculating a time-range variance signal based on variances of the TSCI for each time frame; and

detecting the presence of the person based on at least one of:

extracting peaks on the time-range variance signal,

extracting outliers on the time-range variance signal,

comparing, for each distance range from the receiver, the time-range variance signal with a threshold that varies based on the distance range, or

applying a constant false alarm rate (CFAR) detection rule on a radar spectrogram calculated based on a short-time Fourier transform of the TSCI for each time frame.

28. A method of a human recognition system, comprising:

transmitting a first wireless signal through a wireless channel of a venue;

receiving a second wireless signal through the wireless channel, wherein the second wireless signal comprises a reflection of the first wireless signal by at least one object in the venue;

obtaining a time series of channel information (CI) of the wireless channel based on the second wireless signal;

determining a presence of a person moving in the venue based on the time series of CI (TSCI);

extracting at least one gait feature of the person from the TSCI; and

recognizing an identity of the person based on the at least one gait feature,

wherein recognizing the identity of the person comprises classifying the person as one of known users or an unknown identity, based on a deep learning model and the at least one gait feature of the person,

wherein the deep learning model is pre-trained to register each of the known users based on a fixed and same size of normalized spectrogram for each user.

Assignments (2)

Continuity (41)

Cited By (3)

US 12,288,603 US 12,474,443 US 12,541,537