Granted Patent B2

US 11,340,328 · App. 17/214,838 · Granted May 24, 2022

Inventors: Fengyu Wang (College Park, MD); Beibei Wang (Clarksville, MD); Xiaolu Zeng (Greenbelt, MD); Chenshu Wu (Hong Kong, CN); K. J. Ray Liu (Potomac, MD); Oscar Chi-Lim Au (San Jose, CA)

Assignee: ORIGIN WIRELESS, INC.

G01S7/006G01S7/415G01S13/003G01S13/56H04W84/18

Abstract

Claims (145)

1. A system for wireless monitoring, comprising:

a transmitter configured for transmitting, using N 1 transmit antennas, a first wireless signal through a wireless channel of a venue;

a receiver configured for receiving, using N 2 receive antennas, a second wireless signal through the wireless channel, wherein N 1 and N 2 are positive integers, wherein the second wireless signal comprises a reflection of the first wireless signal by at least one living being having at least one repetitive motion in the venue; and

a processor configured for:

obtaining a plurality of time series of channel information (TSCI) of the wireless channel based on the second wireless signal, wherein each of the plurality of TSCI is associated with a respective transmit antenna of the transmitter and a respective receive antenna of the receiver,

generating, for each living being of the at least one living being, a vital signal representing all repetitive motions of the living being based on the plurality of TSCI,

extracting, from the vital signal of each living being, a heartbeat signal, wherein extracting the heartbeat signal comprises decomposing, for each living being, a CI phase signal associated with the living being to generate the heartbeat signal based on a joint optimization of a decomposition of the CI phase signal, wherein the joint optimization comprises:

determining a number K that represents a quantity of possible signal components of the CI phase signal, wherein K is larger than or equal to a quantity of living beings in the venue,

determining a tradeoff factor for balancing bandwidth constraint and data fidelity,

iteratively optimizing, based on the tradeoff factor, K signal components of the CI phase signal and center frequencies of the K signal components, based on an objective function that maximizes spectrum sparsity and data fidelity of the CI phase signal at the same time, until a convergence of the objective function, and

concurrently generating K decomposed components of the CI phase signal based on the iteratively optimizing, and

monitoring, for each living being in the venue, a heart rate variability based on the heartbeat signal.

2. The system of claim 1 , wherein:

the at least one living being comprises: a human being or an animal;

the first wireless signal is carried on a millimeter wave;

each object in the venue has a location determined based on a plurality of spatial bins in the venue;

each of the plurality of spatial bins is determined by: a direction and a distance range originating from the receiver; and

each direction is associated with at least one of: an angle, an azimuth angle, or an elevation angle.

3. The system of claim 2 , wherein generating the vital signal for each living being comprises:

computing a beamforming based on the plurality of TSCI; and

computing a set of time series of directional channel information (CI) each associated with a direction based on the beamforming.

4. The system of claim 3 , wherein generating the vital signal for each living being further comprises:

for each directional CI associated with a respective direction,

computing, for each time instance, a CI amplitude based on the directional CI for the respective direction and a distance range to obtain CI amplitudes over time,

computing a time average of the CI amplitudes based on a time window, and

detecting object presence at the distance range in the respective direction when the time average is greater than a first threshold; and

determining a set of direction-of-interest's (DoIs) each comprising a direction in which object presence is detected.

5. The system of claim 4 , wherein the first threshold is determined adaptively to filter the CI amplitudes at the distance range in the respective direction based on a two-dimensional constant false alarm rate (CFAR).

6. The system of claim 4 , wherein generating the vital signal for each living being further comprises:

for each DoI in the set of DoIs and for each distance range,

computing a phase variance of a directional CI associated with the DoI over time in a time window,

classifying a detected object at the distance range and the DoI as a static object without repetitive motion, when the phase variance is less than a second threshold, and

classifying a detected object at the distance range and the DoI as a living being with repetitive motion, when the phase variance is greater than or equal to the second threshold.

7. The system of claim 6 , wherein generating the vital signal for each living being further comprises:

determining a plurality of target spatial bins for each detected living being, wherein each of the plurality of target spatial bins is determined by a target DoI and a target distance range; and

for each target spatial bin,

computing an auto-correlation function based on the directional CI associated with the target spatial bin,

determining a first peak of the auto-correlation function,

classifying a motion of the detected living being at the target spatial bin as a repetitive motion when the first peak is greater than a third threshold, and

classifying a motion of the detected living being at the target spatial bin as a random body motion when the first peak is less than or equal to the third threshold.

8. The system of claim 7 , wherein generating the vital signal for each living being further comprises:

computing a set of point-of-interest's (PoIs), wherein each PoI in the set of PoIs is associated with a detected living being at the PoI and is a target spatial bin in the venue associated with a repetitive motion of the detected living being at the PoI;

clustering the set of PoIs to generate at least one PoI cluster with a total cluster number, wherein:

the set of PoIs are clustered without prior knowledge of the total cluster number after clustering, and

the set of PoIs are clustered based on at least one of: a density related to the set of PoIs, a distance between any two PoIs of the set of PoIs, or a threshold related to a size of a living being; and

determining a quantity of target living beings in the venue based on the total cluster number.

9. The system of claim 8 , wherein generating the vital signal for each living being further comprises:

for each of the at least one PoI cluster,

combining PoIs in the PoI cluster to generate a combined PoI based on at least one of: a weighted average of CI phases measured at the PoIs, or a dominant PoI having a highest peak among the first peaks of the auto-correlation functions associated with the PoIs, and

generating a vital signal for a target living being corresponding to the PoI cluster based on the CI phase signal corresponding to the combined PoI, wherein the CI phase signal is associated with all repetitive motions of the target living being.

10. The system of claim 1 , wherein the joint optimization further comprises:

determining whether the K decomposed components comprise a heartbeat component, which has an amplitude located within a first value range and has a center frequency located within a second value range, wherein each of the first value range and the second value range is predetermined based on heartbeat statistics.

11. The system of claim 10 , wherein the joint optimization further comprises:

when there is a heartbeat component in the K decomposed components,

estimating an envelope of the heartbeat component based on at least one of:

smoothing, low pass filtering, spline interpolation, B-spline, cubic spline interpolation, polynomial fitting, or moving average, and

normalizing the heartbeat component by dividing the heartbeat component with the envelope of the heartbeat component to generate a normalized heartbeat signal for the living being.

12. The system of claim 10 , wherein the joint optimization further comprises:

when there is no heartbeat component in the K decomposed components,

updating the number K to generate an updated number K′ to represent an updated quantity of possible signal components of the CI phase signal,

updating the tradeoff factor to generate an updated tradeoff factor for balancing bandwidth constraint and data fidelity, and

iteratively optimizing, based on the updated tradeoff factor, K′ signal components of the CI phase signal and center frequencies of the K′ signal components, based on the objective function that maximizes spectrum sparsity and data fidelity of the CI phase signal at the same time, until a convergence of the objective function, to concurrently generate K′ decomposed components of the CI phase signal.

13. The system of claim 1 , wherein monitoring the heart rate variability further comprises:

for each living being,

determining a heartbeat time for the living being at each time instance to compute heartbeat times, based on at least one of:

identifying peaks of the heartbeat signal,

identifying zero-crossing points of the heartbeat signal, or

performing continuous wavelet transform on the heartbeat signal;

calculating a plurality of inter-beat intervals based on the heartbeat times; and

estimating the heart rate variability for the living being based on statistics of the inter-beat intervals.

14. The system of claim 2 , wherein extracting a heartbeat signal from the vital signal comprises generating, for each living being, the heartbeat signal based on at least one of:

a decomposition of the vital signal based on frequency components of the vital signal; or

a decomposition of a CI amplitude signal associated with the living being.

15. The system of claim 1 , wherein the transmitter and the receiver are physically coupled to each other.

16. A wireless device of a wireless monitoring system, comprising:

a processor;

a memory communicatively coupled to the processor; and

a receiver communicatively coupled to the processor, wherein:

an additional wireless device of the wireless monitoring system is configured for transmitting a first wireless signal through a wireless channel of a venue,

the receiver is configured for receiving a second wireless signal through the wireless channel,

the second wireless signal comprises a reflection of the first wireless signal by at least one living being having at least one repetitive motion in the venue, and

the processor is configured for:

obtaining a time series of channel information (TSCI) of the wireless channel based on the second wireless signal,

generating, for each living being of the at least one living being, a vital signal representing all repetitive motions of the living being based on the TSCI,

decomposing, for each living being, a CI phase signal associated with the living being to generate a heartbeat signal based on a joint optimization of a decomposition of the CI phase signal, wherein the joint optimization comprises:

determining a number K that represents a quantity of possible signal components of the CI phase signal, wherein K is larger than or equal to a quantity of living beings in the venue,

determining a tradeoff factor for balancing bandwidth constraint and data fidelity,

iteratively optimizing, based on the tradeoff factor, K signal components of the CI phase signal and center frequencies of the K signal components, based on an objective function that maximizes spectrum sparsity and data fidelity of the CI phase signal at the same time, until a convergence of the objective function, and

concurrently generating K decomposed components of the CI phase signal based on the iteratively optimizing, and

monitoring, for each living being in the venue, a heart rate variability based on the heartbeat signal.

17. The wireless device of claim 16 , wherein:

the at least one living being comprises: a human being or an animal;

the first wireless signal is carried on a millimeter wave;

each object in the venue has a location determined based on a plurality of spatial bins in the venue;

each of the plurality of spatial bins is determined by: a direction and a distance range originating from the receiver; and

each direction is associated with at least one of: an angle, an azimuth angle, or an elevation angle.

18. The wireless device of claim 17 , wherein generating the vital signal for each living being comprises:

computing a beamforming based on the TSCI;

computing a set of time series of directional channel information (CI) each associated with a direction based on the beamforming;

for each directional CI associated with a respective direction,

computing, for each time instance, a CI amplitude based on the directional CI for the respective direction and a distance range to obtain CI amplitudes over time,

computing a time average of the CI amplitudes based on a time window, and

detecting object presence at the distance range in the respective direction when the time average is greater than a first threshold, wherein the first threshold is determined adaptively to filter the CI amplitudes at the distance range in the respective direction based on a two-dimensional constant false alarm rate (CFAR); and

determining a set of direction-of-interest's (DoIs) each comprising a direction in which object presence is detected.

19. The wireless device of claim 18 , wherein generating the vital signal for each living being further comprises:

for each DoI in the set of DoIs and for each distance range,

computing a phase variance of a directional CI associated with the DoI over time in a time window,

classifying a detected object at the distance range and the DoI as a static object without repetitive motion, when the phase variance is less than a second threshold, and

classifying a detected object at the distance range and the DoI as a living being with repetitive motion, when the phase variance is greater than or equal to the second threshold;

determining a plurality of target spatial bins for each detected living being, wherein each of the plurality of target spatial bins is determined by a target DoI and a target distance range; and

for each target spatial bin,

computing an auto-correlation function based on the directional CI associated with the target spatial bin,

determining a first peak of the auto-correlation function,

classifying a motion of the detected living being at the target spatial bin as a repetitive motion when the first peak is greater than a third threshold, and

classifying a motion of the detected living being at the target spatial bin as a random body motion when the first peak is less than or equal to the third threshold.

20. The wireless device of claim 19 , wherein generating the vital signal for each living being further comprises:

computing a set of point-of-interest's (PoIs), wherein each PoI in the set of PoIs is associated with a detected living being at the PoI and is a target spatial bin in the venue associated with a repetitive motion of the detected living being at the PoI;

clustering the set of PoIs to generate at least one PoI cluster with a total cluster number;

determining a quantity of target living beings in the venue based on the total cluster number; and

for each of the at least one PoI cluster,

combining PoIs in the PoI cluster to generate a combined PoI based on at least one of: a weighted average of CI phases measured at the PoIs, or a dominant PoI having a highest peak among the first peaks of the auto-correlation functions associated with the PoIs, and

generating a vital signal for a target living being corresponding to the PoI cluster based on a CI phase signal corresponding to the combined PoI, wherein the CI phase signal is associated with all repetitive motions of the target living being.

21. The wireless device of claim 16 , wherein the joint optimization further comprises:

determining whether the K decomposed components comprise a heartbeat component, which has an amplitude located within a first value range and has a center frequency located within a second value range, wherein each of the first value range and the second value range is predetermined based on heartbeat statistics.

22. The wireless device of claim 21 , wherein the joint optimization further comprises:

when there is a heartbeat component in the K decomposed components,

estimating an envelope of the heartbeat component based on at least one of:

smoothing, low pass filtering, spline interpolation, B-spline, cubic spline interpolation, polynomial fitting, or moving average, and

normalizing the heartbeat component by dividing the heartbeat component with the envelope of the heartbeat component to generate a normalized heartbeat signal for the living being.

23. The wireless device of claim 21 , wherein the joint optimization further comprises:

when there is no heartbeat component in the K decomposed components,

updating the number K to generate an updated number K′ to represent an updated quantity of possible signal components of the CI phase signal,

updating the tradeoff factor to generate an updated tradeoff factor for balancing bandwidth constraint and data fidelity, and

iteratively optimizing, based on the updated tradeoff factor, K′ signal components of the CI phase signal and center frequencies of the K′ signal components, based on the objective function that maximizes spectrum sparsity and data fidelity of the CI phase signal at the same time, until a convergence of the objective function, to concurrently generate K′ decomposed components of the CI phase signal.

24. A method of a wireless monitoring system, comprising:

transmitting a first wireless signal through a wireless channel of a venue;

receiving a second wireless signal through the wireless channel, wherein the second wireless signal comprises a reflection of the first wireless signal by a plurality of human beings in the venue;

obtaining a time series of channel information (TSCI) of the wireless channel based on the second wireless signal, wherein each CI comprises at least one of: a channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI);

generating, for each of the plurality of human beings, a vital signal representing all repetitive motions of the human being based on the TSCI;

decomposing, for each living being, a CI phase signal associated with the living being to generate a heartbeat signal based on a joint optimization of a decomposition of the CI phase signal, wherein the joint optimization comprises:

determining a number K that represents a quantity of possible signal components of the CI phase signal, wherein K is larger than or equal to a quantity of living beings in the venue,

determining a tradeoff factor for balancing bandwidth constraint and data fidelity,

iteratively optimizing, based on the tradeoff factor, K signal components of the CI phase signal and center frequencies of the K signal components, based on an objective function that maximizes spectrum sparsity and data fidelity of the CI phase signal at the same time, until a convergence of the objective function, and

concurrently generating K decomposed components of the CI phase signal based on the iteratively optimizing; and

simultaneously monitoring, for each of the plurality of human beings, a heart rate variability based on the heartbeat signal.

Assignments (2)

Continuity (42)

Cited By (1)

US 12,474,433