Granted Patent B2

US 11,531,087 · App. 17/180,762 · Granted Dec 20, 2022

Inventors: Yuqian Hu (College Park, MD); Feng Zhang (Greenbelt, MD); Beibei Wang (Clarksville, MD); Chenshu Wu (Hong Kong, CN); K. J. Ray Liu (Potomac, MD); Oscar Chi-Lim Au (San Jose, CA)

Assignee: ORIGIN WIRELESS, INC.

G01S7/415G01S13/003G08B21/043G01S7/417G06N20/00

Abstract

Claims (167)

1. A system for target motion detection, comprising:

a transmitter configured for transmitting a first wireless signal through a wireless multipath channel of a venue;

a receiver configured for receiving a second wireless signal through the wireless multipath channel, wherein the second wireless signal differs from the first wireless signal due to the wireless multipath channel that is impacted by a target motion of an object in the venue; and

a processor configured for:

obtaining a time series of channel information (TSCI) of the wireless multipath channel based on the second wireless signal,

computing a time series of spatial-temporal information (STI) of the object based on the TSCI,

computing, during an offline stage of the system, at least one representative STI template based on a set of training time series of STI (TSSTI), wherein:

each training TSSTI is computed based on a respective training TSCI obtained based on a respective training wireless signal from a respective training transmitter received by a respective training receiver in a respective training venue when a respective training object in the respective training venue undergoes a respective training target motion,

the at least one representative STI template is computed based on cleaning each training TSSTI by segmenting the training TSSTI, based on a mapping between the training TSSTI and an additional training TSSTI, into: a respective initial non-target segment of STI, a respective target segment of STI, and a respective trailing non-target segment of STI, and

segmenting the training TSSTI comprises:

computing a plurality of candidate mappings, wherein each candidate mapping is between a candidate target segment of the training TSSTI and an additional candidate target segment of the additional training TSSTI,

computing a plurality of mapping scores associated with the candidate mappings, wherein each of the plurality of mapping scores is associated with a respective one of the candidate mappings and is normalized based on a length associated with the candidate target segment and the additional candidate target segment, and

computing the mapping based on the plurality of candidate mappings and the associated mapping scores, and

detecting, during an online stage of the system, the target motion of the object based on the time series of STI (TSSTI) and the at least one representative STI template.

2. The system of claim 1 , wherein:

each STI is related to at least one of following features of the object: position change, distance change, speed or acceleration; and

the target motion is a transient motion or a periodic motion.

3. The system of claim 2 , wherein the processor is further configured for:

computing a time series of features (TSF) based on the TSCI, wherein the TSSTI is computed based on the TSF, wherein each feature of the TSF comprises at least one of following characteristics related to a corresponding channel information (CI): magnitude, phase, autocorrelation function (ACF), local maximum, or local minimum.

4. The system of claim 3 , wherein:

the object is a human being;

the target motion is a fall-down motion of the human being;

a feature of the TSF comprises at least one of: a magnitude of a component of a CI, or an autocorrelation function of the TSCI; and

each STI comprises the speed or the acceleration.

5. The system of claim 1 , wherein the TSSTI is segmented based on at least one of: a respective first constraint on the respective initial non-target segment, a respective second constraint on the respective target segment, or a respective third constraint on the respective trailing non-target segment.

6. The system of claim 5 , wherein the processor is further configured for:

constraining the respective initial non-target segment to start from a beginning of the TSSTI and to have a first duration that is at least one of: less than a first threshold, or being a fraction of a duration of the TSSTI;

constraining the respective target segment to have a second duration not greater than a target duration associated with the target motion of the object; or

constraining the respective trailing non-target segment to end at an end of the TSSTI and to have a third duration being a fraction of the duration of the TSSTI.

7. The system of claim 1 , wherein each STI of training TSSTI is mapped to at least one STI in the mapping,

wherein each mapping score is a similarity score or a mismatch score.

8. The system of claim 1 , wherein the processor is further configured for:

determining a particular candidate mapping associated with an optimal mapping score among all the candidate mappings, wherein the optimal mapping score comprises at least one of: a maximum similarity score or a minimum mismatch score;

choosing the particular candidate mapping as the mapping between the two TSSTI; and

choosing the associated optimal mapping score as the associated mapping score between the two TSSTI.

9. The system of claim 1 , wherein the processor is further configured for:

computing iteratively the plurality of mapping scores;

determining a series of partial mappings between subsets of the training TSSTI and subsets of the additional training TSSTI; and

computing iteratively a quantity associated with each partial mapping, wherein each partial mapping is a mapping between a partial segment of the training TSSTI and a partial segment of the additional training TSSTI, wherein the series of partial mappings comprises the plurality of candidate mappings.

10. The system of claim 9 , wherein the processor is further configured for:

in an initial iteration, initializing iteration by determining an initial partial mapping between a single STI of the training TSSTI and a single STI of the additional training TSSTI and computing an initial quantity associated with the initial partial mapping;

in subsequent iterations, computing iteratively the quantity associated with each partial mapping based on a value associated with a partial mapping computed in a previous iteration,

wherein the partial mapping is between a first segment of the training TSSTI and a second segment of the additional training TSSTI, and the additional partial mapping is between a third segment of the training TSSTI and a fourth segment of the additional TSSTI,

wherein the third segment is a subset of the first segment,

wherein the fourth segment is a subset of the second segment; and

computing the plurality of mapping scores based on the quantities associated with the series of partial mappings.

11. The system of claim 9 , wherein the processor is further configured for:

in an initial iteration, initializing iteration by determining a group of initial partial mappings comprising a mapping between a single STI of the training TSSTI and a single STI of the additional training TSSTI, and computing initial quantities associated with the group of initial partial mappings;

in subsequent iterations, computing iteratively quantities associated with a group of partial mappings based on at least one value computed in at least one previous iteration; and

computing the plurality of mapping scores based on the quantities associated with the series of partial mappings.

12. The system of claim 10 , wherein:

the quantity associated with a partial mapping is a function of a measure component associated with the partial mapping and a length component associated with the partial mapping;

the measure component is a similarity component or a mismatch component; and

the processor is further configured for:

in the initial iteration, computing an initial measure component and an initial length component associated with the initial partial mapping, and

in subsequent iterations, computing iteratively a measure component and a length component associated with each partial mapping based on at least one of: a measure component computed in a past iteration or a length component computed in an additional past iteration, and computing the quantity based on the function.

13. The system of claim 7 , wherein the processor is further configured for:

segmenting the training TSSTI based on a plurality of pairwise mappings, wherein each pairwise mapping is a mapping between the training TSSTI and one of the remaining training TSSTI;

computing a plurality of candidate segmentations of the training TSSTI each based on a respective pairwise mapping, each candidate segmentation comprising a candidate initial non-target segment, a candidate target segment, and a candidate trailing non-target segment of the training TSSTI; and

segmenting the training TSSTI by combining the plurality of candidate segmentations.

14. The system of claim 13 , wherein the processor is further configured for:

determining two respective candidate boundary points of each candidate segmentation, wherein the two respective candidate boundary points includes: a respective candidate first boundary point between the respective candidate initial segment and the respective candidate target segment, and a respective candidate second boundary point between the respective candidate target segment and the respective candidate trailing segment of the candidate segmentation;

segmenting the training TSSTI by computing a first boundary point between the initial segment and the target segment of the training TSSTI as a first function of the candidate first boundary points of the plurality of candidate segmentation; and

segmenting the training TSSTI by computing a second boundary point between the target segment and the trailing segment of the training TSSTI as a second function of the candidate second boundary points of the plurality of candidate segmentation.

15. The system of claim 1 , wherein the processor is further configured for:

computing a representative STI template based on at least one of: the target segments of the cleaned training TSSTI, a time derivative of the target segments, or a signal processing of the target segments.

16. The system of claim 15 , wherein the representative STI template is computed by:

computing an optimal TSSTI based on an optimization of a combined score with respect to the set of cleaned training TSSTI,

wherein the optimization comprises at least one of: maximization, minimization, constrained maximization, or constrained minimization,

wherein the combined score comprises an aggregation of at least one of:

a plurality of first scores each being a first pairwise score between the TSSTI and a cleaned training TSSTI based on a pairwise mapping between the TSSTI and the cleaned training TSSTI,

a plurality of second scores each being a second pairwise score between the time derivative of the TSSTI and the time derivative of a cleaned training TSSTI based on the pairwise mapping, or

a plurality of third scores each being a third pairwise score between the signal processed TSSTI and a signal processed cleaned training TSSTI based on the pairwise mapping,

wherein the aggregation comprises at least one of: sum, mean, or discriminative cost,

wherein the discriminative cost comprises a weighted difference between (a) a first normalized aggregation of a first pairwise mapping score between the TSSTI and each of a first subset of the cleaned training TSSTI, and (b) a second normalized aggregation of a second pairwise mapping score between the TSSTI and each of a second subset of the cleaned training TSSTI,

wherein any normalized aggregation comprises at least one of: mean or weighted mean.

17. The system of claim 1 , wherein the processor is further configured for:

in the offline stage, computing the at least one representative STI template based on a first dynamic time warping (DTW); and

in the online stage:

computing a test mapping between the TSSTI and each representative STI template with a respective associated test mapping score based on a second DTW, and

detecting the target motion of the object when any test mapping score satisfies a detection criterion.

18. The system of claim 2 , wherein the processor is further configured for:

before detecting the target motion of the object, detecting a presence of motion of the object based on the TSCI with a first probing rate,

wherein the target motion of the object is detected based on a second probing rate higher than the first probing rate.

19. The system of claim 18 , wherein:

the first wireless signal comprises a time series of probing signals with piecewise-constant instantaneous probing rates corresponding to piecewise-constant instantaneous inter-probe periods;

each CI of the TSCI has a respective piecewise-constant instantaneous CI rate corresponding to a piecewise-constant instantaneous probing rate;

each STI of the TSSTI has a respective piecewise-constant instantaneous STI rate corresponding to a piecewise-constant instantaneous CI rate of the TSCI; and

the processor is further configured for

in a standby mode of the system, configuring the instantaneous STI rates and the instantaneous CI rates to be a default rate by configuring the instantaneous probing rate to be the default rate,

in the standby mode, detecting the presence of the motion based on at least one of: the CI at the default rate, or the STI at the default rate,

in a task mode of the system, configuring the instantaneous STI rates and the instantaneous CI rates to be an elevated rate by configuring the instantaneous probing rate to be the elevated rate, wherein the elevated rate is greater than the default rate,

in the task mode, detecting the target motion of the object based on at least one of: the CI at the elevated rate, or the STI at the elevated rate, and

switching between the task mode and the standby mode.

20. A wireless device of a wireless target motion detection system, comprising:

a processor;

a memory communicatively coupled to the processor; and

a receiver communicatively coupled to the processor, wherein:

an additional wireless device of the wireless target motion detection system is configured for transmitting a first wireless signal through a wireless multipath channel of a venue to the receiver,

the receiver is configured for receiving a second wireless signal through the wireless multipath channel,

the second wireless signal differs from the first wireless signal due to the wireless multipath channel that is impacted by a target motion of an object in the venue, and

the processor is configured for:

obtaining a time series of channel information (TSCI) of the wireless multipath channel based on the second wireless signal,

computing a time series of spatial-temporal information (TSSTI) of the object based on the TSCI,

computing, in an offline stage of the wireless target motion detection system, at least one representative STI template based on a set of training TSSTI, wherein:

each training TSSTI is computed based on a respective training TSCI obtained based on a respective training wireless signal from a respective training wireless device received by a respective training wireless device in a respective training venue when a respective training object in the respective training venue undergoes a respective training target motion,

the at least one representative STI template is computed based on cleaning each training TSSTI by segmenting the training TSSTI into: a respective initial non-target segment of spatial-temporal information (STI), a respective target segment of STI, and a respective trailing non-target segment of STI,

computing the at least one representative STI template comprises computing an optimal TSSTI based on an optimization of a combined score with respect to the set of cleaned training TSSTI,

the optimization comprises at least one of: maximization, minimization, constrained maximization, or constrained minimization,

the combined score comprises an aggregation of at least one of:

 a plurality of first scores each being a first pairwise score between the TSSTI and a cleaned training TSSTI based on a pairwise mapping between the TSSTI and the cleaned training TSSTI,

 a plurality of second scores each being a second pairwise score between a time derivative of the TSSTI and a time derivative of a cleaned training TSSTI based on the pairwise mapping, or

 a plurality of third scores each being a third pairwise score between a signal processed TSSTI and a signal processed cleaned training TSSTI based on the pairwise mapping, and

the aggregation comprises at least one of: sum, mean, or discriminative cost,

the discriminative cost comprises a weighted difference between (a) a first normalized aggregation of a first pairwise mapping score between the TSSTI and each of a first subset of the cleaned training TSSTI, and (b) a second normalized aggregation of a second pairwise mapping score between the TSSTI and each of a second subset of the cleaned training TSSTI,

any normalized aggregation comprises at least one of: mean or weighted mean, and

detecting, in an online stage of the wireless target motion detection system, the target motion of the object based on at least one of: the TSSTI and the at least one representative STI template.

21. The wireless device of claim 20 , wherein the processor is further configured for:

segmenting the training TSSTI based on a mapping between the training TSSTI and an additional training TSSTI, and based on a mapping score between the two TSSTI associated with the mapping, wherein each STI of the training TSSTI is mapped to at least one STI in the mapping, wherein the mapping score is a similarity score or a mismatch score;

computing a plurality of candidate mappings between the two TSSTI;

computing a plurality of mapping scores associated with the candidate mappings;

determining a particular candidate mapping associated with an optimal mapping score among all the candidate mappings, as the mapping between the two TSSTI, wherein the optimal mapping score comprises at least one of: a maximum similarity score or a minimum mismatch score; and

computing a representative STI template based on at least one of: the target segments of the cleaned training TSSTI, a time derivative of the target segments, or a signal processing of the target segments.

22. The wireless device of claim 20 , wherein:

the first wireless signal comprises a time series of probing signals with piecewise-constant instantaneous probing rates corresponding to piecewise-constant instantaneous inter-probe periods;

each CI of the TSCI has a respective piecewise-constant instantaneous CI rate corresponding to a piecewise-constant instantaneous probing rate;

each STI of the TSSTI has a respective piecewise-constant instantaneous STI rate corresponding to a piecewise-constant instantaneous CI rate of the TSCI; and

the processor is further configured for

in a standby mode of the online stage, configuring the instantaneous STI rates and the instantaneous CI rates to be a default rate by configuring the instantaneous probing rate to be the default rate,

in the standby mode, detecting a presence of motion of the object based on at least one of: the CI at the default rate, or the STI at the default rate,

switching from the standby mode to a task mode of the online stage,

in the task mode, configuring the instantaneous STI rates and the instantaneous CI rates to be an elevated rate by configuring the instantaneous probing rate to be the elevated rate, wherein the elevated rate is greater than the default rate, and

in the task mode, detecting the target motion of the object based on at least one of: the CI at the elevated rate, or the STI at the elevated rate.

23. A method of a target motion detection system, comprising:

transmitting, by a transmitter, a first wireless signal through a wireless multipath channel of a venue to a receiver;

receiving, by the receiver in an online stage of the target motion detection system, a second wireless signal through the wireless multipath channel, wherein the second wireless signal differs from the first wireless signal due to the wireless multipath channel that is impacted by a target motion of an object in the venue;

obtaining a time series of channel information (TSCI) of the wireless multipath channel based on the second wireless signal, using a processor, a memory communicatively coupled with the processor and a set of instructions stored in the memory;

computing a time series of spatial-temporal information (TSSTI) of the object based on the TSCI;

in an offline stage before the online stage, computing at least one representative STI template based on a set of training TSSTI, wherein:

each training TSSTI is computed based on a respective training TSCI obtained based on a respective training wireless signal from a respective training transmitter received by a respective training receiver in a respective training venue when a respective training object in the respective training venue undergoes a respective training target motion,

the at least one representative STI template is computed based on cleaning each training TSSTI by segmenting the training TSSTI into: a respective initial non-target segment of STI, a respective target segment of STI corresponding to the respective training target motion of the respective training object, and a respective trailing non-target segment of STI,

computing the at least one representative STI template comprises computing an TSSTI based on an optimization of a combined score with respect to the set of cleaned training TSSTI,

the optimization comprises at least one of: maximization, minimization, constrained maximization, or constrained minimization,

the combined score comprises an aggregation of at least one of:

a plurality of first scores each being a first pairwise score between the TSSTI and a cleaned training TSSTI based on a pairwise mapping between the TSSTI and the cleaned training TSSTI,

a plurality of second scores each being a second pairwise score between a time derivative of the TSSTI and a time derivative of a cleaned training TSSTI based on the pairwise mapping, or

a plurality of third scores each being a third pairwise score between a signal processed TSSTI and a signal processed cleaned training TSSTI based on the pairwise mapping,

the aggregation comprises at least one of: sum, mean, or discriminative cost,

the discriminative cost comprises a weighted difference between (a) a first normalized aggregation of a first pairwise mapping score between the TSSTI and each of a first subset of the cleaned training TSSTI, and (b) a second normalized aggregation of a second pairwise mapping score between the TSSTI and each of a second subset of the cleaned training TSSTI,

any normalized aggregation comprises at least one of: mean or weighted mean; and

detecting the target motion of the object based on the TSSTI and the at least one representative STI template.

24. The method of claim 23 , further comprising:

wherein the training TSSTI is segmented based on a mapping between the training TSSTI and an additional training TSSTI, and based on a mapping score between the two TSSTI associated with the mapping, wherein the mapping score comprises at least one of: a similarity score or a mismatch score,

computing a plurality of candidate mappings between the two TSSTI;

computing a plurality of mapping scores associated with the candidate mappings;

determining a particular candidate mapping associated with an optimal mapping score among all the candidate mappings, as the mapping between the two TSSTI, wherein the optimal mapping score comprises at least one of: a maximum similarity score or a minimum mismatch score; and

computing a representative STI template based on at least one of: the target segments of the cleaned training TSSTI, a time derivative of the target segments, or a signal processing of the target segments.

25. The method of claim 24 , wherein:

the first wireless signal comprises a time series of probing signals with piecewise-constant instantaneous probing rates corresponding to piecewise-constant instantaneous inter-probe periods;

each CI of the TSCI has a respective piecewise-constant instantaneous CI rate corresponding to a piecewise-constant instantaneous probing rate;

each STI of the TSSTI has a respective piecewise-constant instantaneous STI rate corresponding to a piecewise-constant instantaneous CI rate of the TSCI; and

the method further comprises:

in a standby mode of the online stage, configuring the instantaneous STI rates and the instantaneous CI rates to be a default rate by configuring the instantaneous probing rate to be the default rate,

in the standby mode, detecting a presence of motion of the object based on at least one of: the CI at the default rate, or the STI at the default rate,

switching from the standby mode to a task mode of the online stage,

in the task mode, configuring the instantaneous STI rates and the instantaneous CI rates to be an elevated rate by configuring the instantaneous probing rate to be the elevated rate, wherein the elevated rate is greater than the default rate, and

in the task mode, detecting the target motion of the object based on at least one of: the CI at the elevated rate, or the STI at the elevated rate.

Assignments (2)

Continuity (39)

Cited By (2)

US 12,426,074 US 12,681,161