Method, apparatus, and system for wireless monitoring with motion localization
Methods, apparatus and systems for wireless monitoring for motion detection and localization are described. In one example, a described system comprises: Type 1 devices; Type 2 devices; and a processor. Each Type 2 device is configured for: asynchronously receiving, from each of a respective subset of Type 1 devices associated with the Type 2 device, a respective wireless signal through a respective wireless multipath channel impacted by a motion of an object in a venue; asynchronously receiving, from each of a subset of a respective set of at least one other Type 2 device interconnected wirelessly with the Type 2 device, a respective wireless signal through a respective wireless multipath channel impacted by the motion of the object; and obtaining a set of asynchronous time series of channel information (ATSCI), wherein each ATSCI is associated with a respective wireless multipath channel and is obtained based on the asynchronously received respective wireless signal. The processor is configured for monitoring the motion of the object based on the set of ATSCI associated with each Type 2 device.
1 . A system for collaborative wireless motion monitoring, comprising:
a processor; and
a set of access point (AP) devices that are wirelessly interconnected in at least one network in a venue,
for each respective AP device of the set of AP devices, a respective set of client devices each being wirelessly associated with the respective AP device in a respective wireless network in the venue, the respective AP device and the respective set of associated client devices forming a respective AP-group, wherein:
each AP device is associated with a respective region of the venue,
each client device associated with any AP device is associated with a respective region of the venue,
each AP device is wirelessly connected with at least one other AP device of the set of AP devices and serves as a transmitter, a receiver, or both, of wireless probing signals to or from the at least one other AP device,
each client device serves as a transmitter, a receiver, or both, of wireless probing signals to or from the respective associated AP device,
at least one client device serves as the transmitter of the wireless probing signals to the respective associated AP device,
at least one client device serves as the receiver of the wireless probing signals from the respective associated AP device,
each client device is configured for:
asynchronously transmitting to or receiving from the associated AP device a respective wireless probing signal through a respective wireless multipath channel impacted by a motion of an object in the venue, and
after receiving the respective wireless probing signal,
obtaining an asynchronous time series of channel information (ATSCI) of the respective wireless multipath channel based on the respective received wireless probing signal, wherein each channel information (CI) of the ATSCI is one of: channel impulse response (CIR), channel frequency response (CFR) or channel state information (CSI), of the respective wireless multipath channel extracted from the respective received wireless probing signal,
computing an asynchronous time series of motion statistics (ATSMS) based on the ATSCI, and
monitoring the motion of the object based on the ATSMS and ATSCI to obtain a first monitoring result,
each AP device is configured for:
asynchronously receiving from or transmitting to each of the respective set of associated client devices, a respective wireless probing signal through a respective wireless multipath channel impacted by the motion of the object in the venue,
asynchronously receiving, from each of a respective first subset of other AP device of the set of AP devices, a respective wireless probing signal through a respective wireless multipath channel impacted by the motion of the object in the venue,
asynchronously transmitting, to each of a respective second subset of other AP device of the set of AP devices, a respective wireless probing signal through a respective wireless multipath channel impacted by the motion of the object in the venue, wherein at least one of the respective first subset and the respective second subset is non-empty,
obtaining a set of ATSCI, each of which is obtained based on a respective received wireless probing signal,
computing a set of ATSMS, each of which is computed based on a respective ATSCI,
monitoring the motion of the object based on the set of ATSMS,
computing an asynchronous time series of intermediate statistics (ATSIS) for the AP-group associated with the AP device based on the set of ATSMS and any ATSMS obtained from a respective set of associated client devices, wherein each intermediate statistics (IS) of the ATSIS is an aggregate of respective motion statistics (MS) of all the ATSMS associated with the AP-group, and
monitoring the motion of the object further to get a second monitoring result, based on: the ATSIS, the set of ATSMS, any ATSMS and first monitoring results obtained from the respective set of associated client devices, and any ATSIS, ATSMS and second monitoring results obtained from other AP devices; and
a processor configured for:
monitoring the motion of the object based on any ATSIS, any ATSMS and any first and second monitoring results obtained from the set of AP devices and the sets of associated client devices,
computing a likelihood for each client device to detect a motion of the object, wherein the likelihood is smoothed over a time window,
when the motion of the object is detected:
choosing a particular AP-group with largest IS among all the AP-groups to be associated with the detected motion, wherein the particular AP-group includes a particular AP device and a particular set of client devices,
comparing the IS of the particular AP-group with a threshold after the particular AP-group is chosen based on the largest IS,
when the IS of the particular AP-group is larger than the threshold, determining a location of the detected motion to be associated with a first region associated with the particular AP device in the particular AP-group, and
when the IS of the particular AP-group is less than the threshold, determining the location of the detected motion to be associated with a second region associated with a particular client device having a highest likelihood to detect the motion of the object among the particular set of client devices in the particular AP-group, wherein the first region and the second region are non-overlapping.
2 . The system of claim 1 , wherein the processor is physically coupled to at least one of the set of AP devices, wherein a configuration of the client devices and AP devices is maintained in a configuration database, wherein the configuration comprises any addition of new interconnected devices and any removal of existing interconnected devices.
3 . The system of claim 1 , wherein monitoring the motion of the object comprises at least one of:
monitoring the motion of the object individually based on an ATSCI associated with a particular AP device and at least one of: a particular client device associated with the particular AP device, or a particular other AP device interconnected wirelessly with the particular AP device;
monitoring the motion of the object jointly based on at least two ATSCI associated with one particular AP device;
monitoring the motion of the object jointly based on at least two ATSCI each associated with a respective AP device;
monitoring the motion of the object group-wise based on all ATSCI associated with a AP device;
monitoring the motion of the object group-wise based on all ATSCI associated with a group of at least two AP devices; or
monitoring the motion of the object globally based on all ATSCI.
4 . The system of claim 1 , wherein the processor is further configured for:
computing a set of asynchronous motion statistics (MS) based on the set of ATSCI, wherein
each MS is computed based on a respective ATSCI associated with a device-pair comprising a AP device and another device, and
the motion of the object is monitored based on the set of asynchronous MS.
5 . The system of claim 4 , wherein monitoring the motion of the object comprises monitoring the motion of the object individually based on a MS associated with a particular device-pair comprising a particular AP device.
6 . The system of claim 4 , wherein monitoring the motion of the object comprises:
computing at least one joint statistics based on at least two MS associated with one particular AP device; and
monitoring the motion of the object jointly based on the at least one joint statistics.
7 . The system of claim 4 , wherein monitoring the motion of the object comprises:
computing at least one joint statistics based on at least two MS each associated with a respective AP device; and
monitoring the motion of the object jointly based on the at least one joint statistics.
8 . The system of claim 4 , wherein monitoring the motion of the object comprises:
computing at least one group statistics based on all MS associated with a particular AP device; and
monitoring the motion of the object jointly based on the at least one group statistics.
9 . The system of claim 4 , wherein monitoring the motion of the object comprises:
computing at least one group statistics based on all MS associated with a group of at least two AP devices; and
monitoring the motion of the object jointly based on the at least one group statistics.
10 . The system of claim 4 , wherein monitoring the motion of the object comprises:
computing at least one global statistics based on all MS; and
monitoring the motion of the object globally based on the at least one global statistics.
11 . The system of claim 1 , wherein:
each ATSMS is computed based on a respective ATSCI associated with a device-pair comprising a AP device and another device associated with the respective ATSCI; and
each ATSIS is computed based on all ATSMS associated with device-pairs comprising the respective AP device.
12 . The system of claim 11 , wherein monitoring the motion of the object comprises monitoring the motion of the object individually based on an ATSMS associated with a particular device-pair comprising a particular AP device.
13 . The system of claim 11 , wherein monitoring the motion of the object comprises:
computing at least one joint statistics based on at least two ATSMS associated with one particular AP device; and
monitoring the motion of the object jointly based on the at least one joint statistics.
14 . The system of claim 11 , wherein monitoring the motion of the object comprises:
computing at least one joint statistics based on at least two ATSMS each associated with a respective AP device; and
monitoring the motion of the object jointly based on the at least one joint statistics.
15 . The system of claim 11 , wherein monitoring the motion of the object comprises:
computing at least one group statistics based on all ATSMS associated with a particular AP device; and
monitoring the motion of the object jointly based on the at least one group statistics.
16 . The system of claim 11 , wherein monitoring the motion of the object comprises:
computing at least one group statistics based on all ATSMS associated with a group of at least two AP devices; and
monitoring the motion of the object jointly based on the at least one group statistics.
17 . The system of claim 11 , wherein monitoring the motion of the object comprises:
computing at least one global statistics based on all ATSMS; and
monitoring the motion of the object globally based on the at least one global statistics.
18 . The system of claim 11 , wherein each motion statistics (MS) of an ATSMS further comprises at least one of:
motion detection statistics, similarity score, component similarity score, time reversal resonating strength, correlation, inner product, motion identification statistics, event occurrence, or event statistics.
19 . The system of claim 11 , wherein the processor is further configured for:
for each device-pair comprising an AP-client device-pair of associated client device and AP device or an AP-AP device-pair of two interconnected AP devices:
computing an ATSMS based on a respective ATSCI associated with the device-pair, and
applying a respective first processing to the ATSMS with respective parameters; for each AP device;
computing a respective ATSIS for the AP device based on all ATSMS associated with all device-pairs associated with the AP device, and
applying a respective second processing to the respective ATSIS with respective parameters;
associating, based on all ATSIS computed for all the AP devices, the object and its motion to at least one of: a AP device or a client device; and
monitoring the motion of the object based on at least one of: the associated AP device, the associated client device, a device-pair comprising the associated AP device, or a device-pair comprising the associated client device.
20 . The system of claim 19 , wherein each intermediate statistics (IS) of an ATSIS is different from a motion statistics (MS) of an ATSMS and comprises at least one of:
a function of a MS of an ATSMS, a function of more than one MS of an ATSMS, a function of more than one MS of more than one ATSMS, a moving average, a weighted average, an arithmetic mean, a geometric mean, a harmonic mean, a trimmed mean, a center, a centroid, a mean, a mode, a median, a percentile, a filtering, a derivative, a transform, a thresholding, a projection, a decomposition, a component, a feature extraction, a maximization, a minimization, a likelihood, a maximum likelihood, a quantization, a vector quantization, a clustering, a merging, a splitting, a similarity measure between two CI, a distance measure between two CI, a correlation of two CI, motion detection statistics, motion identification statistics, event occurrence, or event statistics.
21 . The system of claim 19 , wherein computing the ATSIS for the AP device comprises computing each intermediate statistics (IS) of the ATSIS based on at least one of:
a sum of motion statistics (MS) associated with all device-pairs associated with the AP device;
a weighted sum of MS associated with all device-pairs associated with the AP device;
a feature of each MS associated with all device-pairs associated with the AP device;
a maximum of all MS associated with all device-pairs associated with the AP device;
a minimum of all MS associated with all device-pairs associated with the AP device;
a derivative of ATSMS associated with all device-pairs associated with the AP device;
a filtering of ATSMS associated with all device-pairs associated with the AP device;
a sliding window of ATSMS associated with all device-pairs associated with the AP device;
an autocorrelation function (ACF) of ATSMS associated with all device-pairs associated with the AP device; or
a weighted sum of a nonlinear transformation of each MS associated with all device-pairs associated with the AP device.
22 . The system of claim 19 , wherein the object and its motion are associated to the particular AP device based on:
comparing intermediate statistics (IS) associated with AP-AP device-pairs with a first threshold;
comparing IS associated with AP-client device-pairs with a second threshold; and
identifying the particular AP device associated to the object and its motion as a candidate AP device associated with at least one of:
an AP-AP device-pair that comprises the candidate AP device and is associated with an IS exceeding the first threshold, or
an AP-client device-pair that comprises the candidate AP device and is associated with an IS exceeding the second threshold.
23 . The system of claim 19 , wherein the object and its motion are associated to the particular client device associated with the particular AP device based on:
comparing intermediate statistics (IS) associated with AP-AP device-pairs with a first threshold;
identifying the particular AP device as a candidate AP device associated with an AP-AP device-pair that comprises the candidate AP device and has an associated IS that is less than the first threshold and is maximum among all IS;
comparing all IS associated with AP-client device-pairs comprising the particular AP device with a second threshold; and
identifying the particular client device as a candidate client device associated with the particular AP device whose AP-client device-pair has an associated IS that is greater than the second threshold and is maximum among all AP-client device-pairs comprising the particular AP device.
24 . The system of claim 1 , wherein the processor is further configured for:
associating each AP device with a region of the venue;
associating each client device with a region of the venue, wherein a region associated with a AP device comprises a union of regions associated with client devices associated with the AP device;
associating a 1-2 device-pair with a region of the venue, wherein the 1-2 device-pair comprises a AP device and a client device associated with the AP device; and
associating a 2-2 device-pair with a region of the venue, wherein the 2-2 device-pair comprises two wirelessly interconnected AP devices.
25 . The system of claim 1 , wherein the processor is further configured for:
associating a AP device with a first region of the venue;
associating a client device with a second region of the venue, wherein the client device is associated with the AP device;
associating an additional AP device with a third region of the venue, wherein the additional AP device is interconnected with the AP device, wherein
the first region and the second region are identical, overlapping, or non-overlapping,
the first region and the third region are identical, overlapping, or non-overlapping, and
the second region and the third region are identical, overlapping, or non-overlapping.
26 . The system of claim 1 , wherein:
the wireless probing signal comprises at least one of:
a pinging signal, sounding signal, beacon signal, pilot signal, probe signal, handshake signal, acknowledgement signal, synchronization signal, enquiry signal, standard compliant signal, standard frame, line-of-sight (LOS) component signal, non-LOS component signal, a plurality of signals, or a series of signals; and
an ATSCI, associated with the particular AP device and another device, is obtained at a layer of the particular AP device and by an application comprising at least one of: a firmware, driver, software, or downloadable app.
27 . A method of a collaborative wireless motion monitoring system, comprising:
interconnecting wirelessly a set of AP devices in a venue, wherein:
each AP device is wirelessly connected with at least one other AP device of the set of AP devices and serves as a transmitter, a receiver, or both, of wireless probing signals to or from the at least one other AP device,
for each respective AP device of the set of AP devices, a respective set of client devices is wirelessly associated with the respective AP device in a respective wireless network in the venue, the respective AP device and the respective set of associated client devices forming a respective AP-group,
each AP device is associated with a respective region of the venue,
each client device associated with any AP device is associated with a respective region of the venue,
each client device serves as a transmitter, a receiver, or both, of wireless probing signals to or from the respective associated AP device,
at least one client device serves as the transmitter of the wireless probing signals to the respective associated AP device, and
at least one client device serves as the receiver of the wireless probing signals from the respective associated AP device;
for each AP device:
asynchronously receiving from or transmitting to each of the respective set of associated client devices, a respective wireless probing signal through a respective wireless multipath channel impacted by a motion of an object in the venue,
for each of a respective subset of other AP device interconnected wirelessly with the AP device, at least one of: asynchronously receiving from, or asynchronously transmitting to, the other AP device, a respective wireless probing signal through a respective wireless multipath channel impacted by the motion of the object in the venue,
obtaining a set of asynchronous time series of channel information (CI) associated with the AP device, wherein each asynchronous time series of CI (ATSCI) is associated with a respective wireless multipath channel and is obtained based on the asynchronously received respective wireless probing signal, wherein each CI of the ATSCI is one of: channel impulse response (CIR), channel frequency response (CFR) or channel state information (CSI), of the respective wireless multipath channel extracted from the received respective wireless probing signal,
computing a set of asynchronous time series of motion statistics (ATSMS), each of which is computed based on a respective ATSCI,
monitoring the motion of the object based on the set of ATSMS,
computing an asynchronous time series of intermediate statistics (ATSIS) for the AP-group associated with the AP device based on the set of ATSMS and any ATSMS obtained from a respective set of associated client devices, wherein each intermediate statistics (IS) of the ATSIS is an aggregate of respective motion statistics (MS) of all the ATSMS associated with the AP-group, and
monitoring the motion of the object further to get a monitoring result, based on: the ATSIS, the set of ATSMS, any ATSMS and monitoring results obtained from the respective set of associated client devices, and any ATSIS, ATSMS and monitoring results obtained from other AP devices;
monitoring the motion of the object in the venue based on any ATSIS, any ATSMS and any monitoring results obtained from the set of AP devices and the sets of associated client devices;
computing a likelihood for each client device to detect a motion of the object, wherein the likelihood is smoothed over a time window;
when the motion of the object is detected:
choosing a particular AP-group with largest IS among all the AP-groups to be associated with the detected motion, wherein the particular AP-group includes a particular AP device and a particular set of client devices,
comparing the IS of the particular AP-group with a threshold after the particular AP-group is chosen based on the largest IS,
when the IS of the particular AP-group is larger than the threshold, determining a location of the detected motion to be associated with a first region associated with the particular AP device in the particular AP-group, and
when the IS of the particular AP-group is less than the threshold, determining the location of the detected motion to be associated with a second region associated with a particular client device having a highest likelihood to detect the motion of the object among the particular set of client devices in the particular AP-group, wherein the first region and the second region are non-overlapping.
28 . The method of claim 27 , further comprising:
adding, to the collaborative wireless motion monitoring system, at least one of: a new client device or a new AP device;
configuring at least one of the new client device or the new AP device, wherein:
the new client device is configured to associate with an existing AP device, and
the new AP device is configured to interconnect with at least one existing AP device;
updating a configuration database of the collaborative wireless motion monitoring system with at least one of the new client device or the new AP device; and
negotiating and initiating wireless signal communication associated with at least one of the new client device or the new AP device.
29 . The method of claim 27 , further comprising:
configuring an existing client device to terminate an existing association with an existing AP device;
configuring the existing client device to associate with a new AP device; and
removing, from the collaborative wireless motion monitoring system, at least one of: a client device or a AP device.
30 . A method of a collaborative wireless motion monitoring system, comprising:
interconnecting wirelessly a set of AP devices in a venue, wherein:
each AP device is wirelessly connected with at least one other AP device of the set of AP devices and serves as a transmitter, a receiver, or both, of wireless probing signals to or from the at least one other AP device,
for each respective AP device of the set of AP devices, a respective set of client devices is wirelessly associated with the respective AP device in a respective wireless network in the venue, the respective AP device and the respective set of associated client devices forming a respective AP-group,
each AP device is associated with a respective region of the venue,
each client device associated with any AP device is associated with a respective region of the venue,
each client device serves as a transmitter, a receiver, or both, of wireless probing signals to or from the respective associated AP device,
at least one client device serves as the transmitter of the wireless probing signals to the respective associated AP device, and
at least one client device serves as the receiver of the wireless probing signals from the respective associated AP device;
for each AP device:
asynchronously receiving from or transmitting to each of the respective set of associated client devices, a respective wireless probing signal through a respective wireless multipath channel impacted by a motion of an object in the venue,
communicating asynchronously a respective wireless probing signal between the AP device and each of a respective subset of other AP device interconnected wirelessly with the AP device, through a respective wireless multipath channel impacted by the motion of the object in the venue,
obtaining a set of asynchronous time series of channel information (CI) associated with the AP device, wherein each asynchronous time series of CI (ATSCI) is of a respective wireless multipath channel and is obtained based on the asynchronously received respective wireless probing signal, wherein each CI of the ATSCI is one of: channel impulse response (CIR), channel frequency response (CFR) or channel state information (CSI), of the respective wireless multipath channel extracted from the received respective wireless probing signal,
computing a set of asynchronous time series of motion statistics (ATSMS), each of which is computed based on a respective ATSCI,
monitoring the motion of the object based on the set of ATSMS,
computing an asynchronous time series of intermediate statistics (ATSIS) for the AP-group associated with the AP device based on the set of ATSMS and any ATSMS obtained from a respective set of associated client devices, wherein each intermediate statistics (IS) of the ATSIS is an aggregate of respective motion statistics (MS) of all the ATSMS associated with the AP-group, and
monitoring the motion of the object further to get a monitoring result, based on: the ATSIS, the set of ATSMS, any ATSMS and monitoring results obtained from the respective set of associated client devices, and any ATSIS, ATSMS and monitoring results obtained from other AP devices;
monitoring the motion of the object in the venue based on any ATSIS, any ATSMS and any monitoring results obtained from the set of AP devices and the sets of associated client devices;
computing a likelihood for each client device to detect a motion of the object, wherein the likelihood is smoothed over a time window;
when the motion of the object is detected:
choosing a particular AP-group with largest IS among all the AP-groups to be associated with the detected motion, wherein the particular AP-group includes a particular AP device and a particular set of client devices,
comparing the IS of the particular AP-group with a threshold after the particular AP-group is chosen based on the largest IS,
when the IS of the particular AP-group is larger than the threshold, determining a location of the detected motion to be associated with a first region associated with the particular AP device in the particular AP-group, and
when the IS of the particular AP-group is less than the threshold, determining the location of the detected motion to be associated with a second region associated with a particular client device having a highest likelihood to detect the motion of the object among the particular set of client devices in the particular AP-group, wherein the first region and the second region are non-overlapping.
31 . The system of claim 19 , wherein the processor is further configured for:
for each AP device:
computing a respective first ATSIS for the AP device based on all respective ATSMS associated with device-pairs comprising the AP device,
determining a respective set of MS comprising MS from the ATSMS associated with the device-pairs comprising the AP device, wherein all MS of the respective set of MS are associated with a time stamp, and
computing a first IS of the respective first ATSIS for the AP device at the time stamp as a first representative value of the respective set of MS, wherein the first representative value comprises one of: a weighted sum, a weighted average, a weighted median, a maximum, or a minimum; and
associating the object and its motion to the particular AP device when the first ATSIS of the set of AP devices satisfy a first criterion.
32 . The system of claim 31 , wherein the processor is further configured for:
associating the object and its motion to the particular AP device at the time stamp when the first IS at the time stamp associated with the set of AP devices satisfy the first criterion;
determining the particular AP device as the AP device whose associated first IS at the time stamp has a magnitude that is maximum among the first IS associated with the set of AP devices at the time stamp, wherein the first criterion is satisfied when the first IS associated with the particular AP device has a magnitude greater than a first threshold;
for each client device associated with the particular AP device: computing a respective second IS for the client device at the time stamp based on all the respective ATSMS associated with device-pairs comprising the particular client device; and
associating, at the time stamp, the object and its motion to a particular client device associated with the particular AP device when a second criterion is satisfied by all the second IS associated with the particular AP device,
wherein the second criterion is satisfied when all of the following are true:
the first criterion is not satisfied by the first IS at the time stamp associated with the set of AP devices,
the second IS for the particular client device has a magnitude that is maximum among all second IS of all client devices associated with the particular AP, and
the second IS of the particular client device has a magnitude that is greater than a second threshold.
33 . The system of claim 32 , wherein the processor is further configured for:
associating the object and its motion to none of the client devices and the AP devices when both the first criterion and the second criterion are not satisfied.
34 . The system of claim 1 , wherein:
each ATSMS is computed based on a respective ATSCI associated with a device-pair comprising a AP device and another device associated with the respective ATSCI; and
each ATSIS is associated with a respective device-pair and is computed based on all ATSMS.
35 . The system of claim 34 , wherein the processor is further configured for:
computing, for each AP device, a respective first ATSIS for the AP device based on all respective ATSMS associated with device-pairs comprising the AP device;
computing, for each client device associated with any AP device, a respective second ATSIS for the client device based on all the respective ATSMS associated with device-pairs comprising the associated AP device;
associating the object and its motion to at least one of: a AP device or a client device based on all first ATSIS and all second ATSIS, wherein there is a second motion of a second object in the venue impacting the wireless multipath channel; and
associating the second object and its motion to at least one of: another AP device or another client device based on all first ATSIS and all second ATSIS.