Cross-correlation system and method for spatial detection using a network of RF repeaters
A cross-correlation system includes control circuitry that trains a deep neural network to cross-correlate sensor data from a plurality of sensors at an input sampling stage, determine a relationship between the cross-correlated sensor data and identities of one or more users represented in the cross-correlated sensor data, and track subsequent movements of the one or more users based on the cross-correlated sensor data. The control circuitry further obtains first sensor data of a first user from a communication system and second sensor data from a first portable device carried by the first user. Cross-correlated information of the first user is obtained by utilizing the trained deep neural network. A first gesture specific to the first user is recognized based on the cross-correlated information of the first user. A first controllable device identified from a plurality of controllable devices is controlled to execute a first action based on the first gesture.
1 . A cross-correlation system, comprising:
control circuitry configured to:
input training sensor data from a plurality of sensors into an input layer of a deep neural network;
train the deep neural network to:
cross-correlate the training sensor data from the plurality of sensors at an input sampling stage,
determine a relationship between the cross-correlated sensor data and an identity of each of one or more users represented in the cross-correlated sensor data, and
track subsequent movements of the one or more users based on the cross-correlated sensor data;
obtain first sensor data of a first user from a communication system and second sensor data from a first portable device carried by the first user;
obtain third sensor data from a plurality of user-devices associated with the first user;
obtain cross-correlated information of the first user based on the trained deep neural network;
recognize a first gesture specific to the first user based on the cross-correlated information of the first user, wherein the first gesture corresponds to a single hand gesture;
detect a first configuration of a first hand of the first user;
split the first gesture into a first component and a second component, wherein a configuration of a defined number of fingers of the first hand form the first component of the first gesture, and a configuration of remaining fingers form the second component of the first gesture;
identify a first controllable device from a plurality of controllable devices based on the first component, and identify a first action based on the second component; and
control the first controllable device identified from the plurality of controllable devices to execute the first action based on the first gesture.
2 . The cross-correlation system according to claim 1 , wherein
the control circuitry is further configured to:
detect a second configuration of a second hand of the first user; and
determine a joint meaning of the first configuration of the first hand and the second configuration of the second hand specific to the first user for the recognition of the first gesture.
3 . The cross-correlation system according to claim 2 , wherein the control circuitry is further configured to:
track at least one of: a change in the second configuration of the second hand while the first configuration is maintained, or an independent movement of the second hand in the second configuration while the first configuration of the first hand is maintained; and
control the identified first controllable device to execute a sequence of second actions for a specified time period based on a continuous tracking of the change in the second configuration, the independent movement of the second hand, or a combination thereof.
4 . The cross-correlation system according to claim 1 , wherein the control circuitry is further configured to:
detect a second configuration of a second hand of the first user;
set the first configuration of the first hand as the first component of the first gesture and the second configuration of the second hand as the second component of the first gesture; and
utilize the first component to control the first controllable device from the plurality of controllable devices and the second component to execute the first action associated with the identified first controllable device.
5 . The cross-correlation system according to claim 1 , wherein the first gesture corresponds to a combination of a voice command and a movement of a set of points in the first sensor data of the first user.
6 . The cross-correlation system according to claim 1 , wherein the control circuitry is further configured to modulate control instructions associated with the first controllable device in a mmWave radio frequency signal of a specified frequency.
7 . The cross-correlation system according to claim 6 , wherein the control circuitry is further configured to distribute the mmWave radio frequency signal that carries the control instructions to a plurality of cascaded repeater devices, wherein at least one repeater device of the plurality of cascaded repeater devices is configured to extract the control instructions from the mmWave radio frequency signal, and provide the control instructions to the first controllable device that is communicatively coupled to the at least one repeater device.
8 . The cross-correlation system according to claim 6 , wherein the specified frequency of the mmWave radio frequency signal is in a range of 55 gigahertz (GHz) to 65 GHz.
9 . The cross-correlation system according to claim 6 , wherein the specified frequency of the mmWave radio frequency signal is 60 gigahertz (GHz).
10 . The cross-correlation system according to claim 1 , wherein the second sensor data is received via at least one of a wireless wide area network signal, a wireless local area network signal, a wireless personal area network signal, or a combination thereof.
11 . The cross-correlation system according to claim 1 , wherein the control circuitry is further configured to:
obtain an image of the first user from an image-capture device; and
cross-correlate the image and the third sensor data with the first sensor data and the second sensor data to obtain additional cross-correlated information of the first user.
12 . The cross-correlation system according to claim 11 , wherein the control circuitry is further configured to obtain a plurality of radio frequency signals corresponding to different communication protocols from a plurality of communication systems, wherein the plurality of communication systems includes at least a radio detection and ranging system, the first portable device, the image-capture device, and the plurality of user-devices.
13 . The cross-correlation system according to claim 12 , wherein the control circuitry is further configured to provide access to a first type of communication network to the plurality of communication systems that are communicatively coupled to the cross-correlation system via a plurality of different types of communication networks.
14 . The cross-correlation system according to claim 13 , wherein each of the plurality of radio frequency signals communicated over a corresponding type of network of the plurality of different types of communication networks has a defined communication range.
15 . The cross-correlation system according to claim 14 , wherein a coverage of the plurality of radio frequency signals corresponding to the different communication protocols is extended beyond the defined communication range based on a distribution of a mmWave radio frequency signal of a specified frequency that includes the plurality of radio frequency signals.
16 . The cross-correlation system according to claim 1 , wherein the trained deep neural network is utilized to identify the first user based on the cross-correlated information of the first user during a movement of the first user.
17 . The cross-correlation system according to claim 1 , wherein the trained deep neural network is utilized to identify the first user based on the cross-correlated information of the first user even when a body shape of the first user is changed.
18 . The cross-correlation system according to claim 1 , wherein a first time slot is used to recognize the first component and a second time slot following the first time slot is used to recognize the second component.
19 . A cross-correlation method, comprising:
inputting, by control circuitry, training sensor data from a plurality of sensors into an input layer of a deep neural network;
training, by the control circuitry, the deep neural network for:
cross-correlating the training sensor data from the plurality of sensors at an input sampling stage,
determining a relationship between the cross-correlated sensor data and an identity of each of one or more users represented in the cross-correlated sensor data, and
tracking subsequent movements of the one or more users based on the cross-correlated sensor data;
obtaining, by the control circuitry, first sensor data of a first user from a communication system and second sensor data from a first portable device carried by the first user;
obtaining, by the control circuitry, third sensor data from a plurality of user-devices associated with the first user;
obtaining, by the control circuitry, cross-correlated information of the first user by utilizing the trained deep neural network;
recognizing, by the control circuitry, a first gesture specific to the first user based on the cross-correlated information of the first user, wherein the first gesture corresponds to a single hand gesture;
detecting, by the control circuitry, a first configuration of a first hand of the first user;
splitting, by the control circuitry, the first gesture into a first component and a second component, wherein a configuration of a defined number of fingers of the first hand form the first component of the first gesture, and a configuration of remaining fingers form the second component of the first gesture;
identifying, by the control circuitry, a first controllable device from a plurality of controllable devices based on the first component, and identify a first action based on the second component; and
controlling, by the control circuitry, the first controllable device identified from the plurality of controllable devices to execute the first action based on the first gesture.
20 . A non-transitory computer-readable medium having stored thereon, computer implemented instructions, which when executed by a computer in a communication apparatus, causes the communication apparatus to execute operations, the operations comprising:
inputting training sensor data from a plurality of sensors into an input layer of a deep neural network;
training the deep neural network for:
cross-correlating the training sensor data from the plurality of sensors at an input sampling stage,
determining a relationship between the cross-correlated sensor data and an identity of each of one or more users represented in the cross-correlated sensor data, and
tracking subsequent movements of the one or more users based on the cross-correlated sensor data;
obtaining first sensor data of a first user from a communication system and second sensor data from a first portable device carried by the first user;
obtaining third sensor data from a plurality of user-devices associated with the first user;
obtaining cross-correlated information of the first user by utilizing the trained deep neural network;
recognizing a first gesture specific to the first user based on the cross-correlated information of the first user, wherein the first gesture corresponds to a single hand gesture;
detecting a first configuration of a first hand of the first user;
splitting the first gesture into a first component and a second component, wherein a configuration of a defined number of fingers of the first hand form the first component of the first gesture, and a configuration of remaining fingers form the second component of the first gesture;
identifying a first controllable device from a plurality of controllable devices based on the first component, and identify a first action based on the second component; and
controlling the first controllable device identified from the plurality of controllable devices to execute the first action based on the first gesture.