Transferring data between different frequency bands

1. A method for transferring data between different frequency bands by generating an image from channel state information (CSI) samples in a communication system, the method comprising:

capturing, utilizing one or more processors, video data associated with a plurality of video frames by capturing video data associated with an i th video frame of the plurality of video frames where i≥1;

obtaining, utilizing the communication system, a first plurality of CSI samples by obtaining an i th subset of the first plurality of CSI samples simultaneously with capturing video data associated with the i th video frame;

obtaining, utilizing the one or more processors, a mapping model by training a neural network, comprising:

initializing the neural network with a plurality of initial weights; and

repeating an iterative process until a termination condition is satisfied, the iterative process comprising:

extracting a training image from an output of the neural network by applying the neural network on the i th subset of the first plurality of CSI samples, comprising:

obtaining a first plurality of feature maps by applying a first plurality of convolutional layers of the neural network on the i th subset of the first plurality of CSI samples;

obtaining a second plurality of feature maps by feeding the first plurality of feature maps to an input of a fully connected layer of the neural network;

obtaining a third plurality of feature maps by applying a residual neural network (ResNet) of the neural network on the second plurality of feature maps; and

upsampling the third plurality of feature maps;

generating a plurality of updated weights by minimizing a loss function of the training image and the i th video frame; and

replacing the plurality of updated weights with the plurality of initial weights;

obtaining, utilizing the communication system, a second plurality of CSI samples; and

obtaining, utilizing the one or more processors, the image by applying the mapping model on the second plurality of CSI samples.

2. A method for transferring data between different frequency bands by generating a data sample in a first frequency band from measurements in a second frequency band, the method comprising:

obtaining, utilizing a first measuring system, a first plurality of samples by measuring a first frequency response of an environment in the first frequency band;

obtaining, utilizing a second measuring system, a second plurality of samples by measuring a second frequency response of the environment in the second frequency band;

obtaining, utilizing one or more processors, a mapping model based on the first plurality of samples and the second plurality of samples;

obtaining, utilizing the second measuring system, a third plurality of samples by measuring a third frequency response of the environment in the second frequency band; and

obtaining, utilizing the one or more processors, the data sample by applying the mapping model on the third plurality of samples.

3. The method of claim 2 , wherein:

measuring the first frequency response comprises capturing, utilizing a video device, a plurality of video frames by capturing video data associated with an i th video frame of the plurality of video frames in an i th time interval where i≥1;

measuring the second frequency response comprises obtaining, utilizing a communication system, a first plurality of channel state information (CSI) samples by obtaining an i th subset of the first plurality of CSI samples in the i th time interval;

measuring the third frequency response comprises obtaining a second plurality of CSI samples; and

obtaining the data sample comprises obtaining an image.

4. The method of claim 3 , wherein obtaining the i th subset of the first plurality of CSI samples comprises:

transmitting a plurality of transmit signals by a transmitter of the communication system;

receiving a plurality of receive signals associated with the plurality of transmit signals by a receiver of the communication system; and

generating the i th subset of the first plurality of CSI samples by:

obtaining a plurality of raw CSI samples based on the plurality of receive signals; and

extracting the i th subset of the first plurality of CSI samples from the plurality of raw CSI samples by compensating a phase offset of each of the plurality of raw CSI samples.

5. The method of claim 4 , wherein:

transmitting the plurality of transmit signals comprises transmitting each of the plurality of transmit signals in a respective sub-carrier of a plurality of sub-carriers and at a respective transmit moment of a plurality of transmit moments in the i th time interval by a respective transmit antenna of a plurality of transmit antennas associated with the transmitter;

receiving the plurality of receive signals comprises receiving each of the plurality of receive signals in a respective sub-carrier of the plurality of sub-carriers and at a respective receive moment of a plurality of receive moments in the i th time interval by a respective receive antenna of a plurality of receive antennas associated with the receiver; and

obtaining the plurality of raw CSI samples comprises obtaining a CSI array of size N 1 ×N 2 ×N 3 by:

generating each of a plurality of CSI vectors of size

N

1

2

×

1

by estimating a respective multiple-input multiple output (MIMO) channel of a plurality of MIMO channels by processing a respective subset of the plurality of receive signals, each of the plurality of MIMO channels comprising a wireless channel between the transmitter and the receiver in a respective sub-carrier of the plurality of sub-carriers and at a respective receive moment of the plurality of receive moments; and

generating the CSI array by setting each element in the CSI array to one of a real part or an imaginary part of a respective element in a respective CSI vector of the plurality of CSI vectors,

wherein:

N 1 =2M t M r ,

M t is a number of the plurality of transmit antennas,

M r is a number of the plurality of receive antennas,

N 2 is a number of the plurality of sub-carriers, and

N 3 is a number of the plurality of receive moments.

6. The method of claim 5 , wherein obtaining the plurality of raw CSI samples further comprises extracting a CSI sub-array of size N 1 ×N 2 ×N 4 from the CSI array where 1≤N 4 <N 3 by:

randomly selecting N 4 sub-arrays of size N 1 ×N 2 out of N 3 sub-arrays of size N 1 ×N 2 in the CSI array; and

generating the CSI sub-array by stacking N 4 sub-arrays.

7. The method of claim 3 , wherein obtaining the mapping model comprises training a neural network by:

initializing the neural network with a plurality of initial weights; and

repeating an iterative process until a termination condition is satisfied, the iterative process comprising:

extracting a training image from an output of the neural network by applying the neural network on the i th subset of the first plurality of CSI samples;

generating a plurality of updated weights by minimizing a loss function of the training image and the i th video frame; and

replacing the plurality of updated weights with the plurality of initial weights.

8. The method of claim 7 , wherein applying the neural network on the i th subset of the first plurality of CSI samples comprises:

obtaining a first plurality of feature maps by applying a first plurality of convolutional layers of the neural network on the i th subset of the first plurality of CSI samples;

obtaining a second plurality of feature maps by feeding the first plurality of feature maps to an input of a fully connected layer of the neural network;

obtaining a third plurality of feature maps by applying a residual neural network (ResNet) of the neural network on the second plurality of feature maps; and

upsampling the third plurality of feature maps.

9. The method of claim 8 , wherein applying the first plurality of convolutional layers comprises extracting the first plurality of feature maps from an output of a (1, L 1 ) th convolutional layer of the first plurality of convolutional layers where L 1 is a number of the first plurality of convolutional layers, extracting the first plurality of feature maps comprising obtaining a (1, l 1 +1) th plurality of feature maps where 1≤l 1 ≤L 1 by:

generating a (1, L 1 ) th plurality of filtered feature maps by applying a (1, l 1 ) th plurality of filters on a (1, l 1 ) th plurality of feature maps, a (1, 1) st plurality of feature maps comprising the i th subset of the first plurality of CSI samples;

generating a (1, l 1 ) th plurality of normalized feature maps by applying an instance normalization process on the (1, l 1 ) th plurality of filtered feature maps; and

generating the (1, l 1 +1) th plurality of feature maps by implementing a (1, l 1 ) th non-linear activation function on each of the (1, l 1 ) th plurality of normalized feature maps.

10. The method of claim 8 , wherein obtaining the second plurality of feature maps further comprises:

extracting an output vector from an output of the fully connected layer;

generating a first latent feature map by generating a matrix from a plurality of elements in the output vector;

obtaining a second latent feature map by applying a padding process on the first latent feature map;

obtaining a fourth plurality of feature maps by applying a second plurality of convolutional layers of the neural network on the second latent feature map by extracting the fourth plurality of feature maps from an output of a (2, L 2 ) th convolutional layer of the second plurality of convolutional layers where L 2 is a number of the second plurality of convolutional layers, extracting the fourth plurality of feature maps comprising obtaining a (2, l 2 +1) th plurality of feature maps where 1≤l 2 ≤L 2 by:

generating a (2, l 2 ) th plurality of filtered feature maps by applying a (2, l 2 +1) th plurality of filters on a (2, l 2 ) th plurality of feature maps, a (2, 1) st plurality of feature maps comprising the second latent feature map;

generating a (2, l 2 ) th plurality of normalized feature maps by applying the instance normalization process on the (2, l 2 ) th plurality of filtered feature maps; and

generating the (2, l 2 +1) th plurality of feature maps by implementing a (2, l 2 ) th non-linear activation function on each of the (2, l 2 ) th plurality of normalized feature maps; and

generating the second plurality of feature maps by applying the padding process on each of the fourth plurality of feature maps.

11. The method of claim 8 , wherein applying the ResNet comprises applying a plurality of ResNet blocks by extracting the third plurality of feature maps from an output of an L r th ResNet block of the plurality of ResNet blocks where L r is a number of the plurality of ResNet blocks, extracting the third plurality of feature maps comprising obtaining an (l r +1) th plurality of residual feature maps by applying an l r th ResNet block of the plurality of ResNet blocks on an l r th plurality of residual feature maps where 1≤l r ≤L r , a first plurality of residual feature maps comprising the second plurality of feature maps and the l r th ResNet block comprising two cascaded convolutional layers and a residual connection.

12. The method of claim 8 , wherein upsampling the third plurality of feature maps comprises extracting the training image from an output of a (3, L 3 ) th convolutional layer of a third plurality of convolutional layers of the neural network where L 3 is a number of the third plurality of convolutional layers, extracting the training image comprising obtaining a (3, l 3 +1) th plurality of feature maps where 1≤ 3 ≤L 3 by:

generating an l 3 th plurality of upsampled feature maps by implementing an upsampling process on a (3, l 3 ) th plurality of feature maps, a (3, 1) st plurality of feature maps comprising the third plurality of feature maps;

generating a (3, l 3 ) th plurality of filtered feature maps by applying a (3, l 3 ) th plurality of filters on the l 3 th plurality of upsampled feature maps;

generating a (3, l 3 ) th plurality of normalized feature maps by applying the instance normalization process on the (3, l 3 ) th plurality of filtered feature maps; and

generating the (3, l 3 +1) th plurality of feature maps by implementing a (3, l 3 ) th non-linear activation function on each of the (3, l 3 ) th plurality of normalized feature maps.

13. The method of claim 7 , wherein minimizing the loss function of the training image and the i th video frame comprises:

obtaining a mask image by extracting a plurality of masks from the i th video frame, each of the plurality of masks associated with a respective dynamic object of a plurality of dynamic objects in the i th video frame;

obtaining a background-subtracted image by multiplying the mask image by the i th video frame; and

minimizing the loss function of the training image and the background-subtracted image.

14. A system for transferring data between different frequency bands by generating an image from channel state information (CSI) samples in a communication system, the system comprising:

a memory having processor-readable instructions stored therein; and

one or more processors configured to access the memory and execute the processor-readable instructions, which, when executed by the one or more processors configures the one or more processors to perform a method, the method comprising:

capturing video data associated with a plurality of video frames by capturing an i th video frame of the plurality of video frames in an i th time interval where i≥1;

obtaining, utilizing the communication system, a first plurality of CSI samples by obtaining an i th subset of the first plurality of CSI samples in the i th time interval;

obtaining a mapping model based on the plurality of video frames and the first plurality of CSI samples;

obtaining, utilizing the communication system, a second plurality of CSI samples; and

obtaining the image by applying the mapping model on the second plurality of CSI samples.

15. The system of claim 14 , wherein obtaining the i th subset of the first plurality of CSI samples comprises:

transmitting a plurality of transmit signals by a transmitter of the communication system, comprising transmitting each of the plurality of transmit signals in a respective sub-carrier of a plurality of sub-carriers and at a respective transmit moment of a plurality of transmit moments in the i th time interval by a respective transmit antenna of a plurality of transmit antennas associated with the transmitter;

receiving a plurality of receive signals associated with the plurality of transmit signals by a receiver of the communication system, comprising receiving each of the plurality of receive signals in a respective sub-carrier of the plurality of sub-carriers and at a respective receive moment of a plurality of receive moments in the i th time interval by a respective receive antenna of a plurality of receive antennas associated with the receiver; and

generating the i th subset of the first plurality of CSI samples by:

obtaining a plurality of raw CSI samples based on the plurality of receive signals; and

extracting the i th subset of the first plurality of CSI samples from the plurality of raw CSI samples by compensating a phase offset of each of the plurality of raw CSI samples.

16. The system of claim 15 , wherein obtaining the plurality of raw CSI samples comprises:

obtaining a CSI array of size N 1 ×N 2 ×N 3 by:

generating each of a plurality of CSI vectors of size

N

1

2

×

1

by estimating a respective multiple-input multiple output (MIMO) channel of a plurality of MIMO channels by processing a respective subset of the plurality of receive signals, each of the plurality of MIMO channels comprising a wireless channel between the transmitter and the receiver in a respective sub-carrier of the plurality of sub-carriers and at a respective receive moment of the plurality of receive moments; and

generating the CSI array by setting each element in the CSI array to one of a real part or an imaginary part of a respective element in a respective CSI vector of the plurality of CSI vectors,

wherein:

N 1 =2M t M r ,

M t is a number of the plurality of transmit antennas,

M r is a number of the plurality of receive antennas,

N 2 is a number of the plurality of sub-carriers, and

N 3 is a number of the plurality of receive moments; and

extracting a CSI sub-array of size N 1 ×N 2 ×N 4 from the CSI array where 1≤N 4 <N 3 by:

randomly selecting N 4 sub-arrays of size N 1 ×N 2 out of N 3 sub-arrays of size N 1 ×N 2 in the CSI array; and

generating the CSI sub-array by stacking N 4 sub-arrays.

17. The system of claim 14 , wherein obtaining the mapping model comprises training a neural network by:

initializing the neural network with a plurality of initial weights; and

repeating an iterative process until a termination condition is satisfied, the iterative process comprising:

extracting a training image from an output of the neural network by applying the neural network on the i th subset of the first plurality of CSI samples, comprising:

obtaining a first plurality of feature maps by applying a first plurality of convolutional layers of the neural network on the i th subset of the first plurality of CSI samples;

obtaining a second plurality of feature maps by feeding the first plurality of feature maps to an input of a fully connected layer of the neural network;

obtaining a third plurality of feature maps by applying a residual neural network (ResNet) of the neural network on the second plurality of feature maps; and

upsampling the third plurality of feature maps;

generating a plurality of updated weights by minimizing a loss function of the training image and the i th video frame; and

replacing the plurality of updated weights with the plurality of initial weights.

18. The system of claim 17 , wherein:

applying the first plurality of convolutional layers comprises extracting the first plurality of feature maps from an output of a (1, L 1 ) th convolutional layer of the first plurality of convolutional layers where L 1 is a number of the first plurality of convolutional layers, extracting the first plurality of feature maps comprising obtaining a (1, l 1 +1) th plurality of feature maps where 1≤l 1 ≤L 1 by:

generating a (1, l 1 ) th plurality of filtered feature maps by applying a (1, l 1 ) th plurality of filters on a (1, l 1 ) th plurality of feature maps, a (1, 1) st plurality of feature maps comprising the i th subset of the first plurality of CSI samples;

generating a (1, l 1 ) th plurality of normalized feature maps by applying an instance normalization process on the (1, l 1 ) th plurality of filtered feature maps; and

generating the (1, l 1 +1) th plurality of feature maps by implementing a (1, l 1 ) th non-linear activation function on each of the (1, l 1 ) th plurality of normalized feature maps; and

obtaining the second plurality of feature maps further comprises:

extracting an output vector from an output of the fully connected layer;

generating a first latent feature map by generating a matrix from a plurality of elements in the output vector;

obtaining a second latent feature map by applying a padding process on the first latent feature map;

obtaining a fourth plurality of feature maps by applying a second plurality of convolutional layers of the neural network on the second latent feature map by extracting the fourth plurality of feature maps from an output of a (2, L 2 ) th convolutional layer of the second plurality of convolutional layers where L 2 is a number of the second plurality of convolutional layers, extracting the fourth plurality of feature maps comprising obtaining a (2, l 2 +1) th plurality of feature maps where 1≤l 2 ≤L 2 by:

generating a (2, l 2 ) th plurality of filtered feature maps by applying a (2, l 2 ) th plurality of filters on a (2, l 2 ) th plurality of feature maps, a (2, 1) st plurality of feature maps comprising the second latent feature map;

generating a (2, l 2 ) th plurality of normalized feature maps by applying the instance normalization process on the (2, l 2 ) th plurality of filtered feature maps; and

generating the (2, l 2 +1) th plurality of feature maps by implementing a (2, l 2 ) th non-linear activation function on each of the (2, l 2 ) th plurality of normalized feature maps; and

generating the second plurality of feature maps by applying the padding process on each of the fourth plurality of feature maps.

19. The system of claim 17 , wherein:

applying the ResNet comprises applying a plurality of ResNet blocks by extracting the third plurality of feature maps from an output of an L r th ResNet block of the plurality of ResNet blocks where L r is a number of the plurality of ResNet blocks, extracting the third plurality of feature maps comprising obtaining an (l r +1) th plurality of residual feature maps by applying an l th r ResNet block of the plurality of ResNet blocks on an l r th plurality of residual feature maps where 1≤l r ≤L r , a first plurality of residual feature maps comprising the second plurality of feature maps and the l r th ResNet block comprising two cascaded convolutional layers and a residual connection; and

upsampling the third plurality of feature maps comprises extracting the training image from an output of a (3, L 3 ) th convolutional layer of a third plurality of convolutional layers of the neural network where L 3 is a number of the third plurality of convolutional layers, extracting the training image comprising obtaining a (3, l 3 +1) th plurality of feature maps where 1≤l 3 ≤L 3 by:

generating an l 3 th plurality of upsampled feature maps by implementing an upsampling process on a (3, l 3 ) th plurality of feature maps, a (3, 1) st plurality of feature maps comprising the third plurality of feature maps;

generating a (3, l 3 ) th plurality of filtered feature maps by applying a (3, l 3 ) th plurality of filters on the l 3 th plurality of upsampled feature maps;

generating a (3, l 3 ) th plurality of normalized feature maps by applying the instance normalization process on the (3, l 3 ) th plurality of filtered feature maps; and

generating the (3, l 3 +1) th plurality of feature maps by implementing a (3, l 3 ) th non-linear activation function on each of the (3, l 3 ) th plurality of normalized feature maps.

20. The system of claim 17 , wherein minimizing the loss function of the training image and the i th video frame comprises:

obtaining a mask image by extracting a plurality of masks from the i th video frame, each of the plurality of masks associated with a respective dynamic object of a plurality of dynamic objects in the i th video frame;

obtaining a background-subtracted image by multiplying the mask image by the i th video frame; and

minimizing the loss function of the training image and the background-subtracted image.