IP Library › Granted Patent US 11,006,383
Granted Patent B2
US 11,006,383 · App. 16/684,746 · Granted May 11, 2021

Mapping and localization using image processing of wireless signals

Inventors: Georgios Trichopoulos (Tempe, AZ); Ahmed Alkhateeb (Chandler, AZ)
Assignee: Arizona Board of Regents on Behalf of Arizona State University
H04W64/00G01S5/0215G01S5/0273G01S5/06G01S5/14H04B3/52H04B7/0413H04B7/0617
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,006,383
App. No.
16/684,746
Granted
May 11, 2021
Kind
B2
Abstract

Mapping and localization using image processing of wireless signals is provided. Embodiments of the present disclosure provide a novel approach for high accuracy mapping of an environment around an antenna (or antenna array) coupled to a radio frequency (RF) transceiver through image processing of RF signals. The image processing includes constructing a map of line-of-sight (LOS) and non-line-of-sight (NLOS) objects in the environment by distinguishing NLOS objects and correctly projecting their positions relative to the LOS objects. In some examples, a three-dimensional (3D) image of the environment around the antenna (or antenna array) is produced. Aspects disclosed herein can further provide simultaneous localization and mapping (SLAM) for a wireless device.

Claims (53)

1. A method for mapping an environment, the method comprising:

transmitting a plurality of radio frequency (RF) signals in the environment;

receiving, at an antenna, an RF response to the plurality of RF signals; and

mapping the environment, comprising:

initially mapping a plurality of objects comprising a line-of-sight (LOS) object and a non-line-of-sight (NLOS) object in the environment based on a scattering of the plurality of RF signals indicated in the RF response, wherein the LOS object is within direct LOS of the antenna and the NLOS object is not within the direct LOS of the antenna;

identifying the NLOS object from the RF response; and

remapping the NLOS object based on relative positioning of the LOS object, the NLOS object, and the antenna receiving the RF response.

2. The method of claim 1 , wherein the mapping of the environment comprises producing a three-dimensional (3D) image of the LOS object and the NLOS object.

3. The method of claim 1 , wherein the mapping of the environment further comprises identifying a plurality of LOS objects comprising the LOS object and a plurality of NLOS objects comprising the NLOS object.

4. The method of claim 1 , wherein:

the plurality of RF signals are transmitted from an antenna array; and

the RF response is received over the antenna array.

5. The method of claim 4 , wherein identifying the LOS object and the NLOS object of the plurality of objects comprises:

the identifying the LOS object having a surface within a line of sight of the antenna array; and

the identifying the NLOS object being positioned out of the line of sight of the antenna array.

6. The method of claim 5 , wherein the mapping of the NLOS object comprises ray tracing from the surface of the LOS object to project a location of the NLOS object.

7. The method of claim 1 , further comprising:

determining a position of a reflective surface which causes primarily specular reflection of the plurality of RF signals; and

correcting the mapping of the environment to account for the position of the reflective surface.

8. The method of claim 7 , wherein the position of the reflective surface is determined through a machine learning algorithm.

9. The method of claim 1 , wherein each of the plurality of RF signals has a frequency between 30 gigahertz (GHz) and 10 terahertz (THz).

10. The method of claim 1 , wherein each of the plurality of RF signals has a frequency between 70 gigahertz (GHz) and 1 terahertz (THz).

11. The method of claim 1 , further comprising receiving sensor data from at least one of an infrared sensor, an optical sensor, or a camera;

wherein the mapping of the environment is further based on the sensor data.

12. The method of claim 1 , further comprising:

receiving a communication signal from a device; and

locating the device within the mapped environment based on the communication signal.

13. A mapping device, comprising:

an antenna;

a transceiver configured to receive a radio frequency (RF) response to a plurality of transmitted RF signals via the antenna; and

a processing device configured to map an environment around the antenna by:

initially mapping a plurality of objects comprising a line-of-sight (LOS) object and a non-line-of-sight (NLOS) object in the environment based on a scattering of the plurality of transmitted RF signals indicated in the RF response, wherein the LOS object is within direct LOS of the antenna and the NLOS object is not within the direct LOS of the antenna;

identifying the NLOS object from the RF response; and

remapping the NLOS object based on relative positioning of the LOS object, the NLOS object, and the antenna.

14. The mapping device of claim 13 , wherein the antenna comprises an antenna array.

15. The mapping device of claim 13 , wherein the processing device is further configured to map the environment around the antenna by producing a three-dimensional (3D) image of the LOS object, the NLOS object, and additional objects in the environment.

16. The mapping device of claim 13 , wherein:

the transceiver comprises a millimeter wave (mmWave) transceiver;

the transceiver is further configured to receive a communication signal from an RF device; and

the processing device is further configured to locate the RF device within the map of the environment from the communication signal.

17. An imaging system, comprising:

a first antenna array; and

a processing device in communication with the first antenna array, the processing device configured to:

receive a first radio frequency (RF) response to a plurality of transmitted RF signals from the first antenna array;

initially map a plurality of objects comprising a first line-of-sight (LOS) object and a non-line-of-sight (NLOS) object around the first antenna array based on a scattering of the plurality of transmitted RF signals indicated in the first RF response, wherein the LOS object is within direct LOS of the first antenna array and the NLOS object is not within the direct LOS of the first antenna array;

identify the NLOS object from the first RF response; and

remap the NLOS object based on relative positioning of the first LOS object, the NLOS object, and the first antenna array to produce a map of an environment around the first antenna array.

18. The imaging system of claim 17 , further comprising a second antenna array in communication with the processing device.

19. The imaging system of claim 18 , wherein the processing device is further configured to:

receive a second RF response from the second antenna array;

map a second LOS object around the second antenna array from the second RF response, wherein the second LOS object is within direct LOS of the second antenna array; and

map the NLOS object further using the mapped second LOS object.

20. The imaging system of claim 17 , wherein the processing device is further configured to remap the NLOS object by ray tracing from a surface of the LOS object to project a location of the NLOS object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2019
From: TRICHOPOULOS, GEORGIOS; ALKHATEEB, AHMED
To: ARIZONA BOARD OF REGENTS ON BEHALF OF ARIZONA STATE UNIVERSITY
Reel/Frame 051026/0946 →
Continuity (2)
Provisional Application 62767820 · Nov 15, 2018
Related Publication 20200163040A1 · May 21, 2020
Cited By (1)
US 12,413,932