IP Library › Granted Patent US 10,834,532
Granted Patent B2
US 10,834,532 · App. 16/548,870 · Granted Nov 10, 2020

Method and system for wireless localization data acquisition and calibration with image localization

Inventors: Jonathan Fuerst (Heidelberg, DE); Guerkan Solmaz (Heidelberg, DE); Ernoe Kovacs (Stuttgart, DE); Kaifei Chen (Berkeley, CA)
Assignee: NEC LABORATORIES EUROPE GMBH
H04W4/023G06T7/75G06T2207/10024G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,834,532
App. No.
16/548,870
Granted
Nov 10, 2020
Kind
B2
Abstract

A method performs wireless localization data acquisition and calibration using a visual 3D model. The method includes: receiving image data and transmitter measurement data from a device; localizing the image data in the visual 3D model to determine a device location in a physical space; and using the device location and the transmitter measurement data to perform at least one of: determining transmitter fingerprint localization data and storing the transmitter fingerprint localization data in a fingerprint database; or calibrating a radio frequency (RF) propagation model for proximity estimation.

Claims (30)

1. A method for wireless localization data acquisition and calibration, the method comprising:

receiving image data and transmitter measurement data from a device;

automatically obtaining device location data by localizing the image data in a pre-constructed visual 3D model of a physical space to determine an estimation of a location of the device in the physical space; and

using the device location data and the transmitter measurement data to perform:

automatically determining transmitter fingerprint localization data by associating the device location data with the transmitter measurement data and storing the transmitter fingerprint localization data in a fingerprint database; and

automatically calibrating a radio frequency (RF) propagation model for proximity estimation based on the device location data and the transmitter measurement data.

2. The method according to claim 1 , wherein the transmitter fingerprint localization data comprises an RSSI value of a transmitter determined by the device, device characteristics of the device, and the device location.

3. The method according to claim 1 , wherein localizing the image data comprises performing six degrees of freedom (6DOF) image localization.

4. The method according to claim 3 , wherein the visual 3D model is a point cloud, and wherein localization comprises finding visually similar features among the 3D model and the image data and computing a geometric relation among the similar features to determine the 6DOF image localization.

5. The method according to claim 1 , the method further comprising, prior to performing the operation of localizing the image data, receiving location image data of the physical space, and constructing the 3D model of the physical space from the location image data.

6. The method according to claim 5 , wherein the location image data comprises RGB image data and depth data.

7. The method according to claim 1 , the method further comprising receiving a plurality of image data, which comprises the image data, and a plurality of transmitter measurement data, comprising the transmitter measurement data, from a plurality of devices, comprising the device.

8. The method according to claim 7 , wherein the devices are mobile devices of crowdsource users.

9. The method according to claim 1 , the method further comprising performing RSSI fingerprint based localization using the fingerprint database and the calibrated RF propagation model.

10. A system comprising a processor and a memory, the memory comprising processor executable instructions that, when executed by the processor, cause the processor to perform the following operations for wireless localization data acquisition and calibration:

receive image data and transmitter measurement data from a device;

automatically obtain device location data by localizing the image data in a pre-constructed visual 3D model of a physical space to determine an estimation of a location of the device in the physical space; and

use the device location data and the transmitter measurement data to:

automatically determine transmitter fingerprint localization data by associating the device location data with the transmitter measurement data, and storing the transmitter fingerprint localization data in a fingerprint database; and

automatically calibrate a radio frequency (RF) propagation model for proximity estimation based on the device location data and the transmitter measurement data.

11. The system according to claim 10 , wherein localizing the image data comprises performing six degrees of freedom (6DOF) image localization.

12. The system according to claim 11 , wherein the visual 3D model is a point cloud, and wherein localization comprises finding visually similar features among the 3D model and the image data and computing a geometric relation among the similar features to determine the 6DOF image localization.

13. The system according to claim 10 , the operations further comprising performing RSSI fingerprint based localization using the fingerprint database and the calibrated RF propagation model.

14. A non-transitory computer-readable medium comprising code for configuring one or more processors to:

receive image data and transmitter measurement data from a device;

automatically obtain device location data by localizing the image data in a pre-constructed visual 3D model of a physical space to determine an estimation of a location of the device in the physical space; and

use the device location data and the transmitter measurement data to:

automatically determine transmitter fingerprint localization data by associating the device location data with the transmitter measurement data, and storing the transmitter fingerprint localization data in a fingerprint database; and

automatically calibrate a radio frequency (RF) propagation model for proximity estimation based on the device location data and the transmitter measurement data.

15. The computer-readable medium according to claim 14 , wherein the visual 3D model is a point cloud, and wherein localization comprises finding visually similar features among the 3D model and the image data and computing a geometric relation among the similar features to determine six degrees of freedom image localization.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2020
From: FUERST, JONATHAN; SOLMAZ, GUERKAN; KOVACS, ERNOE
To: NEC LABORATORIES EUROPE GMBH
Reel/Frame 053927/0159 →
Continuity (2)
Provisional Application 62721652 · Aug 23, 2018
Related Publication 20200068344A1 · Feb 27, 2020
Cited By (2)
US 12,273,154 US 12,738,046