IP Library Granted Patent US 11,467,247
Granted Patent B2
US 11,467,247 · App. 17/107,372 · Granted Oct 11, 2022

Vision and radio fusion based precise indoor localization

Inventors: Mi S. Park (Hillsboro, OR); Lei Yang (Hillsboro, OR); Shao-Wen Yang (Hillsboro, OR); Myung Hwangbo (Lake Oswego, OR); Shahrokh Shahidzadeh (Portland, OR)
Assignee: Intel Corporation
G01S5/0263G01C21/206G06K9/6289G06T7/30G06V20/52G06V40/20H04W4/021H04W4/33G01S5/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,467,247
App. No.
17/107,372
Granted
Oct 11, 2022
Kind
B2
Abstract

An embodiment includes at least one computer readable storage medium comprising instructions that when executed enable a system to: receive (a)(i) first radio signal location data for a first object from a radio sensor; and (a)(ii) first visual signal location data for the first object from a camera sensor; perform feature extraction on (b)(i) the first radio signal location data to determine first extracted radio signal features; and (b)(ii) the first visual signal location data to determine first extracted visual signal features; solve a first association problem between the first extracted radio signal features and the first extracted visual signal features to determine first fused location data; and store the first fused location data in the at least one computer readable storage medium. Other embodiments are described herein.

Claims (49)

1. At least one storage device comprising instructions which, when executed, cause at least one processor to at least:

generate localization information corresponding to a location of a tracked object, wherein the tracked object is a movable object, the localization information based on off-device data from a first electronic device separate from the tracked object and other data based on an output of an environmental sensor carried by the tracked object, the off-device data based at least in part on camera data, wherein the localization information is based on a fusion of the off-device data and the other data; and

initiate upload of the localization information to a second electronic device for update of a map at the second electronic device.

2. The at least one storage device of claim 1 , wherein the first electronic device includes a camera.

3. The at least one storage device of claim 2 , wherein the camera is a stationary camera.

4. The at least one storage device of claim 1 , wherein the environmental sensor is internal to the tracked object.

5. The at least one storage device of claim 1 , wherein the localization information is to represent an orientation of the tracked object.

6. The at least one storage device of claim 1 , wherein the instructions, when executed, cause the at least one processor to cause display of the map with the location of the tracked object.

7. The at least one storage device of claim 1 , wherein the instructions, when executed, cause the at least one processor to calibrate timestamps, the timestamps associated with the output of the environment sensor.

8. The at least one storage device of claim 7 , wherein the calibration of the timestamps is based on a network time protocol.

9. A system comprising:

an environmental sensor to be carried by a tracked object, wherein the tracked object is a movable object;

a transceiver to be carried by the tracked object;

memory to be carried by the tracked object;

processor circuitry to be carried by the tracked object, the processor circuitry to execute instructions to:

generate localization information corresponding to a location of the tracked object, the localization information based on: (a) off-device data from a first electronic device separate from the tracked object, and (b) other data based on an output of the environmental sensor, the off-device data based at least in part on camera data, wherein the localization information is based on a fusion of the off-device data and the other data; and

initiate upload of the localization information to a second electronic device for update of a map at the second electronic device.

10. The system of claim 9 , wherein the transceiver is a wireless local area network transceiver.

11. The system of claim 9 , wherein the first electronic device includes a monocular camera.

12. The system of claim 9 , wherein the environmental sensor is to measure a signal at the tracked object.

13. The system of claim 9 , further including the tracked object, wherein the tracked object is ambulatory.

14. The system of claim 9 , wherein the localization information is to represent an orientation of the tracked object.

15. The system of claim 9 , further including a display to present the map including the location of the tracked object.

16. The system of claim 9 , wherein the processor circuitry is to calibrate timestamps, the timestamps associated with the output of the environmental sensor.

17. The system of claim 16 , wherein the calibration of the timestamps is based on a network time protocol.

18. A method comprising:

generating, by executing instructions with at least one processor, localization information corresponding to a location of a tracked object, wherein the tracked object is a movable object, the localization information based on: (a) off-device data from a first electronic device separate from the tracked object, and (b) other data based on an output of an environmental sensor, the off-device data based at least in part on image data, wherein the localization information is based on a fusion of the off-device data and the other data; and

initiating upload of the localization information to a second electronic device for update of a map at the second electronic device.

19. The method of claim 18 , further including calibrating timestamps, the timestamps associated with the output of the environmental sensor.

20. The method of claim 19 , wherein the calibrating of the timestamps is based on a network time protocol.

21. The method of claim 18 , further including presenting a map to identify the location of the tracked object.

22. The method of claim 18 , wherein the first electronic device includes a camera.

23. The method of claim 22 , wherein the camera is a stationary camera.

24. The method of claim 18 , wherein the environmental sensor is internal to the tracked object.

25. The method of claim 18 , further including accessing the off-device data by communicating using a wireless local area network transceiver.

26. The method of claim 18 , further including measuring a signal at the tracked object using the environmental sensor.

27. The method of claim 18 , wherein the localization information represents an orientation of the tracked object.

28. A system comprising:

means for sensing an environment, the sensing means carried by a tracked object;

means for wirelessly communicating;

means for storing;

means for executing instructions to:

generate localization information corresponding to a location of the tracked object, wherein the tracked object is a movable object, the localization information based on: (a) off-device data from a first electronic device separate from the tracked object, and (b) other data based on an output of the means for sensing, the off-device data based at least in part on camera data, wherein the localization information is based on a fusion of the off-device data and the other data; and

initiate upload of the localization information to a second electronic device for update of a map at the second electronic device.

29. The system of claim 28 , wherein the localization information represents an orientation of the tracked object.

30. The system of claim 28 , further including means for displaying a map including the location of the tracked object.

31. The system of claim 28 , wherein the first electronic device includes a camera.

32. The system of claim 31 , wherein the camera is a monocular camera.

33. The system of claim 28 , wherein the means for sensing is to measure a signal at the tracked object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2021
From: PARK, MI S.; YANG, LEI; YANG, SHAO-WEN; HWANGBO, MYUNG; SHAHIDZADEH, SHAHROKH
To: INTEL CORPORATION
Reel/Frame 055185/0059 →
Continuity (3)
Continuation 16776233 · Jan 29, 2020
Continuation 15716041 · Sep 26, 2017
Related Publication 20210149012A1 · May 20, 2021
Cited By (1)
US 12,696,046