IP Library Granted Patent US 9,411,037
Granted Patent B2
US 9,411,037 · App. 13/212,000 · Granted Aug 9, 2016

Calibration of Wi-Fi localization from video localization

Inventors: Mark Jamtgaard (Mountain View, CA); Nathan Mueller (Mountain View, CA)
Assignee: RETAILNEXT, INC.
G01S5/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,411,037
App. No.
13/212,000
Granted
Aug 9, 2016
Kind
B2
Abstract

In some implementations, video camera networks can be used to track people or objects and determine their locations. In some implementations, location accuracy better than one meter can be achieved using video cameras. According to some implementations, Wi-Fi localization and video localization systems can be fused together to perform calibration. In some implementations, by using the video network to continuously update the Wi-Fi localization calibration database, Wi-Fi location accuracy can be improved.

Claims (68)

1. A method comprising:

obtaining, from a transmitter associated with a movable target, wireless signal pattern data at a plurality of access points corresponding to a plurality of fixed locations, the wireless signal pattern data including a signal strength measured by the plurality of access points at the plurality of fixed locations;

determining a wireless feature vector associated with the movable target, the wireless feature vector being determined based on the wireless signal pattern data;

obtaining video data associated with the movable target;

calculating occupancy probabilities of the movable target based on the video data;

estimating a location of the movable target based on the wireless feature vector and the occupancy probabilities; and

updating calibration data for the plurality of access points corresponding to the plurality of fixed locations based on the estimated location of the movable target.

2. The method of claim 1 , wherein obtaining the wireless signal pattern data is based on receiving wireless signals.

3. The method of claim 1 ,

wherein the calibration data is further updated based on the wireless signal feature vector.

4. The method of claim 1 , wherein estimating the location of the movable target further comprises:

determining a first location of an object based on the video data;

determining a second location of the transmitter based on the wireless signal pattern data obtained from the transmitter; and

determining, based on the first location and the second location, that the object corresponds to the transmitter.

5. The method of claim 1 , wherein estimating the location of the movable target further comprises:

tracking movement of an object along a first path based on the video data;

tracking movement of the transmitter along a second path based on the wireless signal pattern data; and

determining, based on the first path and the second path, that the object corresponds to the transmitter.

6. The method of claim 1 , wherein estimating the location of the movable target further comprises:

receiving video data from a plurality of cameras having overlapping fields of view, the video data including video of an object in the overlapping fields of view; and

determining a three-dimensional position of the object based on the video data received from the plurality of cameras.

7. The method of claim 1 , wherein the wireless signal pattern data identifies each of the plurality of fixed locations.

8. A system comprising:

one or more processors; and

a computer-readable medium including one or more sequences of instructions which, when executed by the one or more processors, causes:

obtaining, from a transmitter associated with a movable target, wireless signal pattern data at a plurality of access points corresponding to a plurality of fixed locations from a transmitter associated with a movable target, the wireless signal pattern data including a signal strength measured by the plurality of access points at the plurality of fixed locations;

determining a wireless feature vector associated with the movable target, the wireless feature vector being determined based on the wireless signal pattern data;

obtaining video data associated with the movable target;

determining occupancy probabilities of the movable target based on the video data;

estimating a location of the movable target based on the wireless feature vector and the occupancy probabilities; and

updating calibration data for the plurality of access points corresponding to the plurality of fixed locations based on the estimated location of the movable target.

9. The system of claim 8 , wherein the instructions for obtaining the wireless signal pattern data is based on receiving wireless signals.

10. The system of claim 8 , wherein

the calibration data is further updated based on the wireless signal feature vector.

11. The system of claim 8 , wherein the instructions that cause estimating the location of the target further comprise instructions that cause:

determining a first location of an object based on the video data;

determining a second location of the transmitter based on the wireless signal pattern data obtained from the transmitter; and

determining, based on the first location and the second location, that the object corresponds to the transmitter.

12. The system of claim 8 , wherein the instructions that cause estimating the location of the movable target further comprise instructions that cause:

tracking movement of an object along a first path based on the video data;

tracking movement of the transmitter along a second path based on the wireless signal pattern data; and

determining, based on the first path and the second path, that the object corresponds to the transmitter.

13. The system of claim 8 , wherein the instructions that cause estimating the location of the movable target further comprise instructions that cause:

receiving video data from a plurality of cameras having overlapping fields of view, the video data including video of an object in the overlapping fields of view; and

determining a three-dimensional position of the object based on the video data received from the plurality of cameras.

14. The system of claim 8 , wherein the wireless signal pattern data identifies each of the plurality of fixed locations.

15. A non-transitory computer-readable medium including one or more sequences of instructions which, when executed by one or more processors, causes:

obtaining, from a transmitter associated with a movable target, wireless signal pattern data at a plurality of access points corresponding to a plurality of fixed locations, the wireless signal pattern data including a signal strength measured by the plurality of access points at the plurality of fixed locations;

determining a wireless feature vector associated with the movable target, the wireless feature vector being determined based on the wireless signal pattern data;

obtaining video data associated with the movable target;

calculating occupancy probabilities of the movable target based on the video data;

estimating a location of the movable target based on the wireless feature vector and the occupancy probabilities; and

updating calibration data for the plurality of access points corresponding to the plurality of fixed locations based on the estimated location of the movable target.

16. The non-transitory computer-readable medium of claim 15 , wherein the instructions for obtaining the wireless signal pattern data is based on receiving wireless signals.

17. The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause:

updating the wireless signal pattern data based on the wireless signal feature vector.

18. The non-transitory computer-readable medium of claim 15 , wherein the instructions that cause estimating the location of the movable target comprise instructions that cause:

determining a first location of an object based on the video data;

determining a second location of the wireless transmitter based on the wireless signals received from the wireless transmitter; and

determining, based on the first location and the second location, that the object corresponds to the wireless transmitter.

19. The non-transitory computer-readable medium of claim 15 , wherein the instructions that cause estimating the location of the movable target comprise instructions that cause:

tracking movement of an object along a first path based on the video data;

tracking movement of the wireless transmitter along a second path based on the wireless signals received from the wireless transmitter; and

determining, based on the first path and the second path, that the object corresponds to the wireless transmitter.

20. The non-transitory computer-readable medium of claim 15 , wherein the instructions that cause determining occupancy probabilities of the movable target based on the video data comprise instructions that cause:

receiving the video data from a plurality of cameras having overlapping fields of view, the video data including video of an object in the overlapping fields of view; and

determining a three-dimensional position of the movable object based on the video data.

21. The non-transitory computer-readable medium of claim 15 , wherein the wireless signal pattern data identifies each of the plurality of fixed locations.

Assignments (17)
RELEASE OF SECURITY INTEREST Recorded Dec 6, 2024
From: MGG INVESTMENT GROUP LP
To: RETAILNEXT, INC.
Reel/Frame 069511/0217 →
SECURITY INTEREST Recorded Dec 5, 2024
From: RETAILNEXT, INC.
To: BAIN CAPITAL CREDIT, LP, AS ADMINISTRATIVE AGENT
Reel/Frame 069495/0690 →
RELEASE OF SECURITY INTEREST Recorded Jul 18, 2023
From: ALTER DOMUS (US) LLC
To: RETAILNEXT, INC.
Reel/Frame 064298/0437 →
SECURITY INTEREST Recorded Jul 13, 2023
From: RETAILNEXT, INC.
To: EAST WEST BANK, AS ADMINISTRATIVE AGENT
Reel/Frame 064247/0925 →
RELEASE OF SECURITY INTEREST Recorded Apr 27, 2021
From: ORIX GROWTH CAPITAL, LLC
To: RETAILNEXT, INC.
Reel/Frame 056056/0825 →
RELEASE OF SECURITY INTEREST Recorded Apr 27, 2021
From: SILICON VALLEY BANK
To: RETAILNEXT, INC.
Reel/Frame 056055/0587 →
SECURITY INTEREST Recorded Apr 23, 2021
From: RETAILNEXT, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 056018/0344 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECTLY IDENTIFIED PATENT APPLICATION NUMBER 14322624 TO PROPERLY REFLECT PATENT APPLICATION NUMBER 14332624 PREVIOUSLY RECORDED ON REEL 044252 FRAME 0867. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Mar 25, 2020
From: RETAILNEXT, INC.
To: SILICON VALLEY BANK
Reel/Frame 053119/0599 →
RELEASE OF SECURITY INTEREST Recorded Aug 28, 2018
From: TRIPLEPOINT VENTURE GROWTH BDC CORP.
To: RETAILNEXT, INC.
Reel/Frame 046957/0896 →
SECURITY INTEREST Recorded Aug 27, 2018
From: RETAILNEXT, INC.
To: ORIX GROWTH CAPITAL, LLC
Reel/Frame 046715/0067 →
RELEASE OF SECURITY INTEREST Recorded Aug 24, 2018
From: WESTERN ALLIANCE BANK AS SUCCESSOR-IN-INTEREST TO BRIDGE BANK
To: NEARBUY SYSTEMS, INC.
Reel/Frame 046692/0481 →
RELEASE OF SECURITY INTEREST Recorded Nov 29, 2017
From: WESTERN ALLIANCE BANK
To: RETAILNEXT, INC.
Reel/Frame 044252/0769 →
SECURITY INTEREST Recorded Nov 29, 2017
From: RETAILNEXT, INC.
To: SILICON VALLEY BANK
Reel/Frame 044252/0867 →
SECURITY INTEREST Recorded Nov 20, 2017
From: RETAILNEXT, INC.
To: TRIPLEPOINT VENTURE GROWTH BDC CORP.
Reel/Frame 044176/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 23, 2016
From: NEARBUY SYSTEMS, INC.
To: RETAILNEXT, INC.
Reel/Frame 038683/0756 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2013
From: JAMTGAARD, MARK; MUELLER, NATHAN
To: NEARBUY SYSTEMS, INC.
Reel/Frame 031392/0433 →
SECURITY AGREEMENT Recorded Sep 13, 2013
From: NEARBUY SYSTEMS, INC.
To: BRIDGE BANK, NATIONAL ASSOCIATION
Reel/Frame 031266/0654 →
Continuity (2)
Provisional Application 61374996 · Aug 18, 2010
Related Publication 20120044355A1 · Feb 23, 2012