IP Library Granted Patent US 11,089,432
Granted Patent B2
US 11,089,432 · App. 16/568,747 · Granted Aug 10, 2021

Tracking indoor objects from an unmanned aerial vehicle

Inventors: Karthikeyan Sundaresan (Manalapan, NJ); Ayon Chakraborty (Plainsboro, NJ); Sampath Rangarajan (Bridgewater, NJ); Ashutosh Dhekne (Atlanta, GA)
H04W4/02H04B7/18506
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,089,432
App. No.
16/568,747
Granted
Aug 10, 2021
Kind
B2
Abstract

Systems and methods for localizing and tracking mobile objects are provided. The method includes determining an initial location of a node based on multi-lateration from an unmanned aerial vehicle and determining a velocity vector associated with the node based on multi-lateration. The method also includes detecting when the node turns. An adaptive aperture is applied to address a non-uniform velocity of the node based on the turn and the velocity vector.

Claims (48)

1. A method for localizing and tracking a mobile object inside a structure, comprising:

creating a synthetic aperture of anchor points from which an unmanned aerial vehicle (UAV) generates range estimates to all reachable ultra-wideband (UWB) nodes from a plurality of reference UWB nodes positioned inside the structure;

determining an initial location and velocity of the UWB node connected to the mobile object based on multi-lateration from the UAV, the range estimates to the reachable UWB nodes, and inertial sensors connected to the UWB node configured to detect and communicate an initial velocity of the mobile object to the UAV, wherein the UAV includes a UAV UWB node in communication with the UWB node of the mobile object and the plurality of reference UWB nodes;

determining a velocity vector associated with the UWB node of the mobile object based on the multi-lateration from the UAV and the inertial sensors connected to the UWB node;

detecting a turning of the UWB node of the mobile object by identification of a non-uniform velocity of the UWB node of the mobile object; and

applying, by a processor device, an adaptive aperture to address the non-uniform velocity of the UWB node of the mobile object based on the turning of the UWB node of the mobile object and the velocity vector associated with the UWB node of the mobile object to improve localization accuracy by removing all history and restarting estimating location of the UWB node on the mobile object when the turning of the UWB node of the mobile object is detected.

2. The method as recited in claim 1 , further comprising:

providing a visualization on an associated display of a position of the mobile object based on a position of the UWB of the mobile object.

3. The method as recited in claim 1 , wherein determining the initial location of the UWB node of the mobile object further comprises:

determining the initial location based on ultra-wideband signaling along with input from inertial sensors of the UAV.

4. The method as recited in claim 1 , further comprising:

determining an absolute location fix of the UAV based on at least one radio frequency beacon positioned on an associated ground vehicle.

5. The method as recited in claim 1 , further comprising utilizing a plurality of UAVs by deploying each of the plurality of UAVs to a different section of a building to provide wider coverage.

6. The method as recited in claim 1 , wherein the plurality of UWB nodes are each connected to a different mobile object from a plurality of mobile objects, the plurality of UWB nodes of the mobile objects being utilized for applying a multi-hop localization process based on a reachability of each of the plurality of UWB nodes from the UAV.

7. The method as recited in claim 6 , wherein the applying the multi-hop localization process further comprises:

dynamically estimating, by each of the plurality of UWB nodes of the mobile objects, a hop status of the each of the plurality of UWB nodes of the mobile objects based on one overheard ranging message from one neighboring UWB node of one of the mobile objects.

8. The method of claim 6 , wherein the plurality of UWB nodes of the mobile objects includes initial UWB nodes in a particular hop and subsequent UWB nodes in a subsequent hop, further comprising:

implementing the initial UWB nodes in the particular hop to serve as beacons to localize the subsequent UWB nodes in the subsequent hop.

9. The method as recited in claim 6 , further comprising:

parallelizing ranging measurements within each hop; and

eliminating redundant message transmissions.

10. The method as recited in claim 1 , further comprising:

collecting ranging information from each of the UWB at least one node of the mobile object by the UAV at least one unmanned aerial vehicle.

11. A computer system for localizing and tracking a mobile object, comprising:

a processor device operatively coupled to a memory device, the processor device being configured to:

create a synthetic aperture of anchor points from which an unmanned aerial vehicle (UAV) generates range estimates to all reachable ultra-wideband (UWB) nodes from a plurality of reference UWB nodes positioned inside the structure

an initial location and velocity of the UWB node connected to the mobile object based on multi-lateration from the UAV, the range estimates to the reachable UWB nodes, and inertial sensors connected to the UWB node configured to detect and communicate an initial velocity of the mobile object to the UAV, wherein the UAV includes a UAV UWB node in communication with the UWB node of the mobile object and the plurality of reference UWB nodes;

determine a velocity vector associated with the UWB node of the mobile object based on the multi-lateration from the UAV and the inertial sensors connected to the UWB node;

detect a turning of the UWB node of the mobile object by identification of a non-uniform velocity of the UWB node of the mobile object; and

apply an adaptive aperture to address the non-uniform velocity of the UWB node of the mobile object based on the turning of the UWB node of the mobile object and the velocity vector associated with the UWB node of the mobile object to improve localization accuracy by removing all history and restarting estimating location of the UWB node on the mobile object when the turning of the UWB node of the mobile object is detected.

12. The system as recited in claim 11 , wherein the processor device is further configured to:

provide a visualization on an associated display of a position of the mobile object based on a position of the UWB of the mobile object.

13. The system as recited in claim 11 , wherein, when determining the initial location of the UWB node of the mobile object, the processor device is further configured to:

determine the initial location based on ultra-wideband signaling along with input from inertial sensors of the UAV.

14. The system as recited in claim 11 , wherein the processor device is further configured to:

determine an absolute location fix of the UAV vehicle based on at least one radio frequency beacon positioned on an associated ground vehicle.

15. The system as recited in claim 11 , comprising a plurality of UAVs and wherein the processor device is further configured to:

deploy each of the plurality of UAVs to a different section of a building to provide wider coverage.

16. The system as recited in claim 11 , wherein the plurality of UWB nodes are each connected to a different mobile object from a plurality of mobile objects, the plurality of UWB nodes of the mobile objects being utilized to:

apply a multi-hop localization process based on a reachability of each of the plurality of UWB nodes from the UAV.

17. The system as recited in claim 16 , wherein, when executing the applying the multi-hop localization process, the processor device is further configured to:

dynamically estimate, by each of the plurality of UWB nodes of the mobile objects, a hop status of the each of the plurality of UWB nodes of the mobile objects based on one overheard ranging message from one neighboring UWB node of one of the mobile objects.

18. A computer program product for localizing and tracking a mobile object inside a structure, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to perform the method comprising:

creating a synthetic aperture of anchor points from which an unmanned aerial vehicle (UAV) generates range estimates to all reachable ultra-wideband (UWB) nodes from a plurality of reference UWB nodes positioned inside the structure

determining an initial location and velocity of the UWB node connected to the mobile object based on multi-lateration from the UAV, the range estimates to the reachable UWB nodes, and inertial sensors connected to the UWB node configured to detect and communicate an initial velocity of the mobile object to the UAV, wherein the UAV includes a UAV UWB node in communication with the UWB node of the mobile object and the plurality of reference UWB nodes;

determining a velocity vector associated with the UWB node of the mobile object based on the multi-lateration from the UAV and the inertial sensors connected to the UWB node;

detecting a turning of the UWB node of the mobile object by identification of a non-uniform velocity of the UWB node of the mobile object; and

applying, by a processor device, an adaptive aperture to address the non-uniform velocity of the UWB node of the mobile object based on the turning of the UWB node of the mobile object and the velocity vector associated with the UWB node of the mobile object to improve localization accuracy by removing all history and restarting estimating location of the UWB node on the mobile object when the turning of the UWB node of the mobile object is detected.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 17, 2021
From: NEC LABORATORIES AMERICA, INC.
To: NEC CORPORATION
Reel/Frame 056570/0807 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2019
From: SUNDARESAN, KARTHIKEYAN; CHAKRABORTY, AYON; RANGARAJAN, SAMPATH; DHEKNE, ASHUTOSH
To: NEC LABORATORIES AMERICA, INC.
Reel/Frame 050357/0865 →
Continuity (2)
Provisional Application 62733670 · Sep 20, 2018
Related Publication 20210067899A1 · Mar 4, 2021
Cited By (1)
US 12,524,015