IP Library Granted Patent US 11,257,225
Granted Patent B2
US 11,257,225 · App. 16/663,948 · Granted Feb 22, 2022

Sensor mapping to a global coordinate system using homography

Inventors: Shahmeer Ali Mirza (Celina, TX); Sailesh Bharathwaaj Krishnamurthy (Irving, TX); Crystal Maung (Dallas, TX)
Assignee: 7-ELEVEN, INC.
G06T7/248G06T7/74G06T2207/20216G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,257,225
App. No.
16/663,948
Granted
Feb 22, 2022
Kind
B2
Abstract

An object tracking system that includes a plurality of sensors and a tracking system. A first sensor from the plurality of sensors is configured to capture a first frame of a global plane for at least a portion of the space. The tracking system is configured to determine a pixel location in the first frame for an object located in the space, and to apply a homography to the pixel location to determine a coordinate in the global plane. The homography is configured to translate between pixel locations in the first frame and coordinates in the global plane.

Claims (88)

1. An object tracking system, comprising:

a plurality of sensors configured in a sensor array positioned above at least a portion of a space, wherein:

a first sensor from the plurality of sensors is configured to capture a first frame of a global plane for at least a portion of the space;

a second sensor from the plurality of sensors is configured to capture a second frame of the global plane for at least a second portion of the space;

the first frame comprises a plurality of pixels; and

each pixel from the plurality of pixels is associated with a pixel location comprising a pixel row and a pixel column; and

a tracking system operably coupled to the plurality of sensors, comprising:

one or more memories operable to store:

a first homography associated with the first sensor, wherein:

the first homography is configured to translate between pixel locations in the first frame and (x,y) coordinates in the global plane; and

a second homography associated with the second sensor, wherein:

the second homography comprises coefficients that translate between the pixel locations in the second frame and (x,y) coordinates in the global plane; and

coefficients of the second homography are different from coefficients of the first homography; and

one or more processors operably coupled to the one or more memories, configured to:

receive the first frame;

determine a third pixel location in the first frame for an object located in the space, wherein the third pixel location comprises a first pixel row and a first pixel column of the first frame;

apply the first homography to the third pixel location to determine a third (x,y) coordinate identifying a third x-value and a third y-value in the global plane;

receive the second frame;

determine a fourth pixel location in the second frame for the object located in the space; and

apply the second homography to the fourth pixel location to determine a fourth (x,y) coordinate identifying a fourth x-value and a fourth y-value in the global plane.

2. The system of claim 1 , wherein the one or more processors are further configured to:

generate an average (x,y) coordinate for the object by computing an average of the third (x,y) coordinate of the object and the fourth (x,y) coordinate of the object.

3. The system of claim 1 , wherein the one or more processors are further configured to:

generate a median (x,y) coordinate for the object by computing a median of the third (x,y) coordinate of the object and the fourth (x,y) coordinate of the object.

4. The system of claim 1 , wherein:

the third (x,y) coordinate is the same as the fourth (x,y) coordinate; and

the third (x,y) coordinate and the fourth (x,y) coordinate are the (x,y) coordinate for the object in the global plane.

5. The system of claim 1 , wherein:

the one or more memories is further operable to store a tracking list associated with the first sensor, wherein the tracking list identifies:

an object identifier for the object; and

the third (x,y) coordinate for the object; and

the one or more processors are further configured to store the third (x,y) coordinate in a tracking list associated with the first sensor in response to determining the third (x,y) coordinate.

6. The system of claim 1 , wherein:

each pixel in the first frame is associated with a pixel value; and

the first homography is further configured to translate between pixel values in the first frame and z-coordinates in the global plane.

7. The system of claim 1 , wherein the sensor array is positioned parallel with the global plane.

8. An object tracking method, comprising:

receiving a first frame from a first sensor of a plurality of sensors configured in a sensor array above at least a portion of a space, wherein:

the first frame is of a global plane for the at least a portion of the space;

the first frame comprises a plurality of pixels; and

each pixel from the plurality of pixels is associated with a pixel location comprising a pixel row and a pixel column;

determining a third pixel location in the first frame for an object located in the space, wherein the third pixel location comprises a first pixel row and a first pixel column of the first frame;

applying a first homography to the third pixel location to determine a third (x,y) coordinate identifying a third x-value and a third y-value in the global plane, wherein the first homography is configured to translate between pixel locations in the first frame and (x,y) coordinates in the global plane;

receiving a second frame of the global plane for at least a second portion of the space from a second sensor;

determining a fourth pixel location in the second frame for the object located in the space; and

applying a second homography to the fourth pixel location to determine a fourth (x,y) coordinate identifying a fourth x-value and a fourth y-value in the global plane, wherein:

the second homography comprises coefficients that translate between the pixel locations in the second frame and (x,y) coordinates in the global plane; and

coefficients of the second homography are different from coefficients of the first homography.

9. The method of claim 8 , further comprising:

generating an average (x,y) coordinate for the object by computing an average of the third (x,y) coordinate of the object and the fourth (x,y) coordinate of the object.

10. The method of claim 8 , further comprising:

generating a median (x,y) coordinate for the object by computing a median of the third (x,y) coordinate of the object and the fourth (x,y) coordinate of the object.

11. The method of claim 8 , further comprising:

storing the third (x,y) coordinate in a tracking list associated with the first sensor in response to determining the third (x,y) coordinate, wherein the tracking list identifies:

an object identifier for the object; and

the third (x,y) coordinate for the object.

12. The method of claim 8 , wherein:

each pixel in the first frame is associated with a pixel value; and

the first homography is further configured to translate between pixel values in the first frame and z-coordinates in the global plane.

13. A computer program comprising executable instructions stored in a non-transitory computer readable medium that when executed by a processor causes the processor to:

receive a first frame from a first sensor of a plurality of sensors configured in a sensor array above at least a portion of a space, wherein:

the first frame is of a global plane for the at least a portion of the space;

the first frame comprises a plurality of pixels; and

each pixel from the plurality of pixels is associated with a pixel location comprising a pixel row and a pixel column;

determine a third pixel location in the first frame for an object located in the space, wherein the third pixel location comprises a first pixel row and a first pixel column of the first frame;

apply a first homography to the third pixel location to determine a third (x,y) coordinate identifying a third x-value and a third y-value in the global plane, wherein the first homography is configured to translate between pixel locations in the first frame and (x,y) coordinates in the global plane;

receive a second frame of the global plane for at least a second portion of the space from a second sensor;

determine a fourth pixel location in the second frame for the object located in the space; and

apply a second homography to the fourth pixel location to determine a fourth (x,y) coordinate identifying a fourth x-value and a fourth y-value in the global plane, wherein:

the second homography comprises coefficients that translate between the pixel locations in the second frame and (x,y) coordinates in the global plane; and

coefficients of the second homography are different from coefficients of the first homography.

14. The computer program of claim 13 , further comprising instructions that when executed by the processor causes the processor to:

generate an average (x,y) coordinate for the object by computing an average of the third (x,y) coordinate of the object and the fourth (x,y) coordinate of the object.

15. The computer program of claim 13 , further comprising instructions that when executed by the processor causes the processor to:

generate a median (x,y) coordinate for the object by computing a median of the third (x,y) coordinate of the object and the fourth (x,y) coordinate of the object.

16. The computer program of claim 13 , further comprising instructions that when executed by the processor causes the processor to:

store the third (x,y) coordinate in a tracking list associated with the first sensor in response to determining the third (x,y) coordinate, wherein the tracking list identifies:

an object identifier for the object; and

the third (x,y) coordinate for the object.

17. The computer program of claim 13 , wherein:

each pixel in the first frame is associated with a pixel value; and

the first homography is further configured to translate between pixel values in the first frame and z-coordinates in the global plane.

18. The method of claim 8 , wherein:

the third (x,y) coordinate is the same as the fourth (x,y) coordinate; and

the third (x,y) coordinate and the fourth (x,y) coordinate are the (x,y) coordinate for the object in the global plane.

19. The computer program of claim 13 , wherein:

the third (x,y) coordinate is the same as the fourth (x,y) coordinate; and

the third (x,y) coordinate and the fourth (x,y) coordinate are the (x,y) coordinate for the object in the global plane.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2019
From: MIRZA, SHAHMEER ALI; KRISHNAMURTHY, SAILESH BHARATHWAAJ; MAUNG, CRYSTAL
To: 7-ELEVEN, INC.
Reel/Frame 050827/0388 →
Continuity (1)
Related Publication 20210125345A1 · Apr 29, 2021