Granted Patent B2

US 12,524,890 · App. 18/476,425 · Granted Jan 13, 2026

Inventors: Xinan Wang (McKinney, TX); Kyle J Dalal (Coppell, TX); Fahad Mirza (Arlington, TX); Jon Andrew Crain (Mechanicsburg, PA)

Assignee: 7-Eleven, Inc.

G06T7/248G06T7/55G06T2207/10028G06T2207/20076G06T2207/20224

Abstract

Claims (127)

1 . An item tracking system, comprising:

a three-dimensional (3D) sensor positioned above a platform, wherein the 3D sensor is configured to capture overhead depth images of the platform, wherein:

each overhead depth image depicts upward-facing surfaces of objects placed on the platform;

each overhead depth image comprises a plurality of pixels distributed across the overhead depth image; and

each of the plurality of pixels is associated with a depth value indicating a distance between the 3D sensor and at least a portion of an upward-facing surface depicted by the pixel; and

one or more processors configured to:

capture a first overhead depth image of the platform using the 3D sensor;

determine a reference average depth value across the plurality of pixels in the first overhead depth image;

capture a plurality of second overhead depth images of the platform using the 3D sensor;

for each second overhead depth image:

determine a second average depth value across the plurality of pixels of the second overhead depth image; and

determine a value of a depth difference parameter by subtracting the second average depth value from the reference average depth value;

determine that the depth difference parameter has stayed constant at a value higher than zero for a duration of a first time interval; and

in response to determining that depth difference parameter has stayed constant at the value higher than zero, determine that an item has been placed on the platform.

2 . The item tracking system of claim 1 , wherein the one or more processors are further configured to determine that the depth difference parameter has stayed constant by:

capturing, at a first time instant within the first time interval, a third overhead depth image of the platform using the 3D sensor;

determining a third average depth value across the plurality of pixels of the third overhead depth image;

determining a first value of the depth difference parameter by subtracting the third average depth value from the reference average depth value;

capturing, at a second time instant within the first time interval after the first time instant, a fourth overhead depth image of the platform using the 3D sensor;

determine a fourth average depth value across the plurality of pixels of the fourth overhead depth image;

determine a second value of the depth difference parameter by subtracting the fourth average depth value from the reference average depth value;

determining a delta difference by:

subtracting the second value from the first value to generate a subtracted value; and

dividing the subtracted value by a time difference between the first time instant and the second time instant;

determining that the delta difference is zero or near zero; and

in response to determining that the delta difference is zero or near zero, determining that the depth difference parameter has stayed constant between the first time instant and the second time instant.

3 . The item tracking system of claim 2 , wherein the one or more processors are further configured to:

determine that the first value and the second value of the depth difference parameter equal the value higher than zero;

in response to determining that the delta difference is zero or near zero and that the first value and the second value of the depth difference parameter equal the value higher than zero, determine that the depth difference parameter has stayed constant at the value higher than zero between the first time instant and the second time instant; and

in response to determining that the depth difference parameter has stayed constant at the value higher than zero between the first time instant and the second time instant, determine that the item has been placed on the platform.

4 . The item tracking system of claim 1 , wherein the one or more processors are further configured to:

detect that the value of the depth difference parameter has changed from zero to above a threshold value, wherein the change in the depth difference parameter from zero to above the threshold value indicates that a user's hand holding the item has moved from a first position that is outside a view of the 3D sensor to a second position on the platform that is within the view of the 3D sensor.

5 . The item tracking system of claim 4 , wherein the one or more processors are further configured to:

after detecting that the value of the depth difference parameter has changed from zero to above the threshold value:

determine that the depth difference parameter has decreased for a duration of a second time interval, wherein the second time interval is before the first time interval; and

in response to determining that the depth difference parameter has decreased for the duration of the second time interval, determine that the user's hand has placed the item on the platform and is moving away from the platform.

6 . The item tracking system of claim 5 , wherein the one or more processors are further configured to determine that the depth difference parameter has decreased for the duration of the second time interval by:

capturing, at a first time instant within the second time interval, a third overhead depth image of the platform using the 3D sensor;

determining a third average depth value across the plurality of pixels of the third overhead depth image;

determining a first value of the depth difference parameter by subtracting the third average depth value from the reference average depth value;

capturing, at a second time instant within the second time interval after the first time instant, a fourth overhead depth image of the platform using the 3D sensor;

determine a fourth average depth value across the plurality of pixels of the fourth overhead depth image;

determine a second value of the depth difference parameter by subtracting the fourth average depth value from the reference average depth value;

determining a delta difference by:

subtracting the second value from the first value to generate a subtracted value; and

dividing the subtracted value by a time difference between the first time instant and the second time instant;

determining that the delta difference is less than zero; and

in response to determining that the delta difference is less than zero, determining that the depth difference parameter has decreased between the first time instant and the second time instant.

7 . The item tracking system of claim 5 , wherein the one or more processors are further configured to determine that the item is ready for identification in response to a combination of:

detecting that the value of the depth difference parameter has changed from zero to above a threshold value;

determining that the depth difference parameter has decreased for the duration of the second time interval; and

determining that depth difference parameter has stayed constant at the value higher than zero.

8 . The item tracking system of claim 7 , wherein the one or more processors are further configured to:

in response to determining that the item is ready for identification, initiate a procedure to identify the item.

9 . The item tracking system of claim 1 , wherein the one or more processors are further configured to:

detect motion near the platform using a proximity sensor; and

capture the plurality of second overhead depth images in response to detecting the motion.

10 . The item tracking system of claim 1 , wherein the first overhead depth image is of the platform without any obstructions between the 3D sensor and a surface of the platform.

11 . A method for identifying an item, comprising:

capturing a first overhead depth image of a platform using a three-dimensional (3D) sensor positioned above the platform, wherein:

the 3D sensor is configured to capture overhead depth images of the platform;

each overhead depth image depicts upward-facing surfaces of objects placed on the platform;

each overhead depth image comprises a plurality of pixels distributed across the overhead depth image; and

each of the plurality of pixels is associated with a depth value indicating a distance between the 3D sensor and at least a portion of an upward-facing surface depicted by the pixel;

determining a reference average depth value across the plurality of pixels in the first overhead depth image;

capturing a plurality of second overhead depth images of the platform using the 3D sensor;

for each second overhead depth image:

determining a second average depth value across the plurality of pixels of the second overhead depth image; and

determining a value of a depth difference parameter by subtracting the second average depth value from the reference average depth value;

determining that the depth difference parameter has stayed constant at a value higher than zero for a duration of a first time interval; and

in response to determining that depth difference parameter has stayed constant at the value higher than zero, determining that an item has been placed on the platform.

12 . The method of claim 11 , wherein determining that the depth difference parameter has stayed constant comprises:

capturing, at a first time instant within the first time interval, a third overhead depth image of the platform using the 3D sensor;

determining a third average depth value across the plurality of pixels of the third overhead depth image;

determining a first value of the depth difference parameter by subtracting the third average depth value from the reference average depth value;

capturing, at a second time instant within the first time interval after the first time instant, a fourth overhead depth image of the platform using the 3D sensor;

determine a fourth average depth value across the plurality of pixels of the fourth overhead depth image;

determine a second value of the depth difference parameter by subtracting the fourth average depth value from the reference average depth value;

determining a delta difference by:

subtracting the second value from the first value to generate a subtracted value; and

dividing the subtracted value by a time difference between the first time instant and the second time instant;

determining that the delta difference is zero or near zero; and

in response to determining that the delta difference is zero or near zero, determining that the depth difference parameter has stayed constant between the first time instant and the second time instant.

13 . The method of claim 12 , further comprising:

determining that the first value and the second value of the depth difference parameter equal the value higher than zero;

in response to determining that the delta difference is zero or near zero and that the first value and the second value of the depth difference parameter equal the value higher than zero, determining that the depth difference parameter has stayed constant at the value higher than zero between the first time instant and the second time instant; and

in response to determining that the depth difference parameter has stayed constant at the value higher than zero between the first time instant and the second time instant, determining that the item has been placed on the platform.

14 . The method of claim 11 , further comprising:

detecting that the value of the depth difference parameter has changed from zero to above a threshold value, wherein the change in the depth difference parameter from zero to above the threshold value indicates that a user's hand holding the item has moved from a first position that is outside a view of the 3D sensor to a second position on the platform that is within the view of the 3D sensor.

15 . The method of claim 14 , further comprising:

after detecting that the value of the depth difference parameter has changed from zero to above the threshold value:

determining that the depth difference parameter has decreased for a duration of a second time interval, wherein the second time interval is before the first time interval; and

in response to determining that the depth difference parameter has decreased for the duration of the second time interval, determining that the user's hand has placed the item on the platform and is moving away from the platform.

16 . The method of claim 15 , wherein determining that the depth difference parameter has decreased for the duration of the second time interval by:

capturing, at a first time instant within the second time interval, a third overhead depth image of the platform using the 3D sensor;

determining a third average depth value across the plurality of pixels of the third overhead depth image;

determining a first value of the depth difference parameter by subtracting the third average depth value from the reference average depth value;

capturing, at a second time instant within the second time interval after the first time instant, a fourth overhead depth image of the platform using the 3D sensor;

determine a fourth average depth value across the plurality of pixels of the fourth overhead depth image;

determine a second value of the depth difference parameter by subtracting the fourth average depth value from the reference average depth value;

determining a delta difference by:

subtracting the second value from the first value to generate a subtracted value; and

dividing the subtracted value by a time difference between the first time instant and the second time instant;

determining that the delta difference is less than zero; and

in response to determining that the delta difference is less than zero, determining that the depth difference parameter has decreased between the first time instant and the second time instant.

17 . The method of claim 15 , wherein determining that the item is ready for identification is in response to a combination of:

detecting that the value of the depth difference parameter has changed from zero to above a threshold value;

determining that the depth difference parameter has decreased for the duration of the second time interval; and

determining that depth difference parameter has stayed constant at the value higher than zero.

18 . The method of claim 17 , further comprising:

in response to determining that the item is ready for identification, initiating a procedure to identify the item.

19 . The method of claim 11 , further comprising:

detecting motion near the platform using a proximity sensor; and

capturing the plurality of second overhead depth images in response to detecting the motion.

20 . A computer-readable medium storing instructions that when executed by one or more processors cause the one or more processors to:

capture a first overhead depth image of a platform using a three-dimensional (3D) sensor positioned above the platform, wherein:

the 3D sensor is configured to capture overhead depth images of the platform;

each overhead depth image depicts upward-facing surfaces of objects placed on the platform;

each overhead depth image comprises a plurality of pixels distributed across the overhead depth image; and

each of the plurality of pixels is associated with a depth value indicating a distance between the 3D sensor and at least a portion of an upward-facing surface depicted by the pixel;

determine a reference average depth value across the plurality of pixels in the first overhead depth image;

capture a plurality of second overhead depth images of the platform using the 3D sensor;

for each second overhead depth image:

determine a second average depth value across the plurality of pixels of the second overhead depth image; and

determine a value of a depth difference parameter by subtracting the second average depth value from the reference average depth value;

determine that the depth difference parameter has stayed constant at a value higher than zero for a duration of a first time interval; and

in response to determining that depth difference parameter has stayed constant at the value higher than zero, determine that an item has been placed on the platform.

Assignments (1)

Continuity (4)

References Cited (59)