Granted Patent B2

US 10,687,045 · App. 16/168,545 · Granted Jun 16, 2020

Abstract

Claims (47)

1. A three-dimensional (3D) imaging system configured to analyze idle time duration in a storage area, the 3D imaging system comprising:

a 3D-depth camera configured to non-concurrently capture frames each having a respective 3D image dataset; and

a 3D data analytics application (app) executing on one or more processors, the 3D data analytics app configured to:

maintain a plurality of records including a first record, a second record, and a third record, the first record being associated with a 3D image dataset of a least-recent frame in a series of frames, the third record being associated with a 3D image dataset of a most-recent frame in the series of frames, and the second record being associated with a 3D image dataset of an intermediate frame, the intermediate frame being captured between the least-recent frame in the series of frames and the most-recent frame in the series of frames;

perform a first comparison between the third record and the first record to determine if a difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is less than or greater than a first threshold value;

perform a second comparison between the third record and the second record to determine if a difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the second record is less than or greater than a second threshold value;

determine, based on the first comparison and the second comparison, an activity status type defining an activity occurring within the storage area during a time segment, the time segment occurring between a time of capture of the least-recent frame in the series of frames and a time of capture of the most-recent frame in the series of frames;

increment, based on the activity status type having a non-loading status type, a cumulative idle time duration; and

provide, via a computer network, the cumulative idle time duration to a computing device having a computing device memory.

2. The 3D imaging system of claim 1 , wherein the cumulative idle time duration is a duration of time during which (i) no item is brought into the storage area, (ii) no item is removed from the storage area, and (iii) no item is moved from one location to another location within the storage area.

3. The 3D imaging system of claim 1 , wherein an association between each of the plurality of records and the respective 3D image dataset of a respective frame includes a respective frame buffer storing the respective 3D image dataset of the respective frame, each of the respective frame buffer being stored in a 3D imaging system memory.

4. The 3D imaging system of claim 1 , wherein the plurality of records forms an array of records.

5. The 3D imaging system of claim 1 , wherein the 3D data analytics app is further configured to pre-process each of the plurality of records prior to perform the first comparison and the second comparison, wherein the pre-processing includes cropping at least a portion of the respective 3D image dataset of a respective frame associated with each of the plurality of records.

6. The 3D imaging system of claim 1 , wherein each of the respective 3D image dataset includes a respective point cloud.

7. The 3D imaging system of claim 6 , wherein the 3D data analytics app is further configured to perform the first comparison between the third record and the first record by comparing at least a portion of the respective point cloud of the 3D image dataset associated with at least a portion of the third record with the respective point cloud of the 3D image dataset associated with the first record.

8. The 3D imaging system of claim 1 , wherein the activity status type having the non-loading status type is determined when:

(i) the first comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is less than the first threshold value; or

(ii) the first comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is greater than the first threshold value, and the second comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the second record is less than the second threshold value.

9. The 3D imaging system of claim 1 , wherein:

the activity status type further includes a loading status type;

the non-loading status type includes any one of (i) an inactive status type, (ii) a person-entry status type, and (iii) a person-exit status type;

the activity status type is determined to have the loading status type when the first comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is greater than the first threshold value, and the second comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the second record is greater than the second threshold value;

the activity status type is determined to have the inactive status type when the first comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is less than the first threshold value, and the second comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the second record is less than the second threshold value;

the activity status type is determined to have the person-entry status type when the first comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is less than the first threshold value, and the second comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the second record is greater than the second threshold value; and

the activity status type is determined to have the person-exit status type when the first comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is greater than the first threshold value, and the second comparison indicates that the difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the second record is less than the second threshold value.

10. The 3D imaging system of claim 9 , where the 3D data analytics app is further configured to:

upon determining the activity status type as having any one of (i) the inactive status type, (ii) the person-exit status type, and (iii) the loading status type:

adjust the series of frames by removing the least-recent frame from the series of frames causing the intermediate frame to become the least-recent frame in the series of frames, and adding a new frame, captured subsequent to the most-recent frame in the series of frames, to the series of frames causing the most-recent frame to become the intermediate frame; and

upon determining the activity status type as having the person-entry status type, to:

adjust the series of frames by maintaining the least-recent frame in the series of frames, and adding the new frame, captured subsequent to the most-recent frame in the series of frames, to the series of frames causing the most-recent frame to become the intermediate frame.

11. The 3D imaging system of claim 10 , wherein the series of frames is limited to three frames.

12. The 3D imaging system of claim 1 , wherein the 3D data analytics app is further configured to reset, based on the activity status type having a loading status type, the cumulative idle time duration.

13. The 3D imaging system of claim 1 , wherein the computing device is at least one of a client device and a server configured to manage a plurality of client devices.

14. A three-dimensional (3D) imaging method for analyzing idle time duration in a storage area, the 3D imaging method comprising:

non-concurrently capturing, via a 3D-depth camera, frames each having a respective 3D image dataset;

maintaining, via 3D data analytics application (app) executing on one or more processors, a plurality of records including a first record, a second record, and a third record, the first record being associated with a 3D image dataset of a least-recent frame in a series of frames, the third record being associated with a 3D image dataset of a most-recent frame in the series of frames, and the second record being associated with a 3D image dataset of an intermediate frame, the intermediate frame being captured between the least-recent frame in the series of frames and the most-recent frame in the series of frames;

performing a first comparison between the third record and the first record to determine if a difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the first record is less than or greater than a first threshold value;

performing a second comparison between the third record and the second record to determine if a difference between the 3D image dataset associated with the third record and the 3D image dataset associated with the second record is less than or greater than a second threshold value;

determining, based on the first comparison and the second comparison, an activity status type defining an activity occurring within the storage area during a time segment, the time segment occurring between a time of capture of the least-recent frame in the series of frames and a time of capture of the most-recent frame in the series of frames;

incrementing, based on the activity status type having a non-loading status type, a cumulative idle time duration; and

providing, via a computer network, the cumulative idle time duration to a computing device having a computing device memory.

15. The 3D imaging method of claim 14 , wherein the cumulative idle time duration is a duration of time during which (i) no item is brought into the storage area, (ii) no item is removed from the storage area, and (iii) no item is moved from one location to another location within the storage area.

16. The 3D imaging method of claim 14 , wherein an association between each of the plurality of records and the respective 3D image dataset of a respective frame includes a respective frame buffer storing the respective 3D image dataset of the respective frame, each of the respective frame buffer being stored in a 3D imaging system memory.

17. The 3D imaging method of claim 14 , wherein the plurality of records forms an array of records.

18. The 3D imaging method of claim 14 , wherein the 3D data analytics app is further configured to pre-process each of the plurality of records prior to perform the first comparison and the second comparison, wherein the pre-processing includes cropping at least a portion of the respective 3D image dataset of a respective frame associated with each of the plurality of records.

19. The 3D imaging method of claim 14 , wherein each of the respective 3D image dataset includes a respective point cloud.

20. The 3D imaging method of claim 19 , wherein the 3D data analytics app is further configured to perform the first comparison between the third record and the first record by comparing at least a portion of the respective point cloud of the 3D image dataset associated with at least a portion of the third record with the respective point cloud of the 3D image dataset associated with the first record.

Assignments (6)