IP Library › Granted Patent US 12,242,916
Granted Patent B2
US 12,242,916 · App. 17/988,706 · Granted Mar 4, 2025

Systems and methods for tracking items

Inventors: Stephen Plummer (Fresno, CA); Alexander Plummer (Fresno, CA); Brandon Lundberg (Fresno, CA)
G06K7/10722G06K7/1413G06T5/80G06T7/248G06T7/536G06T7/73G06T7/74G06T3/02G06T2207/30204G06T2207/30244H04N23/90
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,242,916
App. No.
17/988,706
Granted
Mar 4, 2025
Kind
B2
Abstract

The present invention provides systems and methods for tracking items (e.g., commodities, goods, containers, boxes, packages, etc.) through transportations to multiple locations to allow the position(s) and movement(s) of such items to be accurately tracked and documented, and to allow such items to be quickly identified and located based on tracking records kept within the tracking system. The system may utilize image sensors, image recognition and processes software, position translation software, and a virtual model of the pre-defined space in order to track objects within the defined space and maintain a record of the movement(s) and position(s) of the object within the pre-defined space.

Claims (43)

1. A system for generating and verifying an electronic manifest when preparing or receiving a shipment of objects within a predefined space, comprising:

a. a plurality of unique object markers positioned on each of said objects, the objects being loaded on a transportation vehicle;

b. a predefined space having a plurality of electronic image acquisition devices having a machine vision system, the machine vision system comprising an image sensor and image capture electronics, for acquiring images of said objects;

c. an image processing system for analyzing pixels in the acquired image to determine an identity of each object marker in said acquired image, retrieve object data from a database of each object marker in said acquired image, and determine the distance of said object marker relative to the image acquisition devices; and

d. a user interface operable to display an itemized inventory list of objects in an electronic manifest, a video feed of said plurality of electronic image acquisition devices and superimpose an identification marker over said object markers in said video feed,

wherein said image processing system is operable to continuously scan said predefined space for object markers and determine the position, orientation, and distance of said object based on said unique object marker within said predefined space relative to said image acquisition device, validate said object on said electronic manifest by comparing said object data with manifest data to confirm the appropriate object is loaded on said transportation vehicle, and superimpose said identification marker in said video feed over said object marker using said position, orientation, and distance of said marker.

2. The system of claim 1 , wherein said image processing system is operable to analyze optical features within each of said object markers to read coding provided in said optical features and retrieve identification data from said coding.

3. The system of claim 2 , wherein said image processing system determines if said identification data corresponds to one of said plurality of objects loaded in said inventory list and changes the status of the object to validated in said user interface.

4. The system of claim 1 , wherein said predefined space includes a first side and a second side operable to provide an elongated passageway operable to scan a driver and passenger side of said transportation vehicle.

5. The system of claim 1 , wherein said user interface is loaded with an inventory list containing a plurality of objects.

6. The system of claim 1 , wherein said distance, position, and orientation is determined by said image recognition and processing software using pose estimation, Euclidean, affine, projective, and signed distance transforms.

7. A method for tracking a plurality of objects in a predefined space and validating an itemized inventory list on a user interface of a server computer for verification of an inbound or outbound shipment, comprising:

a. placing a unique object marker on each of said plurality of objects loaded onto a transport vehicle, each of said unique object markers having a machine-readable code corresponding to a record in a database on said server computer that includes identification data and data regarding the object on which said unique object marker is positioned;

b. placing a plurality of image acquisition devices at predetermined locations in said predefined space, each of said image acquisition devices in communication with image recognition and processing software on a machine-readable memory of said server computer, and displaying a video feed of an acquired image on said user interface;

c. loading said itemized inventory list onto said user interface, said itemized inventory list including a validation status, an identification number, and the goods for each item in said inventory list;

d. analyze said acquired images with said image processing software to identify if a unique object marker is present in said image acquisition devices field of view, calculating the distance, position and orientation of the object to determine if the object is within said predefined space;

e. retrieve object identification data for said unique object markers within said predefined space and determine if said identification data of said objects matches an item in said inventory list; and

f. changing said validation status of said item to validated based on said determine if said identification data of said objects match an item in said inventory list,

wherein said video feed of said predefined space is a stream of captured images continuously analyzed by said image recognition and processing software for unique objects markers and object data corresponding to said unique object marker is retrieved from said database and compared to said inventory list until each of said item in said inventory list is validated on said user interface and said outbound or inbound shipment is validated.

8. The method of claim 7 , further comprising:

a. generating a unique identification marker with said processing software, said unique identification marker having said validation status and identification number of a detected unique object marker in said predefined space;

b. superimposing said identification marker over said unique object marker in said video feed based on said distance, position and orientation calculation; and

c. tracking said unique object marker in said video feed and modifying said superimposing said identification marker based on new distance, position, and orientation calculation.

9. The method of claim 8 , wherein said server computer determines if said identification data of said objects matches an item in said inventory list, and if said identification data fails to match an item in said inventory list the object is flagged for review with an invalid identification marker superimposed over the unique object marker and is added to the inventory list as an object requiring review from an operator.

10. The method of claim 8 , further comprising an image acquisition device positioned outside of said predefined space and is operable to identify a machine-readable optical marker positioned on said transport vehicle.

11. The method of claim 10 , wherein said transport vehicle machine-readable optical marker corresponds to an inventory list in said computer database and is operable initiate a loading function that uploads said inventory list to the said user interface.

12. The method of claim 7 , wherein each of said plurality of image acquisition device further comprising a machine vision system that includes an image sensor and image capture electronics, for acquiring images of said predefined space for processing in image recognition and processing software.

13. A method for tracking a plurality of objects in an inbound or outbound shipment, comprising:

a. placing a unique object marker on each of said plurality of objects loaded onto a transport vehicle, each of said unique object markers having a machine-readable code corresponding to a record in a database on a server computer that includes identification data and data regarding the object on which said unique object marker is positioned;

b. placing a plurality of image acquisition devices at predetermined locations in a predefined space, each of said image acquisition devices in communication with image recognition and processing software on a machine-readable memory of said server computer;

c. loading an itemized inventory list onto a user interface, said itemized inventory list including a validation status and identification information regarding each of said plurality of objects;

d. analyze said acquired images with said image processing software to identify if a unique object marker is present in said image acquisition devices field of view, calculating the distance, position and orientation of the object to determine if the object is on said itemized inventory list; and

e. changing said validation status of said item to validated based on said unique object marker determine if said identification data of said objects match an item in said inventory list, wherein said distance, position, and orientation is determined by said image recognition and processing software using pose estimation.

14. The method of claim 13 , wherein said video feed of said predefined space is a stream of captured images continuously analyzed by said image recognition and processing software for unique objects markers and object data corresponding to said unique object marker is retrieved from said database and compared to said inventory list until each of said item in said inventory list is validated on said user interface and said outbound or inbound shipment is validated.

15. The method of claim 13 , further comprising:

a. generating a unique identification marker with said processing software, said unique identification marker having said validation status and identification number of a detected unique object marker in said predefined space;

b. superimposing said identification marker over said unique object marker in said video feed based on said distance, position and orientation calculation; and

c. tracking said unique object marker in said video feed and modifying said superimposing said identification marker based on new distance, position, and orientation calculation.

16. The method of claim 15 , wherein said server computer determines if said identification data of said objects matches an item in said inventory list, and if said identification data fails to match an item in said inventory list the object is flagged for review with an invalid identification marker superimposed over the unique object marker and is added to the inventory list as an object requiring review from an operator.

17. The method of claim 15 , further comprising an image acquisition device positioned outside of said predefined space and is operable to identify a machine-readable optical marker positioned on said transport vehicle.

18. The method of claim 17 , wherein said transport vehicle machine-readable optical marker corresponds to an inventory list in said computer database and is operable initiate a loading function that uploads said inventory list to the said user interface.

19. The method of claim 13 , wherein each of said plurality of image acquisition device further comprising a machine vision system that includes an image sensor and image capture electronics, for acquiring images of said predefined space for processing in image recognition and processing software.

20. The system of claim 13 , wherein said distance, position, and orientation is further determined by said image recognition and processing software using Euclidean, affine, projective, and signed distance transforms.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2022
From: PLUMMER, STEPHEN; PLUMMER, ALEXANDER; LUNDBERG, BRANDON
To: DIVINE LOGIC, INC.
Reel/Frame 062003/0342 →
Continuity (9)
Continuation In Part 17740317 · May 9, 2022
Continuation 17537499 · Nov 30, 2021
Continuation 17408752 · Aug 23, 2021
Continuation 16892290 · Jun 4, 2020
Continuation 16194764 · Nov 19, 2018
Provisional Application 63310762 · Feb 16, 2022
Provisional Application 62675772 · May 24, 2018
Provisional Application 62588261 · Nov 17, 2017
Related Publication 20230092401A1 · Mar 23, 2023
References Cited (126)
US 4684247A · Hammill, III · 1987 [cited by applicant]
US 5051906A · Evans, Jr. et al. · 1991 [cited by applicant]
US 5059789A · Salcudean · 1991 [cited by applicant]
US 5193124A · Subbarao · 1993 [cited by applicant]
US 5367458A · Roberts et al. · 1994 [cited by applicant]
US 5525883A · Avitzour · 1996 [cited by applicant]
US 5602760A · Chacon et al. · 1997 [cited by applicant]
US 5604715A · Aman et al. · 1997 [cited by applicant]
US 5617335A · Hasima et al. · 1997 [cited by applicant]
US 5691527A · Hara et al. · 1997 [cited by applicant]
US 5726435A · Hara et al. · 1998 [cited by applicant]
US 5745036A · Clare · 1998 [cited by applicant]
US 5793934A · Bauer · 1998 [cited by applicant]
US 5828770A · Leis et al. · 1998 [cited by applicant]
US 5832139A · Batterman et al. · 1998 [cited by applicant]
US 5893043A · Moehlenbrink et al. · 1999 [cited by applicant]
US 6064749A · Hirota et al. · 2000 [cited by applicant]
US 6078849A · Brady et al. · 2000 [cited by applicant]
US 6173066B1 · Peurach et al. · 2001 [cited by applicant]
US 6175644B1 · Scola et al. · 2001 [cited by applicant]
US 6266008B1 · Huston et al. · 2001 [cited by applicant]
US 6305891B1 · Burlingame · 2001 [cited by applicant]
US 6434254B1 · Wixson · 2002 [cited by applicant]
US 6542824B1 · Berstis · 2003 [cited by applicant]
US 6556722B1 · Russell et al. · 2003 [cited by applicant]
US 6661449B1 · Sogawa · 2003 [cited by applicant]
US 6697761B2 · Akatsuka et al. · 2004 [cited by applicant]
US 6728582B1 · Wallack · 2004 [cited by applicant]
US 6732045B1 · Irmer · 2004 [cited by applicant]
US 6750769B1 · Smith · 2004 [cited by applicant]
US 6859729B2 · Breakfield et al. · 2005 [cited by applicant]
US 6919880B2 · Morrison et al. · 2005 [cited by applicant]
US 6934540B2 · Twitchell, Jr. · 2005 [cited by applicant]
US 6952488B2 · Kelly et al. · 2005 [cited by applicant]
US 7013026B2 · Takehara et al. · 2006 [cited by applicant]
US 7032823B2 · Nojiri · 2006 [cited by applicant]
US 7164359B2 · Waterhouse et al. · 2007 [cited by applicant]
US 7194330B2 · Carson · 2007 [cited by applicant]
US 7298314B2 · Schantz et al. · 2007 [cited by applicant]
US 7339469B2 · Braun · 2008 [cited by applicant]
US 7370803B2 · Mueller et al. · 2008 [cited by applicant]
US 7372451B2 · Dempski · 2008 [cited by applicant]
US 7468821B2 · Hashiguchi et al. · 2008 [cited by applicant]
US 7512262B2 · Criminisi et al. · 2009 [cited by applicant]
US 7616156B2 · Smith et al. · 2009 [cited by applicant]
US 7646336B2 · Tan et al. · 2010 [cited by applicant]
US 7663671B2 · Gallagher et al. · 2010 [cited by applicant]
US 7667646B2 · Kalliola et al. · 2010 [cited by applicant]
US 7681796B2 · Cato et al. · 2010 [cited by applicant]
US 7721967B2 · Mueller et al. · 2010 [cited by applicant]
US 7845560B2 · Emanuel et al. · 2010 [cited by applicant]
US 7864159B2 · Sweetser et al. · 2011 [cited by applicant]
US 7957833B2 · Beucher et al. · 2011 [cited by applicant]
US 8144920B2 · Kansal et al. · 2012 [cited by applicant]
US 8196835B2 · Emanuel · 2012 [cited by examiner]
US 8279069B2 · Sawyer · 2012 [cited by applicant]
US 8280173B2 · Kato et al. · 2012 [cited by applicant]
US 8284045B2 · Twitchell, Jr. · 2012 [cited by applicant]
US 8358099B2 · Yamaguchi · 2013 [cited by applicant]
US 8381982B2 · Kunzig et al. · 2013 [cited by applicant]
US 8526672B2 · Momoi · 2013 [cited by applicant]
US 8561897B2 · Kunzig et al. · 2013 [cited by applicant]
US 8565913B2 · Emanuel et al. · 2013 [cited by applicant]
US 8582867B2 · Litvak · 2013 [cited by applicant]
US 8805002B2 · BenHimane et al. · 2014 [cited by applicant]
US 8831352B2 · Gao et al. · 2014 [cited by applicant]
US 8836575B2 · Nishiyama · 2014 [cited by applicant]
US 8903430B2 · Sands et al. · 2014 [cited by applicant]
US 8913792B2 · BenHimane et al. · 2014 [cited by applicant]
US 9341720B2 · Garin et al. · 2016 [cited by applicant]
US 9342724B2 · McCloskey et al. · 2016 [cited by applicant]
US 9357181B2 · Fujimatsu et al. · 2016 [cited by applicant]
US 9607388B2 · Lin et al. · 2017 [cited by applicant]
US 9626709B2 · Koch et al. · 2017 [cited by applicant]
US 9779286B2 · Morishita · 2017 [cited by applicant]
US 9875579B2 · Menozzi et al. · 2018 [cited by applicant]
US 9946963B2 · Samara et al. · 2018 [cited by applicant]
US 9990733B2 · Tsuji · 2018 [cited by applicant]
US 10019693B2 · Wolf et al. · 2018 [cited by applicant]
US 10019878B2 · Mains, Jr. · 2018 [cited by applicant]
US 10181197B2 · Hirasawa et al. · 2019 [cited by applicant]
US 10552964B2 · Tsuji · 2020 [cited by applicant]
US 10685197B2 · Plummer et al. · 2020 [cited by applicant]
US 11292463B2 · Hardy et al. · 2022 [cited by applicant]
US 20030144813A1 · Takemoto et al. · 2003 [cited by applicant]
US 20040016077A1 · Song et al. · 2004 [cited by applicant]
US 20040183751A1 · Dempski · 2004 [cited by applicant]
US 20060132311A1 · Kruest et al. · 2006 [cited by applicant]
US 20060184013A1 · Emanuel et al. · 2006 [cited by applicant]
US 20080226130A1 · Kansal et al. · 2008 [cited by applicant]
US 20100091096A1 · Oikawa et al. · 2010 [cited by applicant]
US 20110121068A1 · Emanuel et al. · 2011 [cited by applicant]
US 20120251011A1 · Gao et al. · 2012 [cited by applicant]
US 20130043826A1 · Workman et al. · 2013 [cited by applicant]
US 20130050502A1 · Saito et al. · 2013 [cited by applicant]
US 20130200811A1 · Steininger et al. · 2013 [cited by applicant]
US 20140247278A1 · Samara et al. · 2014 [cited by applicant]
US 20140285133A1 · Toledo et al. · 2014 [cited by applicant]
US 20150302500A1 · Koch et al. · 2015 [cited by applicant]
US 20150356345A1 · Velozo et al. · 2015 [cited by applicant]
US 20160110976A1 · Mains, Jr. · 2016 [cited by applicant]
US 20160203357A1 · Morishita · 2016 [cited by applicant]
US 20160247318A2 · Menozzi et al. · 2016 [cited by applicant]
US 20160335780A1 · Tsuji · 2016 [cited by applicant]
US 20170014203A1 · De Mathelin et al. · 2017 [cited by applicant]
US 20170091704A1 · Wolf et al. · 2017 [cited by applicant]
US 20170109875A1 · Shevchenko et al. · 2017 [cited by applicant]
US 20200110526A1 · Ano et al. · 2020 [cited by applicant]
EP 0367526A2 · 1990 [cited by applicant]
EP 1437636A1 · 2004 [cited by applicant]
EP 2695134A2 · 2014 [cited by applicant]
IN 1987CHE2008A · 2010 [cited by applicant]
WO 2019100011A1 · 2019 [cited by applicant]
Indian Patent Application 1987/CHE/2008 English language document. [cited by applicant]
A new method of camera pose estimation using 2D-3D corner correspondence, by Shi et al., publication; Apr. 15, 2004. Available online; “https://www.sciencedirect.com/science/article/abs/pii/S0167865504000790?via%3Dihub”. [cited by applicant]
Pose Determination of a Three-Dimensional Object Using Triangle Pairs, by Linnainmaa et al., from IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10. No. 5. Sep. 1988. Available online; “https://iee… [cited by applicant]
International search report for PCT/US2005/043755, mailing date Nov. 6, 2006. Available online; “https://patentscope.wipo.int/search/en/detail.jsf?docPN=EP1828862”. [cited by applicant]
NASA Technical Memorandum 107144, Feb. 1996, Color Image Processing and Object Tracking System, by Klimek et al. Available online; “https://ntrs.nasa.gov/citations/19960016954”. [cited by applicant]
Off-Grid Solar Power in Rural India, Author Pratima Bisen Kanudia, ETSAP meeting, Lisbon, Dec. 10, 2012, available online; “chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://iea-etsap.org/workshop/lisbon_dec20… [cited by applicant]
International Application No. PCT/US18/61839—International Search Report and Written Opinion. World Intellectual Property Organization International Search Authority, Mar. 7, 2019. [cited by applicant]
Examination report No. 1 for Australian patent application No. 2018368776. [cited by applicant]
Original document of EP2695134A2 as WO2012138585A2, available online; “https://worldwide.espacenet.com/patent/search/family/046927345/publication/EP2695134A2?q=pn%3DEP2695134A2”. [cited by applicant]
Original document of EP0367526A2, available online; “https://worldwide.espacenet.com/patent/search/family/023009429/publication/EP0367526A2?q=pn%3DEP0367526A2”. [cited by applicant]
Original document of EP1437636A1, available online; “https://worldwide.espacenet.com/patent/search/family/032501497/publication/EP1437636A1?q=pn%3DEP1437636A1”. [cited by applicant]
Original document of WO2019100011A1, available online; “https://worldwide.espacenet.com/patent/search/family/066532409/publication/WO2019100011A1?q=pn%3DWO2019100011A1”. [cited by applicant]
U.S. Appl. No. 60/635,813 original document. [cited by applicant]
Cited By (2)
US 12,606,204 US 12,699,960