IP Library Granted Patent US 11,074,680
Granted Patent B2
US 11,074,680 · App. 16/510,708 · Granted Jul 27, 2021

Management and display of object-collection data

Inventors: Brent Ronald Frei (Bellevue, WA); Dwight Galen McMaster (Burien, WA); Michael Racine (Bellevue, WA); Jacobus du Preez (Snoqualmie, WA); William David Dimmit (Seattle, WA); Isabelle Butterfield (Bellevue, WA); Clifford Holmgren (Spanaway, WA); Dafydd Daniel Rhys-Jones (Tacoma, WA); Thayne Kollmorgen (Eugene, OR); Vivek Ullal Nayak (Seattle, WA)
Assignee: TERRACLEAR INC.
G06T7/0002A01B43/00A01B59/042A01B69/001B64C39/024B64D47/08G05D1/0038G05D1/101G06K9/0063G06K9/00657G06K9/00664G06K9/4604G06K9/6227G06K9/6228G06K9/6256G06K9/6261G06K9/6262G06N3/0454G06N3/08G06T3/00G06T7/13G06T7/60G06T7/62G06T7/70G06T7/73G08G5/0034G08G5/0069B64C2201/123B64C2201/126B64C2201/127B64C2201/128B64C2201/146G06T2207/10032G06T2207/20081G06T2207/20084G06T2207/20104G06T2207/30188G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,074,680
App. No.
16/510,708
Granted
Jul 27, 2021
Kind
B2
Abstract

An object identification and collection method is disclosed. The method includes receiving a pick-up path that identifies a route in which to guide an object-collection system over a target geographical area to pick up objects, determining a current location of the object-collection system relative to the pick-up path, and guiding the object-collection system along the pick-up path over the target geographical area based on the current location. The method further includes capturing images in a direction of movement of the object-collection system along the pick-up path, identifying a target object in the images; tracking movement of the target object through the images, determining that the target object is within range of an object picker assembly on the object-collection system based on the tracked movement of the target object, and instructing the object picker assembly to pick up the target object.

Claims (99)

1. A method, comprising:

receiving a pick-up path that identifies a route in which to guide an object-collection system over a target geographical area to pick up objects;

determining a current location of the object-collection system relative to the pick-up path;

guiding the object-collection system along the pick-up path over the target geographical area based on the current location;

capturing a plurality of images in a direction of movement of the object-collection system along the pick-up path;

identifying, in the plurality of images, a swath area of an object picker assembly on the object-collection system;

modifying the plurality of images such that there is an even and uniform pixel distribution within the identified swath area;

identifying a target object in the swath area within the plurality of modified images based on a dataset of known object features;

tracking movement of the target object through the plurality of modified images as the object-collection system is being guided along the pick-up path;

identifying a position of the object picker assembly based on an analysis of the plurality of modified images;

determining that the object picker assembly is aligned with the target object and the target object is within range of the object picker assembly based on the tracked movement of the target object and the identified position of the object picker assembly; and

instructing the object picker assembly to pick up the target object.

2. The method of claim 1 , wherein guiding the object-collection system along the pick-up path includes:

selecting a travel waypoint on the pick-up path;

determining a travel direction from the current location to the travel waypoint; and

providing guide information from the current location to the travel waypoint on the pick-up path.

3. The method of claim 1 , further comprising:

displaying, to a user, guide information from the current location to a location of a next target object.

4. The method of claim 1 , further comprising:

determining guide information from the current location to a location of a next target object; and

presenting automatic travel instructions to a motor-control system that autonomously controls the object-collection system to the location of the next target object.

5. The method of claim 1 , further comprising:

obtaining a plurality of image tracking portions of a first image of the plurality of images;

determining a feature characteristic in each of the plurality of image tracking portions;

tracking movement of the feature characteristic across the plurality of images;

determining a speed of movement of each feature characteristic based on the tracked movement; and

determining an overall speed of the object-collection system based on a combination of the speed of movement of each feature characteristic for the plurality of image tacking portions.

6. The method of claim 1 , further comprising:

displaying the plurality of images to a user; and

displaying a representation of the pick-up path to the user.

7. The method of claim 6 , wherein displaying the plurality of images includes:

detecting a boundary of the target object; and

augmenting the plurality of images to include the boundary of the target object.

8. An object-collection system, comprising:

an object picker assembly configured to capture a target object;

a camera configured to capture a plurality of images in a direction of movement of the object-collection system;

a processor; and

a memory that stores computer instructions that, when executed by the processor, cause the processor to:

receive a pick-up path that identifies a route in which to guide the object-collection system over a target geographical area to pick up objects;

determine a current location of the object-collection system relative to the pick-up path;

guide the object-collection system along the pick-up path over the target geographical area based on the current location;

capture the plurality of images via the camera as the object-collection system is traveling over the target geographical area;

identify, in the plurality of images, a swath area of the object picker assembly;

modifying the plurality of images such that there is an even and uniform pixel distribution within the identified swath area;

identify the target object in the plurality of modified images based on a dataset of known object features;

track movement of the target object through the plurality of modified images;

determine that the target object is within range of the object picker assembly on the object-collection system based on the tracked movement of the target object; and

instruct the object picker assembly to pick up the target object.

9. The system of claim 8 , wherein execution of the computer instructions by the processor to guide the object-collection system along the pick-up path causes the processor to:

select a travel waypoint on the pick-up path;

determine a travel direction from the current location to the travel waypoint; and

provide guide information from the current location to the travel waypoint on the pick-up path.

10. The system of claim 8 , wherein execution of the computer instructions by the processor causes the processor to:

display, to a user, guide information from the current location to a location of a next target object.

11. The system of claim 8 , wherein execution of the computer instructions by the processor causes the processor to:

determining guide information from the current location to a location of a next target object; and

presenting automatic travel instructions to a motor-control system that autonomously controls the object-collection system to the location of the next target object.

12. The system of claim 8 , wherein execution of the computer instructions by the processor causes the processor to:

obtain a plurality of image tracking portions of a first image of the plurality of images;

determine a feature characteristic in each of the plurality of image tracking portions;

track movement of the feature characteristic across the plurality of images;

determine a speed of movement of each feature characteristic based on the tracked movement; and

determine an overall speed of the object-collection system based on a combination of the speed of movement of each feature characteristic for the plurality of image tacking portions.

13. The system of claim 8 , wherein execution of the computer instructions by the processor causes the processor to:

display the plurality of images to a user; and

display a representation of the pick-up path to the user.

14. The system of claim 13 , wherein execution of the computer instructions by the processor to display the plurality of images causes the processor to:

detect a boundary of the target object; and

augment the plurality of images to include the boundary of the target object.

15. A non-transitory processor-readable storage medium that stores computer instructions that, when executed by a processor on a computer, cause the processor to perform actions, the actions comprising:

receiving a pick-up path that identifies a route in which to guide an object-collection system over a target geographical area to pick up objects;

determining a current location of the object-collection system relative to the pick-up path;

guiding the object-collection system along the pick-up path over the target geographical area based on the current location;

capturing a plurality of images in a direction of movement of the object-collection system along the pick-up path;

identifying a target object in the plurality of images based on a dataset of known object features;

tracking movement of the target object through the plurality of images;

identifying a position of an object picker assembly on the object-collection system based on an analysis of the plurality of images;

determining that the object picker assembly is aligned with the target object and the target object is within range of the object picker assembly based on the tracked movement of the target object and the identified position of the object picker assembly; and

instructing the object picker assembly to pick up the target object.

16. The non-transitory processor-readable storage medium of claim 15 , wherein guiding the object-collection system along the pick-up path includes:

selecting a travel waypoint on the pick-up path;

determining a travel direction from the current location to the travel waypoint; and

providing guide information from the current location to the travel waypoint on the pick-up path.

17. The non-transitory processor-readable storage medium of claim 15 , further comprising:

displaying, to a user, guide information from the current location to a location of a next target object.

18. The non-transitory processor-readable storage medium of claim 15 , further comprising:

determining guide information from the current location to a location of a next target object; and

presenting automatic travel instructions to a motor-control system that autonomously controls the object-collection system to the location of the next target object.

19. The non-transitory processor-readable storage medium of claim 15 , further comprising:

obtaining a plurality of image tracking portions of a first image of the plurality of images;

determining a feature characteristic in each of the plurality of image tracking portions;

tracking movement of the feature characteristic across the plurality of images;

determining a speed of movement of each feature characteristic based on the tracked movement; and

determining an overall speed of the object-collection system based on a combination of the speed of movement of each feature characteristic for the plurality of image tacking portions.

20. The non-transitory processor-readable storage medium of claim 15 , further comprising:

detecting a boundary of the target object;

augmenting the plurality of images to include the boundary of the target object;

displaying the plurality of augmented images to a user; and

displaying a representation of the pick-up path to the user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 21, 2020
From: FREI, BRENT RONALD; MCMASTER, DWIGHT GALEN; RACINE, MICHAEL; DU PREEZ, JACOBUS; DIMMIT, WILLIAM DAVID; BUTTERFIELD, ISABELLE; HOLMGREN, CLIFFORD; RHYS-JONES, DAFYDD DANIEL; KOLLMORGEN, THAYNE; NAYAK, VIVEK ULLAL
To: TERRACLEAR INC.
Reel/Frame 051890/0755 →
Continuity (2)
Provisional Application 62697057 · Jul 12, 2018
Related Publication 20200019778A1 · Jan 16, 2020
Cited By (1)
US 12,249,097