IP Library › Granted Patent US 11,789,459
Granted Patent B2
US 11,789,459 · App. 17/748,454 · Granted Oct 17, 2023

Vehicle controllers for agricultural and industrial applications

Inventors: David E. Bertucci (Sunnyvale, CA); Igino C. Cafiero (Palo Alto, CA); Aubrey C. Donnellan (San Mateo, CA); Thuy T. Nguyen (Albany, CA); Vaibhav K. Viswanathan (Omaha, NE)
Assignee: Bear Flag Robotics, Inc.
G05D1/0274A01B79/005G05D1/0088G05D1/0212G05D1/0276G06F16/29G06F18/2113G06T7/70G06V10/757G06V20/56G06V20/58G05D2201/0201G06T2207/10028G06T2207/30188G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,789,459
App. No.
17/748,454
Granted
Oct 17, 2023
Kind
B2
Abstract

Systems and methods for vehicle controllers for agricultural and industrial applications are described. For example, a method includes accessing a map data structure storing a map representing locations of physical objects in a geographic area; accessing current point cloud data captured using a distance sensor connected to a vehicle; detecting a crop row based on the current point cloud data; matching the detected crop row with a crop row represented in the map; determining an estimate of a current location of the vehicle based on a current position in relation to the detected crop row; and controlling one or more actuators to cause the vehicle to move from the current location of the vehicle to a target location.

Claims (75)

1. A system, comprising:

a sensor connected to a vehicle;

actuators configured to control motion of the vehicle; and

a processing apparatus configured to:

access current point cloud data captured using the sensor;

detect a crop row based on the current point cloud data;

detect an end of the crop row based on point cloud data captured using the sensor;

plan a turn at the end of the crop row;

control one or more of the actuators to cause the vehicle to execute the turn;

after executing the turn, detect a last seen plant at the end of the crop row;

based on a position of the last seen plant, identify a trajectory to enter a next agricultural lane bounded by the crop row; and

control one or more of the actuators to cause the vehicle to enter the next agricultural lane.

2. The system of claim 1 , in which the processing apparatus is configured to:

detect a furrow in the ground based on the current point cloud data;

match the detected furrow with a furrow represented in a map; and

determine an estimate of a current location of the vehicle based on a current position in relation to the detected furrow.

3. The system of claim 1 , in which the processing apparatus is configured to:

filter the current point cloud data to consider only points in a zone of interest when comparing the current point cloud data to point cloud data of a map to match the detected crop row with a crop row represented in the map, where in the zone of interest is limited to a range of heights and a maximum distance from the vehicle.

4. The system of claim 1 , comprising:

one or more image sensors connected to the vehicle; and

actuators configured to control operation of an implement, wherein the implement is connected to the vehicle and configured to selectively perform an operation in a vicinity of the vehicle, and in which the processing apparatus is configured to:

receive image data, captured using the one or more image sensors, depicting one or more plants in the vicinity of the vehicle;

detect the one or more plants based on the image data;

responsive to detecting the one or more plants, adjust implement control data; and

control one or more of the actuators, based on the adjusted implement control data, to cause the implement to deposit an input on the one or more plants.

5. The system of claim 4 , in which the one or more image sensors comprise a normalized difference vegetation index camera connected to the vehicle, in which the processing apparatus is configured to:

receive normalized difference vegetation index data, captured using the normalized difference vegetation index camera, for the one or more plants, wherein adjusting the implement control data includes adjusting an amount of input to be deposited on the one or more plants based on the normalized difference vegetation index data.

6. The system of claim 4 , in which the vehicle is a tractor and the implement is a sprayer.

7. The system of claim 4 , in which the implement control data is adjusted based on a count of the one or more plants detected in the vicinity of the vehicle.

8. The system of claim 1 , in which the vehicle is a tractor.

9. A method comprising:

accessing current point cloud data captured using a sensor connected to a vehicle;

detecting a crop row based on the current point cloud data;

detecting an end of the crop row based on point cloud data captured using the sensor;

planning a turn at the end of the crop row;

controlling one or more actuators to cause the vehicle to execute the turn;

after executing the turn, detecting a last seen plant at the end of the crop row;

based on a position of the last seen plant, identifying a trajectory to enter a next agricultural lane bounded by the crop row; and

controlling one or more actuators to cause the vehicle to enter the next agricultural lane.

10. The method of claim 9 , comprising:

detecting a furrow in the ground based on the current point cloud data;

matching the detected furrow with a furrow represented in a map; and

determining an estimate of a current location of the vehicle based on a current position in relation to the detected furrow.

11. The method of claim 9 , comprising:

filtering the current point cloud data to consider only points in a zone of interest when comparing the current point cloud data to point cloud data of a map to match the detected crop row with at crop row represented in the map, where in the zone of interest is limited to a range of heights and a maximum distance from the vehicle.

12. The method of claim 9 , comprising:

receiving image data, captured using one or more image sensors connected to the vehicle, depicting one or more plants in a vicinity of the vehicle;

detecting the one or more plants based on the image data;

responsive to detecting the one or more plants, adjusting implement control data; and

controlling, based on the adjusted implement control data, an implement to deposit an input on the one or more plants, wherein the implement is connected to the vehicle and configured to selectively deposit input in a vicinity of the vehicle.

13. The method of claim 12 , in which the one or more image sensors comprise a normalized difference vegetation index camera connected to the vehicle, and the method comprising:

receiving normalized difference vegetation index data, captured using the normalized difference vegetation index camera, for the one or more plants, wherein adjusting the implement control data includes adjusting an amount of input to be deposited on the one or more plants based on the normalized difference vegetation index data.

14. The method of claim 12 , in which the vehicle is a tractor and the implement is a sprayer.

15. The method of claim 12 , in which the implement control data is adjusted based on a count of the one or more plants detected in the vicinity of the vehicle.

16. The method of claim 9 , in which the vehicle is a tractor.

17. A non-transitory computer-readable storage medium storing executable instructions that, when executed by a processor, facilitate performance of operations comprising:

accessing current point cloud data captured using a sensor connected to a vehicle;

detecting a crop row based on the current point cloud data;

detecting an end of the crop row based on point cloud data captured using the sensor;

planning a turn at the end of the crop row;

controlling one or more actuators to cause the vehicle to execute the turn;

after executing the turn, detecting a last seen plant at the end of the crop row;

based on a position of the last seen plant, identifying a trajectory to enter a next agricultural lane bounded by the crop row; and

controlling one or more actuators to cause the vehicle to enter the next agricultural lane.

18. The non-transitory computer-readable storage medium of claim 17 , in which the operations comprise:

detecting a furrow in the ground based on the current point cloud data;

matching the detected furrow with a furrow represented in a map; and

determining an estimate of a current location of the vehicle based on a current position in relation to the detected furrow.

19. The non-transitory computer-readable storage medium of claim 17 , in which the operations comprise:

filtering the current point cloud data to consider only points in a zone of interest when comparing the current point cloud data to point cloud data of a map to match the detected crop row with a crop row represented in the map, where in the zone of interest is limited to a range of heights and a maximum distance from the vehicle.

20. The non-transitory computer-readable storage medium of claim 17 , in which the operations comprise:

receiving image data, captured using one or more image sensors connected to the vehicle, depicting one or more plants in a vicinity of the vehicle;

detecting the one or more plants based on the image data;

responsive to detecting the one or more plants, adjusting implement control data; and

controlling, based on the adjusted implement control data, an implement to deposit an input on the one or more plants, wherein the implement is connected to the vehicle and configured to selectively deposit input in a vicinity of the vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2023
From: BEAR FLAG ROBOTICS, INC.
To: DEERE & COMPANY
Reel/Frame 065960/0225 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2022
From: BERTUCCI, DAVID E.; CAFIERO, IGINO C.; DONNELLAN, AUBREY C.; NGUYEN, THUY T.; VISWANATHAN, VAIBHAV K.
To: BEAR FLAG ROBOTICS, INC.
Reel/Frame 059959/0865 →
Continuity (4)
Continuation 16521511 · Jul 24, 2019
Continuation 16288441 · Feb 28, 2019
Provisional Application 62703718 · Jul 26, 2018
Related Publication 20220272890A1 · Sep 1, 2022