IP Library Granted Patent US 11,778,934
Granted Patent B2
US 11,778,934 · App. 16/918,388 · Granted Oct 10, 2023

Agricultural lane following

Inventors: Ameer Ellaboudy (Newark, CA); Aubrey C. Donnellan (San Mateo, CA); Austin Chun (San Jose, CA); Igino C. Cafiero (Palo Alto, CA); David E. Bertucci (Sunnyvale, CA); Thuy T. Nguyen (Albany, CA)
Assignee: Bear Flag Robotics, Inc.
A01B69/001A01B69/008B60W40/114G06V10/454G06V10/764G06V20/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,778,934
App. No.
16/918,388
Granted
Oct 10, 2023
Kind
B2
Abstract

Systems and methods for agricultural lane following are described. For example, a method includes accessing range data captured using a distance sensor connected to a vehicle and/or image data captured using an image sensor connected to a vehicle; detecting a crop row based on the range data and/or the image data to obtain position data for the crop row; determining, based on the position data for the crop row, a yaw and a lateral position of the vehicle with respect to a lane bounded by the crop row; and based on the yaw and the lateral position, controlling the vehicle to move along a length of the lane bounded by the crop row.

Claims (46)

1. A method comprising:

accessing image data captured using one or more image sensors connected to a vehicle;

detecting a crop row based on the image data to obtain position data for the crop row, wherein the crop row includes a raised planting bed;

determining, based on the position data for the crop row, a yaw and a lateral position of the vehicle with respect to a lane bounded by the crop row;

detecting one or more edges of the raised planting bed based on the image data, and associating the one or more edges of the raised planting bed with positions;

fitting a line to the positions associated with an edge of the raised planting bed, wherein fitting the line includes determining a least squares fit of a line parallel to a ground plane to the positions associated with the edge of the raised planting bed;

determining the yaw and the lateral position based on the line; and

based on the yaw and the lateral position, controlling the vehicle to move along a length of the lane bounded by the crop row.

2. The method of claim 1 , in which the crop row is a left crop row, comprising:

detecting a right crop row based on the image data to obtain position data for the right crop row;

fitting a first line to position data for multiple plants in the left crop row;

fitting a second line to position data for multiple plants in the right crop row; and

determining the yaw and the lateral position based on the first line and the second line.

3. The method of claim 1 , comprising:

determining a bounding box for a plant of the crop row based on the image data; and

determining a distance from the vehicle to a plant in the crop row based on an assumed height of a bottom of the bounding box relative to the one or more image sensors.

4. The method of claim 1 , comprising:

determining a distance from the vehicle to a plant in the crop row based on a size in pixels of the plant as it appears in the image data and a constant physical size parameter of the plant.

5. The method of claim 1 , in which one or more image sensors include two image sensors with overlapping fields of view, comprising:

determine a distance from the vehicle to a plant in the crop row based on stereoscopic signal processing of image data from the two image sensors depicting the plant.

6. The method of claim 1 , comprising:

accessing range data captured using a distance sensor connected to the vehicle, in which the distance sensor is any of a radar sensor and a lidar sensor; and

determining a position of a plant in the crop row based on the image data and the range data.

7. The method of claim 1 , in which the one or more image sensors comprise a normalized difference vegetation index camera connected to the vehicle, comprising:

accessing normalized difference vegetation index data, captured using the normalized difference vegetation index camera; and

detecting the crop row based on the normalized difference vegetation index data.

8. The method of claim 1 , comprising:

inputting the image data to a neural network to detect a first plant of the crop row and obtain position data for the first plant;

fitting a first line to position data for multiple plants of the crop row, including the position data for the first plant; and

determining the yaw and the lateral position based on the first line.

9. A system comprising:

an image sensor connected to a vehicle;

actuators configured to control motion of the vehicle; and

a processing apparatus configured to:

access image data captured using the image sensor;

detect a crop row based on the image data to obtain position data for the crop row, wherein the crop row includes a raised planting bed;

determine, based on the position data for crop row, a yaw and a lateral position of the vehicle with respect to a lane bounded by the crop row;

detect one or more edges of the raised planting bed based on the image data, and associating the one or more edges of the raised planting bed with positions;

fit a line to the positions associated with an edge of the raised planting bed wherein fitting the line includes determining a least squares fit of a line parallel to a ground plane to the positions associated with the edge of the raised planting bed;

determine the yaw and the lateral position based on the line; and

based on the yaw and the lateral position, control the vehicle to move along a length of the lane bounded by the crop row.

10. The system of claim 9 , in which the crop row is a left crop row, and the processing apparatus is configured to:

detect a right crop row based on the image data to obtain position data for the right crop row;

fit a first line to position data for multiple plants in the left crop row;

fit a second line to position data for multiple plants in the right crop row; and

determine the yaw and the lateral position based on the first line and the second line.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2023
From: BEAR FLAG ROBOTICS, INC.
To: DEERE & COMPANY
Reel/Frame 065960/0225 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 23, 2020
From: ELLABOUDY, AMEER; DONNELLAN, AUBREY C.; CHUN, AUSTIN; CAFIERO, IGINO C.; BERTUCCI, DAVID E.; NGUYEN, THUY T.
To: BEAR FLAG ROBOTICS, INC.
Reel/Frame 054440/0452 →
Continuity (2)
Provisional Application 62869865 · Jul 2, 2019
Related Publication 20210000006A1 · Jan 7, 2021
Cited By (1)
US 12,543,620