IP Library Granted Patent US 11,420,652
Granted Patent B2
US 11,420,652 · App. 16/917,862 · Granted Aug 23, 2022

System and method of control for autonomous or remote-controlled vehicle platform

Inventors: Richard Connell (Moline, IL); Michael M. Miller (Moline, IL); Joshua Lawson (Moline, IL); Suryansh Saxena (Pittsburgh, PA); Matthew Levine (Pittsburgh, PA); Prathamesh Kini (Pittsburgh, PA); Raphael Bouterige (Pittsburgh, PA); Matthew Glisson (Pittsburgh, PA); Raymond Russell (Pittsburgh, PA); James Ketterer (Pittsburgh, PA); Herman Herman (Gibsonia, PA)
Assignees: DEERE & COMPANY; CARNEGIE MELLON UNIVERSITY
B60W60/0015B60W30/08G06V20/58B60W2420/52B60W2520/14B60W2520/18H04W4/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,420,652
App. No.
16/917,862
Granted
Aug 23, 2022
Kind
B2
Abstract

A system and method for controlling a vehicle platform, the system comprising on onboard controller and an off-board controller that work together to provide autonomous navigation in fields or similar areas where the vehicle is deployed, perception for obstacle detection and avoidance, and a user interface for user/vehicle interaction and control.

Claims (47)

1. A system for controlling a vehicle operating autonomously in an agricultural field and having at least one sensor capable of providing data related to an environment of the vehicle, the system comprising:

an onboard controller connected to a data port on the vehicle for receiving the data and developing a navigation plan, the onboard controller comprising:

an interface control module that provides instructions to low-level controllers to execute the navigation plan,

a pose estimation module that estimates a pose of the vehicle with respect to a global frame,

a perception module that identifies obstacles within a vicinity of the vehicle,

a path navigation module that tracks the vehicle and sends navigation updates to the interface control module, and

an obstacle avoidance module that plans a path to avoid the obstacles identified by the perception module;

an off-board controller that sends mission data and a set of way-points to the onboard controller,

wherein the onboard controller develops the navigation plan based on at least one of the mission data and the set of way-points, and

wherein the set of way-points define a path to cover an area of the agricultural field identified in the mission data.

2. The system of claim 1 , wherein the off-board controller comprises:

a mission configuration module that receives mission details from a user to generate a mission output;

a static machine module that receives the mission output and generates the set of way-points used by the onboard controller to develop the navigation plan.

3. The system of claim 2 , wherein the off-board controller further comprises:

a machine state module that receives state data from the vehicle and displays the state data on a user interface.

4. The system of claim 2 , wherein the mission details are selected from a group consisting of application area, application rate, navigation speed, vehicle width, vehicle turning threshold, and refueling threshold.

5. The system of claim 2 , wherein the way-points comprise tender and normal way-points.

6. The system of claim 1 , wherein the off-board controller further comprises:

an override controls module that sends override data to the onboard controller, wherein the override data interrupts autonomous operation of the vehicle.

7. The system of claim 6 , wherein the override data comprises a message related to the mission data.

8. The system of claim 6 , wherein the override data comprise a message when received by the onboard controller causes the vehicle to stop.

9. The system of claim 1 , wherein the onboard controller further comprises:

a remote operation module that receives data from the at least one sensor and transmits the data to the off-board controller for display on a user interface.

10. The system of claim 1 , wherein the low-level controllers are selected from a group consisting of a steering controller, a braking controller, and a propulsion controller.

11. The system of claim 1 , wherein the pose estimation module estimates the pose of the vehicle by using an extended Kalman filter that assumes a uniform velocity model.

12. The system of claim 11 , wherein the pose is updated using data from the at least one sensor.

13. The system of claim 1 , wherein the perception module creates a map in a global coordinate frame using data from the at least one sensor.

14. The system of claim 1 , wherein the perception module identifies objects in the environment without reliance on image features.

15. The system of claim 1 , wherein the off-board controller is connected to the data port via a wireless link.

16. The system of claim 1 , wherein the at least one sensor is selected from a group consisting of: an imaging system, a stereo camera, a radar system, a lidar system, a location-determining receiver, a satellite navigation receiver, a dead-reckoning sensor, an odometer, a gyroscope, an accelerometer, an inertial measurement unit, and a tilt, roll, and yaw sensor.

17. The system of claim 1 , wherein the data comprises at least one of position data, motion data, and attitude data.

18. A method of controlling an autonomous vehicle operating in an agricultural field and having an onboard controller comprising:

generating mission data and a set of way-points using an off-board controller,

wherein the mission data comprises an initial map of a region to be traversed by the autonomous vehicle, and

wherein the set of way-points define a path to cover the region;

receiving the mission data and set of way-points at an onboard controller;

developing a path plan using at least one of the mission data and set of way-points;

determining a pose of the vehicle;

navigating the path plan by tracking the set of way-points; and

detecting objects within the path plan using a sensor, wherein the sensor provides information about an environment surrounding the autonomous vehicle.

19. The method of claim 18 , where detecting objects comprises:

computing depth information from a stereo camera and a lidar point cloud having a plurality of points, wherein the points are projected into a local reference frame;

accumulating the plurality of points over a period of time in a global coordinate frame identified in the pose;

clustering voxels from the global coordinate frame voxel clusters; and

determining an objectness-score based on the voxel clusters.

20. The method of claim 19 , further comprising:

identifying an object using the objectness-score.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2022
From: SAXENA, SURYANSH; LEVINE, MATTHEW; KINI, PRATHAMESH; BOUTERIGE, RAPHAEL; GLISSON, MATTHEW; RUSSELL, RAYMOND; KETTERER, JAMES; HERMAN, HERMAN
To: CARNEGIE MELLON UNIVERSITY
Reel/Frame 059520/0375 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 16, 2021
From: CONNELL, RICHARD; MILLER, MICHAEL M.; LAWSON, JOSHUA; HAGIE MANUFACTURING COMPANY, LLC
To: DEERE & COMPANY
Reel/Frame 055943/0372 →
Continuity (2)
Provisional Application 62894685 · Aug 30, 2019
Related Publication 20210061307A1 · Mar 4, 2021
Cited By (2)
US 12,403,590 US 12,447,829