IP Library › Granted Patent US 10,748,042
Granted Patent B2
US 10,748,042 · App. 16/015,665 · Granted Aug 18, 2020

Measuring crop residue from imagery using a machine-learned convolutional neural network

Inventors: Luca Ferrari (Modena, IT); John H. Posselius (Ephrata, PA); James W. Henry (Saskatoon, CA); Taylor C. Bybee (Logan, UT)
Assignees: CNH Industrial Canada, Ltd.; Autonomous Solutions, Inc.
G06K9/66A01B63/14A01B76/00G06K9/00657A01B49/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,748,042
App. No.
16/015,665
Granted
Aug 18, 2020
Kind
B2
Abstract

The present disclosure provides systems and methods that measure crop residue in a field from imagery of the field. In particular, the present subject matter is directed to systems and methods that include or otherwise leverage a machine-learned convolutional neural network to determine a level of crop residue for a portion of a field based at least in part on imagery of such portion of the field captured by an imaging device. For example, the imaging device can be a camera positioned in a downward-facing direction and physically coupled to a work vehicle or an implement towed by the work vehicle through the field.

Claims (93)

1. A computing system physically located on at least one of a work vehicle or an implement towed by the work vehicle, the computing system comprising:

one or more processors; and

one or more non-transitory computer-readable media that collectively store:

a machine-learned convolutional neural network configured to receive imagery and to process the imagery to select a level of crop residue from a plurality of pre-defined levels of crop residue; and

instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising:

obtaining image data that depicts a portion of a field, the image data being captured by an imaging device that is physically located on at least one of the work vehicle or the implement towed by the work vehicle,

inputting the image data into the machine-learned convolutional neural network;

receiving an output of the machine-learned convolutional neural network for the image data, wherein the output of the machine-learned convolutional neural network assigns to the image data a particular one of the plurality of pre-defined levels of crop residue, and

controlling an operation of at least one of the work vehicle or the implement towed by the work vehicle based at least in part on the output of the machine-learned convolutional neural network.

2. The computing system of claim 1 , wherein the plurality of pre-defined levels of crop residue comprise eleven or twenty-one crop residue percentages.

3. The computing system of claim 1 , wherein the machine-learned convolutional neural network is configured to provide the output at a final fully-connected layer of the machine-learned convolutional neural network.

4. The computing system of claim 3 , wherein the final fully-connected layer of the machine-learned convolutional neural network applies a softmax function.

5. The computing system of claim 1 , wherein the image data comprises a plurality of image frames.

6. The computing system of claim 1 , wherein the image data comprises a single image frame and wherein the operations further comprise:

obtaining a plurality of image frames that depict respective portions of the field;

performing the operations recited in claim 1 for each of the plurality of image frames to determine a plurality of frame-wise crop residue levels respectively for the plurality of image frames; and

determining a total crop residue value-for the field based at least in part on the plurality of frame-wise crop residue levels.

7. A computer-implemented method, comprising:

obtaining, by a computing system comprising one or more computing devices, image data that depicts a portion of a field, the computing system being physically located on at least one of a work vehicle or an implement towed by the work vehicle, the image data being captured by an imaging device that is physically located on at least one of the work vehicle or the implement towed by the work vehicle;

inputting, by the computing system, the image data into a machine-learned convolutional neural network that is configured to receive imagery and to process the imagery to select a level of crop residue from a plurality of pre-defined levels of crop residue;

receiving, by the computing system, an output of the machine-learned convolutional neural network for the image data, wherein the output of the machine-learned convolutional neural network assigns to the image data a particular one of the plurality of pre-defined levels of crop residue; and

controlling, by the computing system, an operation of at least one of the work vehicle or the implement towed by the work vehicle based at least in part on the output of the machine-learned convolutional neural network.

8. The computer-implemented method of claim 7 , wherein the plurality of pre-defined levels of crop residue consist of eleven or twenty-one crop residue percentages.

9. The computer-implemented method of claim 7 , wherein the machine-learned convolutional neural network is configured to provide the output at a final fully-connected layer of the machine-learned convolutional neural network.

10. The computer-implemented method of claim 9 , wherein the final fully-connected layer of the machine-learned convolutional neural network applies a softmax function.

11. The computer-implemented method of claim 7 , wherein the image data comprises a plurality of image frames.

12. The computer-implemented method of claim 7 , wherein the image data comprises a single image frame and wherein the method further comprises:

obtaining, by the computing system, a plurality of image frames that depict respective portions of the field;

performing, by the computing system, the computer-implemented method of claim 7 for each of the plurality of image frames to determine a plurality of frame-wise crop residue levels respectively for the plurality of image frames; and

determining, by the computing system, a total crop residue value for the field based at least in part on the plurality of frame-wise crop residue levels.

13. An agricultural work vehicle or implement, comprising:

one or more imaging devices; and

a controller comprising one or more processors and one or more non-transitory computer-readable media that collectively store:

a machine-learned convolutional neural network configured to receive imagery and to process the imagery to select a level of crop residue from a plurality of pre-defined levels of crop residue; and

instructions that, when executed by the one or more processors, configure the controller to perform operations, the operations comprising:

obtaining image data that depicts a portion of a field;

inputting the image data into the machine-learned convolutional neural network;

receiving an output of the machine-learned convolutional neural network for the image data, wherein the output of the machine-learned convolutional neural network assigns to the image data a particular one of the plurality of pre-defined levels of crop residue; and

controlling an operation of the agricultural work vehicle or implement based at least in part on the output of the machine-learned convolutional neural network.

14. The agricultural work vehicle or implement of claim 13 , wherein the plurality of pre-defined levels of crop residue comprise a plurality of bands of crop residue percentage.

15. The agricultural work vehicle or implement of claim 13 , wherein the machine-learned convolutional neural network is configured to provide the output at a final fully-connected layer of the machine-learned convolutional neural network.

16. The agricultural work vehicle or implement of claim 15 , wherein the final fully-connected layer of the machine-learned convolutional neural network applies a softmax function.

17. The agricultural work vehicle or implement of claim 13 , wherein the image data comprises a plurality of image frames.

18. The agricultural work vehicle or implement of claim 13 , wherein the image data comprises a single image frame and wherein the operations further comprise:

obtaining a plurality of image frames that depict respective portions of the field;

performing the operations recited in claim 13 for each of the plurality of image frames to determine a plurality of frame-wise crop residue values respectively for the plurality of image frames; and

determining a total crop residue value for the field based at least in part on the plurality of frame-wise crop residue values.

19. A computing system, comprising:

one or more processors; and

one or more non-transitory computer-readable media that collectively store:

a machine-learned convolutional neural network configured to receive imagery and to process the imagery to select a level of crop residue from a plurality of pre-defined levels of crop residue; and

instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising:

obtaining a plurality of image frames that depict respective portions of a field;

inputting each of the plurality of image frames into the machine-learned convolutional neural network;

receiving an output of the machine-learned convolutional neural network for each of the plurality of image frames, wherein the output of the machine-learned convolutional neural network assigns to each image frame a particular one of the plurality of pre-defined levels of crop residue such that a plurality of frame-wise crop residue levels are respectively determined for the plurality of image frames; and

determining a total crop residue value for the field based at least in part on the plurality of frame-wise crop residue levels.

20. The computing system of claim 19 , wherein:

the computing system is physically located on at least one of a work vehicle or an implement towed by the work vehicle;

obtaining the plurality of image frames comprises obtaining the plurality of image frames captured by an imaging device that is physically located on at least one of the work vehicle or the implement towed by the work vehicle; and

the operations further comprise controlling an operation of at least one of the work vehicle or the implement towed by the work vehicle based at least in part on at least one of the plurality of frame-wise crop residue levels or the determined total crop residue value for the field.

21. A computer-implemented method, comprising:

obtaining, by a computing system comprising one or more computing devices, a plurality of image frames that depict respective portions of a field;

inputting, by the computing system, the plurality of image frames into a machine-learned convolutional neural network that is configured to receive imagery and to process the imagery to select a level of crop residue from a plurality of pre-defined levels of crop residue;

receiving, by the computing system, an output of the machine-learned convolutional neural network for the image data, wherein the output of the machine-learned convolutional neural network assigns to each image frame a particular one of the plurality of pre-defined levels of crop residue such that a plurality of frame-wise crop residue levels are respectively determined for the plurality of image frames; and

determining, by the computing system, a total crop residue value for the field based at least in part on the plurality of frame-wise crop residue levels.

22. The computer-implemented method of claim 21 , wherein:

the computing system is physically located on at least one of a work vehicle or an implement towed by the work vehicle;

obtaining, by the computing system, the plurality of image frames comprises obtaining the plurality of image frames captured by an imaging device that is physically located on at least one of the work vehicle or the implement towed by the work vehicle; and

the method further comprises controlling, by the computing system, an operation of at least one of the work vehicle or the implement towed by the work vehicle based at least in part on at least one of the plurality of frame-wise crop residue levels or the determined total crop residue value for the field.

23. A computing system comprising:

one or more processors; and

one or more non-transitory computer-readable media that collectively store:

a machine-learned convolutional neural network configured to receive imagery and to process the imagery to select a level of crop residue from a plurality of pre-defined levels of crop residue; and

instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising:

obtaining image data that depicts a portion of a field;

inputting the image data into the machine-learned convolutional neural network; and

receiving an output of the machine-learned convolutional neural network for the image data, wherein the output of the machine-learned convolutional neural network assigns to the image data a particular one of the plurality of pre-defined levels of crop residue,

wherein the machine-learned convolutional neural network is configured to provide the output at a final fully-connected layer of the machine-learned convolutional neural network.

24. The computing system of claim 23 , wherein the final fully-connected layer of the machine-learned convolutional neural network applies a softmax function.

25. The computing system of claim 23 , wherein:

the computing system is physically located on at least one of a work vehicle or an implement towed by the work vehicle;

obtaining the image data of the field comprises obtaining image data captured by an imaging device that is physically located on at least one of the work vehicle or the implement towed by the work vehicle; and

the operations further comprise controlling an operation of at least one of the work vehicle or the implement towed by the work vehicle based at least in part on the output of the machine-learned convolutional neural network.

26. A computer-implemented method, comprising:

obtaining, by a computing system comprising one or more computing devices, image data that depicts a portion of a field;

inputting, by the computing system, the image data into a machine-learned convolutional neural network that is configured to receive imagery and to process the imagery to select a level of crop residue from a plurality of pre-defined levels of crop residue; and

receiving, by the computing system, an output of the machine-learned convolutional neural network for the image data, wherein the output of the machine-learned convolutional neural network assigns to the image data a particular one of the plurality of pre-defined levels of crop residue,

wherein the machine-learned convolutional neural network is configured to provide the output at a final fully-connected layer of the machine-learned convolutional neural network.

27. The computer-implemented method of claim 26 , wherein the final fully-connected layer of the machine-learned convolutional neural network applies a softmax function.

28. The computer-implemented method of claim 26 , wherein:

the computing system is physically located on at least one of a work vehicle or an implement towed by the work vehicle;

obtaining, by the computing system, the image data of the field comprises obtaining image data captured by an imaging device that is physically located on at least one of the work vehicle or the implement towed by the work vehicle; and

the method further comprises controlling, by the computing system, an operation of at least one of the work vehicle or the implement towed by the work vehicle based at least in part on the output of the machine-learned convolutional neural network.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2018
From: FERRARI, LUCA; POSSELIUS, JOHN H.; HENRY, JAMES W.; BYBEE, TAYLOR C.
To: CNH INDUSTRIAL CANADA, LTD.; AUTONOMOUS SOLUTIONS, INC.
Reel/Frame 046177/0379 →
Continuity (1)
Related Publication 20190392269A1 · Dec 26, 2019