IP Library Granted Patent US 11,058,042
Granted Patent B2
US 11,058,042 · App. 16/720,021 · Granted Jul 13, 2021

Method and apparatus for automated plant necrosis

Inventor: Lee Kamp Redden (Palo Alto, CA)
Assignee: Blue River Technology Inc.
A01B41/06A01D75/00A01G7/00A01M21/00G06K9/00657G06K9/2036G06K9/3241G06T7/11G06T7/66G06T7/73G06K2209/17G06T2207/20164G06T2207/30128G06T2207/30188G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,058,042
App. No.
16/720,021
Granted
Jul 13, 2021
Kind
B2
Abstract

A method of real-time plant selection and removal from a plant field including capturing a first image of a first section of the plant field, segmenting the first image into regions indicative of individual plants within the first section, selecting the optimal plants for retention from the first image based on the first image and the previously thinned plant field sections, sending instructions to the plant removal mechanism for removal of the plants corresponding to the unselected regions of the first image from the second section before the machine passes the unselected regions, and repeating the aforementioned steps for a second section of the plant field adjacent the first section in the direction of machine travel.

Claims (98)

1. A method comprising:

accessing an image of a field of plants with a camera physically attached to a machine travelling in a direction through the field;

determining, from the image, a first plant to be retained in the field and a second plant to be removed from the field, wherein a first portion of the first plant overlaps with a second portion of the second plant;

determining an amount of overlap between the first portion and the second portion;

responsive to determining the amount of overlap exceeds a threshold, determining to remove the first plant from the field; and

activating a plant removal mechanism to remove the first plant as the plant removal mechanism passes by the first plant while moving through the field.

2. The method of claim 1 , wherein identifying the first plant to be retained in the field and the second plant to be removed from the field comprises:

identifying one or more points of interest located in a region of the image;

classifying a subset of the identified one or more points of interest as plant centers;

segmenting the image into a plurality of sub-regions, wherein each sub-region of the plurality of sub-regions includes a point of interest classified as a plant center and a set of cultivation parameters;

for each sub-region, classifying each sub-region of the plurality of sub-regions as including a plant in a first plurality of plants to be retained in the field or as including a plant in a second plurality of plants to be removed from the field;

wherein the classification for each sub-region of the plurality of sub-regions improves the cultivation parameters for sub-regions in the first plurality of plants; and

wherein the first plurality of plants comprises the first plant to be retained in the field, and the second plurality of plants comprises the second plant to be removed from the field.

3. The method of claim 1 , further comprising:

segmenting the image into a plurality of sub-regions, wherein each sub-region of the plurality of sub-regions includes a set of cultivation parameters;

for each sub-region, classifying the sub-region as including a plant in a first group of plants to be retained in the field, or including a plant in a second group of plants to be removed from the field based on the set of cultivation parameters for that sub-region; and

determining a pattern of retained plants for a region of the field in the image based on the sub-regions of the plurality of sub-regions classified as including plants in the first group of plants.

4. The method of claim 3 , wherein determining the pattern of retained plants in the image comprises:

determining a plurality of candidate patterns for plants in the first and second groups of plants, each candidate pattern of the plurality of candidate patterns describing a unique combination of plants to be retained in the field and plants to be removed from the field;

determining, for each candidate pattern of the plurality of candidate patterns, a cost value for removing plants from the field; and

selecting a candidate pattern of the plurality of candidate patterns with a lowest cost value as the plant pattern.

5. The method of claim 4 , wherein the plant pattern is determined based on one or more of the following:

a categorization of the candidate plant patterns; and

a classification of the candidate plant patterns.

6. The method of claim 3 , wherein determining the plant pattern for the region of the field comprises:

determining an additional plant pattern for a different region of the field, wherein:

the additional plant pattern includes plants to be retained and plants to be removed in the different region, and

the additional plant pattern includes at least a portion of the plant pattern; and

generating an aggregate plant pattern by combining the additional plant pattern and the plant pattern.

7. The method of claim 1 , wherein determining the amount of overlap further comprises:

determining a distance between a plant center of the first plant and a plant center of the second plant;

comparing the determined distance to a threshold distance; and

responsive to the determined distance being below the threshold distance, determining that the first plant and second plant overlap.

8. The method of claim 1 , further comprising:

responsive to determining the amount of overlap exceeds a threshold, determining to retain the second plant from the field; and

updating a plant pattern to include the second plant from the field to be retained.

9. The method of claim 1 , wherein the plant removal mechanism comprises a spray nozzle, the method further comprising:

determining that the activated plant removal mechanism cannot retain the first plant to be retained in the field due to the first plant being within a range of the spray nozzle;

generating an updated plant pattern comprising instructions to remove the first plant from the field;

determining a cost value for the updated plant pattern; and

responsive to determining the cost value to be below a threshold cost value, instructing the plant removal mechanism to remove the first plant from the field according to the updated plant pattern.

10. The method of claim 1 , further comprising:

determining that the activated plant removal mechanism cannot remove the second plant from the field;

generating an updated plant pattern comprising instructions to retain the second plant in the field;

determining a cost value for the updated plant pattern; and

responsive to determining the cost value to be below a threshold cost value, instructing the plant removal mechanism to retain the second plant in the field according to the updated plant pattern.

11. A non-transitory computer readable medium storing instructions, the instructions when executed cause a processor physically attached to a machine travelling in a direction through a field of plants to:

access an image of a field of plants with a camera physically attached to a machine travelling in a direction through the field;

determine, from the image, a first plant to be retained in the field and a second plant to be removed from the field, wherein a first portion of the first plant overlaps with a second portion of the second plant;

determine an amount of overlap between the first portion and the second portion;

responsive to determining the amount of overlap exceeds a threshold, determine to remove the first plant from the field; and

activate a plant removal mechanism to remove the first plant as the plant removal mechanism passes by the first plant while moving through the field.

12. A system comprising:

a camera physically attached to a machine traveling in a direction through a field, the camera configured to capture an image of a field of plants;

a plant removal mechanism attached to the machine; and

a non-transitory computer readable medium storing instructions that, when executed, cause a processor physically attached to the machine to:

access an image of a field of plants with a camera physically attached to a machine travelling in a direction through the field;

determine, from the image, a first plant to be retained in the field and a second plant to be removed from the field, wherein a first portion of the first plant overlaps with a second portion of the second plant;

determine an amount of overlap between the first portion and the second portion;

responsive to determining the amount of overlap exceeds a threshold, determine to remove the first plant from the field; and

activate a plant removal mechanism to remove the first plant as the plant removal mechanism passes by the first plant while moving through the field.

13. The system of claim 12 , wherein instructions to identify the first plant to be retained in the field and the second plant to be removed from the field further cause the processor to:

identify one or more points of interest located in a region of the image;

classify a subset of the identified one or more points of interest as plant centers;

segment the image into a plurality of sub-regions, wherein each sub-region of the plurality of sub-regions includes a point of interest classified as a plant center and a set of cultivation parameters;

for each sub-region, classifying each sub-region of the plurality of sub-regions as including a plant in a first plurality of plants to be retained in the field or as including a plant in a second plurality of plants to be removed from the field;

wherein the classification for each sub-region of the plurality of sub-regions improves the cultivation parameters for sub-regions in the first plurality; and

wherein the first plurality of plants comprises the first plant to be retained in the field, and the second plurality of plants comprises the second plant to be removed from the field.

14. The system of claim 12 , further comprising instructions that cause the processor to:

segment the image into a plurality of sub-regions, wherein each sub-region of the plurality of sub-regions includes a set of cultivation parameters;

for each sub-region, classify the sub-region as a plant in a first group of plants to be retained in the field, or a plant in a second group of plants to be removed from the field based on the set of cultivation parameters for that sub-region; and

determine a pattern of retained plants for a region of the field in the image based on the sub-regions of the plurality of sub-regions classified as including plants in the first group of plants.

15. The system of claim 14 , wherein instructions to determine the pattern of retained plants in the image further cause the processor to:

determine a plurality of candidate patterns for the plants in the first and second groups of plants, each candidate pattern of the plurality of candidate patterns describing a unique combination of plants to be retained in the field and plants to be removed from the field;

determine, for each candidate pattern of the plurality of candidate patterns, a cost value for removing plants from the field; and

select a candidate pattern of the plurality of candidate patters with a lowest cost value as the plant pattern.

16. The system of claim 14 , wherein instructions to determine the plant pattern for the region of the field further cause the processor to:

determine an additional plant pattern for a different region of the field, wherein:

the additional plant pattern includes plants to be retained and plants to be removed in the ether different region, and

the additional plant pattern includes at least a portion of the plant pattern; and

generate an aggregate plant pattern by combining the additional plant pattern and the optimal plant pattern.

17. The system of claim 12 , wherein instructions to determine the amount of overlap further cause the processor to:

determine a distance between a plant center of the first plant and a plant center of the second plant;

compare the determined distance to a threshold distance; and

responsive to the determined distance being below the threshold distance, determine that the first plant and second plant overlap.

18. The system of claim 12 , further comprising instructions that cause the processor to:

responsive to determining the amount of overlap exceeds a threshold, determine to retain the second plant from the field; and

update a plant pattern to include the second plant from the field to be retained.

19. The system of claim 12 , wherein the plant removal mechanism comprises a spray nozzle, the instructions further causing the processor to:

determine that the activated plant removal mechanism cannot retain the first plant to be retained in the field due to the first plant being within a range of the spray nozzle;

generate an updated plant pattern comprising instructions to remove the first plant from the field;

determine a cost value for the updated plant pattern; and

responsive to determining the cost value to be below a threshold cost value, instruct the plant removal mechanism to remove the first plant from the field according to the updated plant pattern.

20. The system of claim 12 , further comprising instructions that cause the processor to:

determine that the activated plant removal mechanism cannot remove the second plant from the field;

generate an updated plant pattern comprising instructions to retain the second plant in the field;

determine a cost value for the updated plant pattern; and

responsive to determining the cost value to be below a threshold cost value, instruct the plant removal mechanism to retain the second plant in the field according to the updated plant pattern.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2024
From: BLUE RIVER TECHNOLOGY INC.
To: DEERE & COMPANY
Reel/Frame 069164/0195 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 16, 2020
From: REDDEN, LEE
To: BLUE RIVER TECHNOLOGY INC.
Reel/Frame 051529/0140 →
Continuity (6)
Continuation 15665025 · Jul 31, 2017
Continuation 14713362 · May 15, 2015
Continuation 13788359 · Mar 7, 2013
Provisional Application 61608005 · Mar 7, 2012
Provisional Application 61609767 · Mar 12, 2012
Related Publication 20200187406A1 · Jun 18, 2020
Cited By (2)
US 12,201,044 US 12,317,881