IP Library › Granted Patent US 11,455,794
Granted Patent B2
US 11,455,794 · App. 16/923,161 · Granted Sep 27, 2022

System and method for orchard recognition on geographic area

Inventors: Cheng-Ying Chou (Taipei, TW); Yu-Fang Hsieh (Taipei, TW); Yen-Shuo Chen (Taipei, TW); Po-Ting Bertram Liu (Taipei, TW)
Assignee: National Taiwan University
G06V20/188A01G17/00G06N3/04G06V10/273
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,455,794
App. No.
16/923,161
Granted
Sep 27, 2022
Kind
B2
Abstract

A system and a method for recognition of an orchard on a geographic area are provided. The system includes a pre-processing module for deriving a target section of an aerial image containing a parcel of an orchard, an image optimization module for performing customized image processing on the target section of the aerial image, and a recognition module for determining a type and a border of the orchard present on the target section of the aerial image with a deep learning mechanism. Accordingly, farmers and agricultural entities can effectively monitor orchards within different geographic areas so as to yield better fruit production and conduct better fruit quality control and land utilization.

Claims (48)

1. A computer-implemented system for recognizing an orchard on a geographic area, the computer-implemented system having a processor by which programs executed causing the system to perform:

deriving a target section of an aerial image containing a parcel of the orchard and a mask corresponding to a shape of the orchard from the aerial image and a shapefile;

conditioning the target section of the aerial image; and

determining a type of the orchard within the target section of the aerial image by a deep learning mechanism based on at least one of fruiters growing characteristic and agricultural operation characteristic, and determining a border of the orchard within the target section of the aerial image by the deep learning mechanism based on the mask corresponding to the shape of the orchard and the conditioned target section of the aerial image containing the parcel of the orchard.

2. The computer-implemented system of claim 1 , wherein the border of the orchard winds along the orchard.

3. The computer-implemented system of claim 1 , wherein conditioning the target section of the aerial image further comprises:

determining type of flaws in the target section of the aerial image with a classifier; and

performing at least one of cloud and haze removal, color calibration, image modulating and shadow removal on the target section based on the type of flaws with an optimizer.

4. The computer-implemented system of claim 1 , wherein the deep learning mechanism determines the type of the orchard and the border of the orchard within the target section of the aerial image by:

identifying a region of interest containing the orchard in the target section of the aerial image;

recognizing fruiters planted within the orchard in the region of interest to determine the type of the orchard;

generating a mask for the region of interest to segment the border of the orchard, the mask marking a border shape of the orchard in the region of interest; and

outputting an image marked with the type and the border of the orchard.

5. The computer-implemented system of claim 4 , wherein the programs executed by the processor further cause the system to perform: presenting the image marked with the type and the border of the orchard onto a user interface.

6. The computer-implemented system of claim 1 , wherein deriving the target section of the aerial image containing the parcel of the orchard and the mask corresponding to the shape of the orchard from the aerial image and the shapefile further comprises:

generating the parcel of the orchard by comparing the aerial image to the shapefile of the orchard, the shapefile having shape information of the orchard;

segmenting the aerial image to produce the target section of the aerial image containing the parcel of the orchard; and

generating the mask corresponding to the shape of the orchard using the shapefile of the orchard.

7. The computer-implemented system of claim 6 , wherein at least one of the parcel of the orchard and the mask corresponding to the shape of the orchard act as training data for the deep learning mechanism.

8. The computer-implemented system of claim 7 , wherein the deep learning mechanism is trained to determine the type of the orchard by extracting a feature in the aerial image corresponding to a characteristic of fruiters planted within the border of the orchard.

9. The computer-implemented system of claim 1 , wherein the programs executed by the processor further cause the system to perform: receiving the aerial image and the shapefile.

10. The computer-implemented system of claim 1 , wherein the programs executed by the processor further cause the system to:

classify type of flaws in the target section of the aerial image; and

perform at least one of cloud and haze removal, color calibration, image modulating and shadow removal on the target section based on the type of flaws.

11. A computer-implemented method for recognizing an orchard on a geographic area, comprising:

deriving a target section of an aerial image containing a parcel of the orchard from the aerial image and a shapefile;

deriving a mask corresponding to a shape of the orchard from the shapefile;

conditioning the target section of the aerial image; and

determining a type of the orchard within the target section of the aerial image by a deep learning mechanism based on at least one of fruiters growing characteristic and agricultural operation characteristic; and

determining a border of the orchard within the target section of the aerial image by the deep learning mechanism based on the mask corresponding to the shape of the orchard and the conditioned target section of the aerial image containing the parcel of the orchard.

12. The computer-implemented method of claim 11 , wherein the border of the orchard winds along the orchard.

13. The computer-implemented method of claim 11 , wherein conditioning the target section of the aerial image comprises:

determining type of flaws in the target section of the aerial image with a classifier; and

performing at least one of cloud and haze removal, color calibration, image modulating and shadow removal on the target section of the aerial image based on the type of flaws with an optimizer.

14. The computer-implemented method of claim 11 , wherein determining the type of the orchard within the target section of the aerial image by the deep learning mechanism comprises:

identifying a region of interest containing the orchard in the target section of the aerial image; and

extracting a feature in the target section of the aerial image corresponding to a characteristic of a fruiter planted within the orchard.

15. The computer-implemented method of claim 14 , wherein determining the border of the orchard by the deep learning mechanical comprises:

generating a mask for the region of interest to segment the border of the orchard, the mask marking a border shape of the orchard in the region of interest.

16. The computer-implemented method of claim 11 , wherein deriving the target section of the aerial image containing the parcel of the orchard from the aerial image and the shapefile comprises:

generating the parcel of the orchard by comparing the aerial image to the shapefile of the orchard, the shapefile having shape information of the orchard; and

segmenting the aerial image to produce the target section of the aerial image containing the parcel of the orchard.

17. The computer-implemented method of claim 16 , wherein deriving the mask corresponding to the shape of the orchard from the shapefile comprises:

generating the mask corresponding to the shape of the orchard using the shapefile of the orchard.

18. The computer-implemented method of claim 11 , further comprising:

outputting an image marked with the type and the border of the orchard; and

presenting the image marked with the type and the border of the orchard and the aerial image onto a user interface.

19. The computer-implemented method of claim 11 , wherein the deep learning mechanism is a Mask R-CNN model.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 8, 2020
From: CHOU, CHENG-YING; HSIEH, YU-FANG; CHEN, YEN-SHUO; LIU, PO-TING BERTRAM
To: NATIONAL TAIWAN UNIVERSITY
Reel/Frame 053145/0389 →
Continuity (2)
Provisional Application 62871252 · Jul 8, 2019
Related Publication 20210012109A1 · Jan 14, 2021