IP Library › Granted Patent US 11,710,214
Granted Patent B2
US 11,710,214 · App. 17/907,695 · Granted Jul 25, 2023

Cloud-based framework for processing, analyzing, and visualizing imaging data

Inventors: Ioannis Ampatzidis (Gainesville, FL); Victor H. Meirelles Partel (Gainesville, FL); Lucas Fideles Costa (Gainesville, FL)
Assignee: University of Florida Research Foundation, Incorporated
G06T3/4038G06F18/40G06T1/20G06T7/38G06T7/70G06T7/97G06V10/16G06V10/25G06V10/82G06V10/94G06V10/945G06V10/95G06V10/96G06V20/17G06V20/188G06T2207/20104G06T2207/30188G06V2201/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,710,214
App. No.
17/907,695
Granted
Jul 25, 2023
Kind
B2
Abstract

Embodiments of the present disclosure provide methods, apparatus, systems, computing devices, computing entities, and/or the like for detecting objects located in an area of interest. In accordance with one embodiment, a method is provided comprising: receiving, via an interface provided through a general instance on a cloud environment, imaging data comprising raw images collected on the area of interest; upon receiving the images: activating a central processing unit (CPU) focused instance on the cloud environment and processing, via the image, the raw images to generate an image map of the area of interest; and after generating the image map: activating a graphical processing unit (GPU) focused instance on the cloud environment and performing object detection, via the image, on a region within the image map by applying one or more object detection algorithms to the region to identify locations of the objects in the region.

Claims (83)

1. A method for detecting a plurality of objects located in an area of interest, the method comprising:

receiving, via an interface provided through a general instance on a cloud environment, imaging data comprising a plurality of raw images collected on the area of interest;

upon receiving the plurality of raw images:

activating, via the general instance, a central processing unit (CPU)-focused instance on the cloud environment based at least in part on a CPU-focused machine configuration image;

upon activating the CPU-focused instance, processing, via the CPU-focused instance, the plurality of raw images to stitch together each of the raw images of the plurality of raw images to generate an image map of the area of interest; and

upon completion of generating the image map, closing, via the general instance, the CPU-focused instance on the cloud environment; and

after generating the image map:

activating, via the general instance, a graphical processing unit (GPU)-focused instance on the cloud environment based at least in part on a GPU-focused machine configuration image;

upon activating the GPU-focused instance, performing object detection, via the GPU-focused instance, on at least a region within the image map by applying one or more object detection algorithms to the region of the image map to identify locations of the plurality of objects in the region of the image map; and

upon completion of detecting the plurality of objects for the region, closing, via the general instance, the GPU-focused instance on the cloud environment; and

performing one or more cloud-based actions based at least in part on the plurality of objects detected for the region within the image map.

2. The method of claim 1 , wherein performing the object detection on the region within the image map comprises:

preprocessing the image map to reduce variation in the image map resulting from capturing the plurality of raw images;

applying a first object detection algorithm of the one or more object detection algorithms to identify initial locations of the plurality of objects in the region of the image map;

identifying one or more object patterns in the image map;

analyzing the one or more object patterns to identify one or more false positives in the initial locations of the plurality of objects;

removing the one or more false positives from the initial locations of the plurality of objects; and

after removing the one or more false positives from the initial locations of the plurality of objects, applying a second object detection algorithm of the one or more object detection algorithms to each of the one or more object patterns to identify the locations of the plurality of objects.

3. The method of claim 2 , wherein the area of interest comprises a tree grove, the plurality of objects comprises a plurality of trees, and the one or more object patterns comprise one or more rows of trees.

4. The method of claim 1 further comprising receiving input, via the interface, originating from a user, wherein the input comprises at least one of the region within the image map, an object spacing identifying an average space between objects located in the region, or one or more blank areas found in the region to be skipped by the one or more object detection algorithms.

5. The method of claim 1 , wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more cloud-based actions comprise:

generating processing data for the plurality of trees, the processing data comprising at least one of a total number of trees for the plurality of trees, one or more tree gap counts, an average value of tree heights, a tree height for one or more of the plurality of trees, a canopy area estimation for one or more of the plurality of trees, a yield estimation of fruit for one or more of the plurality of trees, an estimation of tree ages for one or more of the plurality of trees, an estimation of tree health for one or more of the plurality of trees, estimated nutrient concentrations for one or more of the plurality of trees, or a fertility map based at least in part on estimated nutrient concentrations for the plurality of trees; and

providing at least a portion of the processing data for display via the interface on a user device.

6. The method of claim 5 , wherein the one or more cloud-based actions comprise:

receiving an input indicating a selection of the region from a user via the user device; and

responsive to receiving the input indicating the selection of the region, providing a map of the region for display via the interface on the user device, the map displaying the plurality of trees detected for the region.

7. The method of claim 1 , wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more cloud-based actions comprise:

generating an application map identifying an application rate for at least the region within the image map; and

downloading the application map to a smart sprayer system configured to use the application map to control flow of a liquid being applied to the plurality of trees in the region.

8. An apparatus operating within a general instance of a cloud environment for detecting a plurality of objects located in an area of interest, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the at least one processor, cause the apparatus to at least:

receive, via an interface, imaging data comprising a plurality of raw images collected on the area of interest;

upon receiving the plurality of raw images:

activate a central processing unit (CPU)-focused instance on the cloud environment based at least in part on a CPU-focused machine configuration image, wherein upon activating the CPU-focused instance, the CPU-focused instance processes the plurality of raw images to stitch together each of the raw images of the plurality of raw images to generate an image map of the area of interest; and

upon completion of generating the image map, close the CPU-focused instance on the cloud environment; and

after the image map is generated:

activate a graphical processing unit (GPU)-focused instance on the cloud environment based at least in part on a GPU-focused machine configuration image, wherein upon activating the GPU-focused instance, the GPU-focused instance performs object detection on at least a region within the image map by applying one or more object detection algorithms to the region of the image map to identify locations of the plurality of objects in the region of the image map; and

upon completion of detecting the plurality of objects for the region, close the GPU-focused instance on the cloud environment; and

perform one or more cloud-based actions based at least in part on the plurality of objects detected for the region within the image map.

9. The apparatus of claim 8 , wherein the GPU-focused instance performs the object detection on the region within the image map by:

preprocessing the image map to reduce variation in the image map resulting from capturing the plurality of raw images;

applying a first object detection algorithm of the one or more object detection algorithms to identify initial locations of the plurality of objects in the region of the image map;

identifying one or more object patterns in the image map;

analyzing the one or more object patterns to identify one or more false positives in the initial locations of the plurality of objects;

removing the one or more false positives from the initial locations of the plurality of objects; and

after removing the one or more false positives from the initial locations of the plurality of objects, applying a second object detection algorithm of the one or more object detection algorithms to each of the one or more object patterns to identify the locations of the plurality of objects.

10. The apparatus of claim 9 , wherein the area of interest comprises a tree grove, the plurality of objects comprises a plurality of trees, and the one or more object patterns comprise one or more rows of trees.

11. The apparatus of claim 8 , wherein the GPU-focused instance performs the object detection of the region of the image map by receiving input, via the interface, originating from a user, the input comprising at least one of the region within the image map, an object spacing identifying an average space between objects located in the region, or one or more blank areas found in the region to be skipped by the one or more object detection algorithms.

12. The apparatus of claim 8 , wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more cloud-based actions comprise:

generating processing data for the plurality of trees, the processing data comprising at least one of a total number of trees for the plurality of trees, one or more tree gap counts, an average value of tree heights, a tree height for one or more of the plurality of trees, a canopy area estimation for one or more of the plurality of trees, a yield estimation of fruit for one or more of the plurality of trees, an estimation of tree ages for one or more of the plurality of trees, an estimation of tree health for one or more of the plurality of trees, estimated nutrient concentrations for one or more of the plurality of trees, or a fertility map based at least in part on estimate nutrient concentrations for the plurality of trees; and

providing at least a portion of the processing data for display via the interface on a user device.

13. The apparatus of claim 12 , wherein the one or more cloud-based actions comprise:

receiving an input indicating a selection of the region from a user via the user device; and

responsive to receiving the input indicating the selection of the region, providing a map of the region for display via the interface on the user device, the map displaying the plurality of trees detected for the region.

14. The apparatus of claim 8 , wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more cloud-based actions comprise:

generating an application map identifying an application rate for at least the region within the image map; and

downloading the application map to a smart sprayer system configured to use the application map to control flow of a liquid being applied to the plurality of trees in the region.

15. A non-transitory computer storage medium comprising instructions for detecting a plurality of objects located in an area of interest, the instructions being configured to cause one or more processors operating within a general instance of a cloud environment to at least perform operations configured to:

receive, via an interface, imaging data comprising a plurality of raw images collected on the area of interest;

upon receiving the plurality of raw images:

activate a central processing unit (CPU)-focused instance on the cloud environment based at least in part on a CPU-focused machine configuration image, wherein upon activating the CPU-focused instance, the CPU-focused instance processes the plurality of raw images to stitch together each of the raw images of the plurality of raw images to generate an image map of the area of interest; and

upon completion of generating the image map, close the CPU-focused instance on the cloud environment; and

after the image map is generated:

activate a graphical processing unit (GPU)-focused instance on the cloud environment based at least in part on a GPU-focused machine configuration image, wherein upon activating the GPU-focused instance, the GPU-focused instance performs object detection on at least a region within the image map by applying one or more object detection algorithms to the region of the image map to identify locations of the plurality of objects in the region of the image map; and

upon completion of detecting the plurality of objects for the region, close the GPU-focused instance on the cloud environment; and

perform one or more cloud-based actions based at least in part on the plurality of objects detected for the region within the image map.

16. The non-transitory computer storage medium of claim 15 , wherein the GPU-focused instance performs the object detection on the region within the image map by:

preprocessing the image map to reduce variation in the image map resulting from capturing the plurality of raw images;

applying a first object detection algorithm of the one or more object detection algorithms to identify initial locations of the plurality of objects in the region of the image map;

identifying one or more object patterns in the image map;

analyzing the one or more object patterns to identify one or more false positives in the initial locations of the plurality of objects;

removing the one or more false positives from the initial locations of the plurality of objects; and

after removing the one or more false positives from the initial locations of the plurality of objects, applying a second object detection algorithm of the one or more object detection algorithms to each of the one or more object patterns to identify the locations of the plurality of objects.

17. The non-transitory computer storage medium of claim 16 , wherein the area of interest comprises a tree grove, the plurality of objects comprises a plurality of trees, and the one or more object patterns comprise one or more rows of trees.

18. The non-transitory computer storage medium of claim 15 , wherein the GPU-focused instance performs the object detection of the region of the image map by receiving input, via the interface, originating from a user, the input comprising at least one of the region within the image map, an object spacing identifying an average space between objects located in the region, or one or more blank areas found in the region to be skipped by the one or more object detection algorithms.

19. The non-transitory computer storage medium of claim 15 , wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more cloud-based actions comprise:

generating processing data for the plurality of trees, the processing data comprising at least one of a total number of trees for the plurality of trees, one or more tree gap counts, an average value of tree heights, a tree height for one or more of the plurality of trees, a canopy area estimation for one or more of the plurality of trees, a yield estimation of fruit for one or more of the plurality of trees, an estimation of tree ages for one or more of the plurality of trees, an estimation of tree health for one or more of the plurality of trees, estimated nutrient concentrations for one or more of the plurality of trees, or a fertility map based at least in part on estimated nutrient concentrations for the plurality of trees; and

providing at least a portion of the processing data for display via the interface on a user device.

20. The non-transitory computer storage medium of claim 19 , wherein the one or more cloud-based actions comprise:

receiving an input indicating a selection of the region from a user via the user device; and

responsive to receiving the input indicating the selection of the region, providing a map of the region for display via the interface on the user device, the map displaying the plurality of trees detected for the region.

21. The non-transitory computer storage medium of claim 15 , wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more cloud-based actions comprise:

generating an application map identifying an application rate for at least the region within the image map; and

downloading the application map to a smart sprayer system configured to use the application map to control flow of a liquid being applied to the plurality of trees in the region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2022
From: AMPATZIDIS, IOANNIS; MEIRELLES PARTEL, VICTOR H.; COSTA, LUCAS FIDELES
To: UNIVERSITY OF FLORIDA RESEARCH FOUNDATION, INCORPORATED
Reel/Frame 061251/0419 →
Continuity (3)
Provisional Application 63199961 · Feb 5, 2021
Provisional Application 63013606 · Apr 22, 2020
Related Publication 20230124398A1 · Apr 20, 2023