IP Library Granted Patent US 9,928,659
Granted Patent B2
US 9,928,659 · App. 15/101,081 · Granted Mar 27, 2018

Method and system for generating augmented reality agricultural presentations

Inventors: William Christopher Dean (Raleigh, NC); Ernest Earon (Thornhill, CA); Seyed Ali Shahdi (Whitby, CA); Patrick Neal Lohman (Noblesville, IN); Andrew Thomas Slater (Raleigh, NC); Greer Monterastelli (Buffalo, WY)
Assignee: Precisionhawk Inc.
G06T19/006G01C11/025G06T11/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,928,659
App. No.
15/101,081
Granted
Mar 27, 2018
Kind
B2
Abstract

A method for generating an augmented reality presentation of a region-of-interest, comprising: capturing raw image data of the region-of-interest using a first sensor mounted in an unmanned aerial vehicle flown over the region-of-interest; transferring the raw image data to a server from the unmanned aerial vehicle; using a processor, generating geo-referenced data from the raw image data at the server; transferring the geo-referenced data to a wireless device; capturing live image data of the region-of-interest using a second sensor mounted in the wireless device; and, overlaying the geo-referenced data on the live image data to generate the augmented reality presentation at the wireless device.

Claims (48)

1. A method for generating an augmented reality presentation of a region-of-interest, comprising:

capturing raw image data of the region-of-interest using a first sensor mounted in an unmanned aerial vehicle flown over the region-of-interest;

transferring the raw image data to a server from the unmanned aerial vehicle;

using a processor, generating geo-referenced data from the raw image data at the server;

transferring the geo-referenced data to a wireless device;

capturing live image data of the region-of-interest using a second sensor mounted in the wireless device; and,

overlaying the geo-referenced data on the live image data to generate the augmented reality presentation at the wireless device.

2. The method of claim 1 and further comprising presenting the augmented reality presentation on a display of the wireless device.

3. The method of claim 1 wherein the first and second sensors are first and second cameras, respectively.

4. The method of claim 1 wherein the region-of-interest is one of a farm field and located in a farm field.

5. A method comprising:

accessing, by a mobile computing device comprising a processor device, geo-referenced imagery of a region of a world, the geo-referenced imagery corresponding to specific locations of the region of the world;

receiving, by a camera of the mobile computing device, real-time real-world imagery of a first scene of a portion of the region of the world that is within a field of view (FOV) of the camera;

determining, by the mobile computing device, a first subset of the geo-referenced imagery that corresponds to the first scene; and

presenting the first subset of the geo-referenced imagery that corresponds to the first scene overlaid on top of the real-world imagery of the first scene on a display device of the mobile computing device.

6. The method of claim 5 wherein the first subset of the geo-referenced imagery comprises textual imagery that refers to specific locations within the first scene.

7. The method of claim 5 wherein the geo-referenced imagery comprises a normalized difference vegetation index (“NDVI”) imagery.

8. The method of claim 5 wherein determining the first subset of the geo-referenced imagery that corresponds to the first scene further comprises:

determining an orientation and location of the FOV of the camera with respect to the world; and

determining the first subset of the geo-referenced imagery that corresponds to the first scene based on the orientation and location of the FOV of the camera with respect to the world.

9. The method of claim 5 wherein the first subset of the geo-referenced imagery comprises imagery comprising colors, each different color corresponding to a health of vegetation in the first scene, and each different color overlaying a portion of the first scene having vegetation with the health that corresponds to the different color.

10. The method of claim 5 further comprising:

determining, by the mobile computing device, that the camera is receiving real-time real-world imagery of a second scene of the portion of the region of the world;

determining, by the mobile computing device, a second subset of the geo-referenced imagery that corresponds to the second scene; and

presenting the second subset of the geo-referenced imagery that corresponds to the second scene overlaid on top of the real-world imagery of the second scene on the display device of the mobile computing device.

11. The method of claim 10 wherein determining, by the mobile computing device, that the camera is receiving the real-time real-world imagery of the second scene of the world further comprises determining that an orientation of the camera has moved from a first orientation to a second orientation.

12. A mobile computing device comprising:

a camera having a field of view (FOV), the camera configured to receive real-time real-world imagery of a first scene of a portion of a region of a world within the FOV of the camera;

a display device; and

a processing device coupled to the camera and the display device, the processing device configured to:

access geo-referenced imagery of the region of the world, the geo-referenced imagery comprising imagery that corresponds to specific locations of the region of the world;

determine a first subset of the geo-referenced imagery that corresponds to the first scene; and

present the first subset of the geo-referenced imagery that corresponds to the first scene overlaid on top of the real-time real-world imagery of the first scene on the display device.

13. The mobile computing device of claim 12 wherein the first subset of the geo-referenced imagery comprises textual imagery that refers to specific locations within the first scene.

14. The mobile computing device of claim 12 , wherein to determine the first subset of the geo-referenced imagery that corresponds to the first scene, the processor device is further configured to:

determine an orientation and location of the FOV of the camera with respect to the world; and

determine the first subset of the geo-referenced imagery that corresponds to the first scene based on the orientation and location of the FOV of the camera with respect to the world.

15. The mobile computing device of claim 12 wherein the first subset of the geo-referenced imagery comprises imagery comprising colors, each different color corresponding to a health of vegetation in the first scene, and each different color overlaying a portion of the first scene having vegetation with the health that corresponds to the different color.

16. The mobile computing device of claim 12 wherein the processor device is further configured to:

determine that the camera is receiving real-time real-world imagery of a second scene of the world;

determine a second subset of the geo-referenced imagery that corresponds to the second scene; and

present the second subset of the geo-referenced imagery that corresponds to the second scene overlaid on top of the real-time real-world imagery of the second scene on the display device.

17. The mobile computing device of claim 16 wherein to determine that the camera is receiving the real-time real-world imagery of the second scene of the world, the processor device is further configured to determine that an orientation of the camera has moved from a first orientation to a second orientation.

18. A computer program product stored on a non-transitory computer-readable storage medium and including instructions configured to cause a processor device to:

access imagery received in real-time by a camera, the imagery comprising real-world imagery of a first scene of a portion of a region of a world within a field of view (FOV) of the camera;

access geo-referenced imagery of the region of the world, the geo-referenced imagery comprising imagery that corresponds to specific locations of the region of the world;

determine a first subset of the geo-referenced imagery that corresponds to the first scene; and

present the first subset of the geo-referenced imagery that corresponds to the first scene overlaid on top of the real-world imagery of the first scene on a display device of a mobile computing device.

Assignments (4)
SECURITY INTEREST Recorded Dec 11, 2020
From: PRECISIONHAWK CANADA INC.
To: EASTWARD FUND MANAGEMENT, LLC
Reel/Frame 054613/0724 →
SECURITY INTEREST Recorded Jul 6, 2020
From: PRECISIONHAWK CANADA INC.
To: EASTWARD FUND MANAGEMENT, LLC
Reel/Frame 053123/0702 →
CHANGE OF NAME Recorded Jun 3, 2020
From: PRECISIONHAWK INC.
To: PRECISIONHAWK CANADA INC.
Reel/Frame 052819/0778 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2016
From: DEAN, WILLIAM CHRISTOPHER; EARON, ERNEST; SHAHDI, SEYED ALI; LOHMAN, PATRICK NEAL; SLATER, ANDREW THOMAS; MONTERASTELLI, GREER
To: PRECISIONHAWK INC.
Reel/Frame 038780/0080 →
Continuity (2)
Provisional Application 61925045 · Jan 8, 2014
Related Publication 20160307373A1 · Oct 20, 2016