Granted Patent B2

US 11,150,089 · App. 15/396,096 · Granted Oct 19, 2021

Inventors: Bernard J. Michini (San Francisco, CA); Brett Michael Bethke (Millbrae, CA); Hui Li (San Francisco, CA)

Assignee: Skydio, Inc.

G01C15/02G01C11/02B64C39/024B64C2201/123

Abstract

Claims (72)

1. A computerized method performed by two unmanned vehicles, the method comprising:

navigating a first unmanned vehicle within a first area during a first time period;

capturing, by the first unmanned vehicle, location information of the first unmanned vehicle while being navigated within the first area, using a first satellite navigation receiver;

storing in memory, by the first unmanned vehicle, location information of the first unmanned vehicle in association with timestamps indicating when a given item of location information of the first unmanned vehicle was determined;

navigating a second unmanned vehicle within the first area during at least a portion of the first time period, the second unmanned vehicle comprising an unmanned aerial vehicle, a camera, and a second satellite navigation receiver, the second satellite navigation receiver having lower precision and lower accuracy than the first satellite navigation receiver;

capturing images of the first area by the camera of the second unmanned vehicle, wherein one or more of the captured images are captured when the second unmanned vehicle is flying above the first unmanned vehicle and which include the first unmanned vehicle and one or more landmarks;

storing in memory, by the second unmanned vehicle, the captured images in association with corresponding location information and timestamps indicating when a given image was captured;

receiving from the first unmanned vehicle, at a first system, the location information of the first unmanned vehicle in association with the timestamps indicating when a given item of location information of the first unmanned vehicle was determined;

receiving from the second unmanned vehicle, at the first system, the captured images in association with corresponding location information and timestamps indicating when a given image was captured;

correlating, by the first system, a first captured image of the captured images that includes the first unmanned vehicle and a first landmark, with first location information of the first unmanned vehicle by using a time stamp associated with the first captured image and a time stamp associated with the first location information;

selecting the first landmark as a control point; and

storing an identification of the first landmark in association with the first location information.

2. The method of claim 1 , wherein selecting the first landmark as a control point comprises:

generating a score for the first landmark, the score based on one or more of the following criteria:

landmark dimensionality,

landmark shape, landmark contrast,

landmark movement,

landmark absolute size, and

landmark image size.

3. The method of claim 1 , wherein selecting the first landmark as a control point comprises:

determining if the first landmark corresponds to a preferred landmark type.

4. The method of claim 1 , wherein the second unmanned vehicle when navigated within the first area during at least a portion of the first time period, is on a mission unrelated to an identification of potential control points.

5. The method of claim 1 , wherein the first system, correlates the first captured image of the captured images that includes the first unmanned vehicle and the first landmark, with first location information of the first unmanned vehicle comprises by using location information associated with the first captured image.

6. The method of claim 1 , wherein the first unmanned vehicle is the same as the second unmanned vehicle navigated at separate times with different payload configurations.

7. A system comprising

one or more computer processors; and

one or more computer storage media storing instructions that when executed by the one or more computer processors cause the one or more computer processors to:

obtain a plurality of images captured by a first UAV positioned above a second UAV during implementation of a flight plan, each image including the second UAV and one or more landmarks representing fixed features of a real-world geographic area;

obtain time data associated with the capture of images by the first UAV, location information recorded by the second UAV while proximate to the one or more landmarks and time data associated with the recorded location information;

assign, for each image using time data associated with the capture of each image and the time data associated with the recorded location information, the second UAV included in the image as a ground control point associated with determined location information of the second UAV included in the image;

obtain a resulting image from one or more of the images and associated information describing the assigned ground control points, wherein one or more points included in the resulting image are associated with location information; and

determine location information of the landmarks included in the resulting image, and assigning the one or more landmarks as ground control points for use in subsequent flight plans.

8. The system of claim 7 , wherein the recorded location information has greater than a threshold accuracy, and wherein the threshold accuracy is 30 centimeters.

9. The system of claim 7 , wherein the location information is recorded, by the second UAV, using a Carrier-Phase Enhanced GNSS receiver.

10. The system of claim 7 , wherein the time data associated with capture of images by the first UAV is obtained from flight logs generated by the first UAV, and wherein the recorded location information and corresponding time data is obtained from flight logs generated by the second UAV.

11. The system of claim 7 , wherein obtaining a resulting image comprises:

providing, to an outside system, the images and the associated information describing the assigned ground control points; and

receiving, from the outside system, the resulting image.

12. The system of claim 11 , wherein the outside system is a photogrammetry system, and wherein the resulting image is a one or more of a geo-rectified image or a ortho-rectified image.

13. The system of claim 7 , wherein assigning the second UAV included in a particular image as a ground control point comprises:

determining, using the time data associated with the capture of the particular image and the time data associated with recorded location information, location information of the second UAV that is included in the particular image; and

assigning, for the particular image, the second UAV as a ground control point associated with the determined location information.

14. The system of claim 13 , wherein determining location information of the second UAV that is included in the particular image comprises:

determining estimated location information using recorded location information that is associated with time data within a threshold amount of time to the time data associated with the capture of the particular image.

15. A system comprising:

a first unmanned aerial vehicle comprising one or more processors;

a second unmanned aerial vehicle comprising one or more processors;

one or more computer processors; and

one or more computer storage media storing instructions that when executed by the one or more computer processors cause the one or more processors to:

navigate a first unmanned vehicle within a first area during a first time period;

capture, by the first unmanned vehicle, location information of the first unmanned vehicle while being navigated within the first area, using a high precision, high accuracy satellite navigation receiver;

store in memory, by the first unmanned vehicle, location information of the first unmanned vehicle in association with timestamps indicating when a given item of location information of the first unmanned vehicle was determined;

navigate a second unmanned vehicle within the first area during at least a portion of the first time period, the second unmanned vehicle comprising an unmanned aerial vehicle, a camera, and a satellite navigation receiver of lower precision and accuracy then the satellite navigation receiver of the first unmanned vehicle;

capture images of the first area by the camera of the second unmanned vehicle, wherein one or more of the captured images are captured when the second unmanned vehicle is flying above the first unmanned vehicle and which include the first unmanned vehicle and one or more landmarks;

store in memory, by the second unmanned vehicle, the captured images in association with corresponding location information and timestamps indicating when a given image was captured by the second unmanned vehicle;

receive from the first unmanned vehicle, at a first system, the location information of the first unmanned vehicle in association with the timestamps indicating when a given item of location information of the first unmanned vehicle was determined;

receive from the second unmanned vehicle, at the first system, the captured images in association with corresponding location information and timestamps indicating when a given image was captured;

correlate, by the first system, a first captured image of the captured images that includes the first unmanned vehicle and a first landmark, with first location information of the first unmanned vehicle by using a time stamp associated with the first captured image and a time stamp associated with the first location information;

select the first landmark as a control point; and

store an identification of the first landmark in association with the first location information.

16. The system of claim 15 , wherein selecting the first landmark as a control point comprises:

generating a score for the first landmark, the score based on one or more of the following criteria:

landmark dimensionality,

landmark shape, landmark contrast,

landmark movement,

landmark absolute size, and

landmark image size.

17. The system of claim 15 , wherein selecting the first landmark as a control point comprises:

determining if the first landmark corresponds to a preferred landmark type.

18. The system of claim 15 , wherein the second unmanned vehicle when navigated within the first area during at least a portion of the first time period, is on a mission unrelated to an identification of potential control points.

19. The system of claim 15 , wherein the first system correlates the first captured image of the captured images that includes the first unmanned vehicle and the first landmark, with first location information of the first unmanned vehicle comprises by using location information associated with the first captured image.

20. The system of claim 15 , wherein the first unmanned vehicle is the same as the second unmanned vehicle navigated at separate times with different payload configurations.

Assignments (11)

Continuity (2)

Cited By (3)

US 12,276,500 US 12,276,501 US 12,504,454