IP Library Granted Patent US 11,827,352
Granted Patent B2
US 11,827,352 · App. 17/314,885 · Granted Nov 28, 2023

Visual observer for unmanned aerial vehicles

Inventors: Eric Miller (Granville, OH); Jacob Daniel Phillips (Boston, MA)
Assignee: SKYDIO, INC.
B64C39/024B64D47/08G05D1/0094G05D1/1064G05D1/12G06N3/08G06V10/25G06V10/764G06V10/80G06V10/82G06V20/17G06V20/52H04N23/695B64U2101/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,827,352
App. No.
17/314,885
Granted
Nov 28, 2023
Kind
B2
Abstract

In some examples, a device may receive, from a first camera, a plurality of images of an airspace corresponding to an area of operation of an unmanned aerial vehicle (UAV). The device may detect, based on the plurality of images from the first camera, a candidate object approaching or within the airspace. Based on detecting the candidate object, the device may control a second camera to direct a field of view of the second camera toward the candidate object. Further, based on images from the second camera captured at a first location and images from at least one other camera captured at a second location, the candidate object may be determined to be an object of interest. In addition, at least one action may be taken based on determining that the candidate object is the object of interest.

Claims (52)

1. A system comprising:

a first device including a plurality of cameras and at least one processor, the at least one processor configured by executable instructions to:

receive, from a first camera of the plurality of cameras, a plurality of images of an airspace corresponding to an area of operation of a first unmanned aerial vehicle (UAV);

detect, based on the plurality of images from the first camera, using at least a first computer vision algorithm, a candidate object approaching or within the airspace;

determine, based on a second computer vision algorithm, an indication of whether an output of the first computer vision algorithm is correct;

based at least on detecting the candidate object and determining that the output of the first computer vision algorithm is indicated to be correct, control movement of a movable camera of the plurality of cameras to direct a field of view of the movable camera toward the candidate object;

receive images from the movable camera captured at a first location of the first device, and receiving images from a second camera captured at a second location of a second device;

determine, based on the images from the first movable camera and the images from the second camera that the candidate object is an object of interest; and

perform at least one action based on determining that the candidate object is the object of interest.

2. The system as recited in claim 1 , the at least one processor further configured by the executable instructions to perform, as the at least one action, at least one of:

sending an instruction to cause the first UAV to perform an action based on the object of interest; or

sending a notification to a computing device to provide information related to the object of interest.

3. The system as recited in claim 1 , the at least one processor further configured by the executable instructions to use a third computer vision algorithm to determine, based on the images from the movable camera and the images from the second camera at the second location, that the candidate object is an object of interest.

4. The system as recited in claim 1 , wherein the first device includes a plurality of radially arranged fixed cameras including the first camera, and wherein the movable camera is mounted between fields of view of two of the radially arranged fixed cameras.

5. The system as recited in claim 1 , the at least one processor further configured by the executable instructions to:

based on detecting the candidate object, send, to the second device, at least an indication of a location of the candidate object; and

receive, at least partially in response to sending the indication of the location, the images from the second camera.

6. The system as recited in claim 1 , wherein the movable camera has a level of optical magnification that is greater than an optical magnification of the first camera.

7. A method comprising:

receiving, by one or more processors, from a first camera, a plurality of images of an airspace corresponding to an area of operation of a first unmanned aerial vehicle (UAV);

detecting, using a first computer vision algorithm, and based on the plurality of images from the first camera, a candidate object approaching or within the airspace;

determining, based on a second computer vision algorithm, an indication of whether an output of the first computer vision algorithm is correct;

based on detecting the candidate object and determining that the output of the first computer vision algorithm is indicated to be correct, controlling a second camera to direct a field of view of the second camera toward the candidate object;

determining, based on images from the second camera captured at a first location and images from at least one other camera captured at a second location that the candidate object is an object of interest; and

performing at least one action based on determining that the candidate object is the object of interest.

8. The method as recited in claim 7 , wherein performing the at least one action includes at least one of:

sending an instruction to cause the first UAV to take an action based on the object of interest; or

sending a notification to a computing device to provide information related to the object of interest.

9. The method as recited in claim 7 , further comprising using a third computer vision algorithm to determine, based on images from the second camera and images from at least one other camera at the second location that the candidate object is an object of interest.

10. The method as recited in claim 7 , wherein the determining, based on the images from the second camera captured at the first location and the images from the at least one other camera captured at the second location that the candidate object is the object of interest is performed by a processor onboard the first UAV.

11. The method as recited in claim 7 , wherein the determining, based on the images from the second camera captured at the first location and the images from the at least one other camera captured at the second location that the candidate object is the object of interest is performed by a computing device located over a network from at least one of the second camera or the other camera.

12. The method as recited in claim 7 , further comprising:

receiving information related to a communication broadcasted by the object of interest, the communication indicating a location of the object of interest; and

determining that the candidate object is the object of interest based partially on the communication.

13. The method as recited in claim 7 , wherein at least one of the second camera or the other camera is mounted on a second UAV.

14. The method as recited in claim 7 , wherein the second camera has a level of optical magnification that is greater than an optical magnification of the first camera.

15. An apparatus comprising:

a plurality of cameras and at least one processor configured by executable instructions to perform operations comprising:

receiving, from a first camera of the plurality of cameras, a plurality of images of an airspace corresponding to an area of operation of a first unmanned aerial vehicle (UAV);

detecting, using a first computer vision algorithm, and based on the plurality of images from the first camera, a candidate object approaching or within the airspace;

determining, based on a second computer vision algorithm, an indication of whether an output of the first computer vision algorithm is correct;

based on detecting the candidate object and determining that the output of the first computer vision algorithm is indicated to be correct, controlling movement of a movable camera of the plurality of cameras to direct a field of view of the movable camera toward the candidate object; and

sending images captured using the movable camera at a first location to at least one of the first UAV or a computing device located over a network, the at least one of the first UAV or the computing device configured to perform recognition on the sent images.

16. The apparatus as recited in claim 15 , the operations further comprising sending an indication of a location of the candidate object to another apparatus, the sending causing, at least in part, the other apparatus to direct a field of view of a movable camera of the other apparatus toward the candidate object.

17. The apparatus as recited in claim 15 , the operations further comprising:

receiving information related to a communication broadcasted by the object of interest, the communication indicating a location of the object of interest; and

determining that the candidate object is the object of interest based partially on the communication.

18. The apparatus as recited in claim 15 , wherein:

the plurality of cameras comprises a plurality of radially arranged cameras including the first camera; and

the movable camera is mounted between fields of view of two of the radially arranged cameras.

19. The apparatus as recited in claim 15 , wherein the apparatus is mounted on a second UAV.

20. The apparatus as recited in claim 15 , wherein the movable camera has a level of optical magnification that is greater than an optical magnification of the first camera.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY
To: SKYDIO, INC.
Reel/Frame 072107/0066 →
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: HERCULES CAPITAL, INC.
To: SKYDIO, INC.
Reel/Frame 072128/0698 →
SECURITY INTEREST Recorded Dec 5, 2024
From: SKYDIO, INC.
To: ACQUIOM AGENCY SERVICES LLC
Reel/Frame 069516/0452 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Nov 9, 2021
From: SKYDIO, INC.
To: HERCULES CAPITAL, INC., AS COLLATERAL AND ADMINISTRATIVE AGENT
Reel/Frame 058081/0677 →
SECURITY INTEREST Recorded Nov 8, 2021
From: SKYDIO, INC.
To: SILICON VALLEY BANK
Reel/Frame 058053/0768 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2021
From: MILLER, ERIC; PHILLIPS, JACOB DANIEL
To: SKYDIO, INC.
Reel/Frame 056283/0330 →
Continuity (2)
Provisional Application 63021329 · May 7, 2020
Related Publication 20210350162A1 · Nov 11, 2021