IP Library Granted Patent US 10,466,695
Granted Patent B2
US 10,466,695 · App. 15/791,230 · Granted Nov 5, 2019

User interaction paradigms for a flying digital assistant

Inventors: Abraham Bachrach (San Francisco, CA); Adam Bry (San Mateo, CA); Matthew Donahoe (Burlingame, CA)
Assignee: SKYDIO, INC.
G05D1/0038B64C39/00B64C39/024G05D1/0016G05D1/0044G05D1/0094G06F3/00G06F3/04883H04N5/00B64C2201/027B64C2201/042B64C2201/127B64C2201/141G06F2203/04808
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,466,695
App. No.
15/791,230
Granted
Nov 5, 2019
Kind
B2
Abstract

Methods and systems are described for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone. In some embodiments, a user may control image capture from an FDA by adjusting the position and orientation of a PMD. In other embodiments, a user may input a touch gesture via a touch display of a PMD that corresponds with a flight path to be autonomously flown by the FDA.

Claims (50)

1. An unmanned aerial vehicle (UAV) configured to capture aerial video of multiple subjects in a physical environment, the UAV comprising:

an image capture device;

a wireless network interface configured for wireless communication with a plurality of mobile devices; and

a computer processor configured to:

receive, via the wireless network interface, a first image capture request from a first mobile device of the plurality of mobile devices;

determine, in response to the image capture request, a first position of the first mobile device relative to the UAV based at least in part on images of the physical environment received via the image capture device;

generate control commands configured to cause the UAV to maneuver such that the image capture device tracks the first position of the first mobile device;

receive, via the wireless network interface, a second image capture request from a second mobile device of the plurality of mobile devices, the second mobile device different than the first mobile device;

determine, in response to the image capture request, a second position of the second mobile device relative to the UAV based at least in part on images of the physical environment received via the image capture device; and

generate control commands configured to cause the UAV to maneuver such that the image capture device tracks the position of the second mobile device.

2. The UAV of claim 1 , wherein the computer processor is further configured to:

receive, via the wireless network interface, a third image capture request designating a third mobile device of the plurality of mobile devices, the third image capture request received from a fourth mobile device of the plurality of mobile devices;

determine, in response to the third image capture request, a third position of the third mobile device relative to the UAV based at least in part on images of the physical environment received via the image captured device; and

generate control commands configured to cause the UAV to maneuver such that the image capture device tracks the third position of the third mobile device.

3. The UAV of claim 1 , wherein the computer processor is further configured to:

authenticate the first image capture request from the first mobile device before generating control commands configured to cause the UAV to maneuver.

4. The UAV of claim 3 , wherein authenticating the first image capture request includes confirming a valid payment associated with the first image capture request.

5. The UAV of claim 4 , wherein the valid payment is any of a one-time fee or a subscription.

6. The UAV of claim 1 , wherein each of the plurality of mobile devices are associated with a different one of a plurality of users attending an event at a physical venue.

7. The UAV of claim 6 , wherein the computer processor is further configured to:

generate control commands configured to cause the UAV to hover over the physical venue until an image captured request is received from one of the plurality of mobile devices.

8. The UAV of claim 1 , wherein the computer processor is further configured to:

generate control commands configured to cause the UAV to maneuver such that the image capture device is no longer tracking the first position of the first mobile device after a period of time has elapsed from when the first image capture request was received from the first mobile device.

9. The UAV of claim 1 , wherein the wireless network interface is further configured to transmit a live video stream, captured by the image capture device, to the first mobile device as the image capture device tracks the position of the first mobile device.

10. The UAV of claim 9 , wherein the live video stream is displayed at a display of the first mobile device.

11. The UAV of claim 1 , wherein the image capture device is coupled to the UAV by a gimbal device configured to adjust an orientation of image capture device relative to the UAV.

12. The UAV of claim 11 , wherein the gimbal device is part of a hybrid mechanical-digital gimbal system.

13. The UAV of claim 12 , wherein the hybrid mechanical-digital gimbal system includes:

a mechanical gimbal configured to rotate the image capture device about a first axis relative to the UAV; and

an image processing system configured to rotate images captured by the image capture device about a second axis relative to the UAV.

14. A method for capturing aerial video of multiple subjects in a physical environment by an unmanned aerial vehicle (UAV), the UAV including an image capture device, the method comprising:

receiving, by the UAV, a first wireless communication including a first image capture request from a first mobile device of a plurality of mobile devices;

determining, by the UAV, in response to the first image capture request, a first position of the first mobile device relative to the UAV based at least in part on images of the physical environment received via the image capture device;

autonomously maneuvering, by the UAV, such that the image capture device tracks the first position of the first mobile device;

receiving, by the UAV, a second wireless communication including a second image capture request from a second mobile device of the plurality of mobile devices, the second mobile device different than the first mobile device;

determining, by the UAV, in response to the second image capture request, a second position of the second mobile device relative to the UAV based at least in part on images of the physical environment received via the image capture device; and

autonomously maneuvering, by the UAV, such that the image capture device tracks the second position of the second mobile device.

15. The method of claim 14 , further comprising:

transmitting, by the UAV, a live video stream captured by the image capture device, to the first mobile device.

16. The method of claim 14 , further comprising:

authenticating, by the UAV, the first image capture request before autonomously maneuvering such that the image capture device tracks the first position of the first mobile device.

17. The method of claim 14 , further comprising:

receiving, by the UAV, a third wireless communication including a third image capture request designating a third mobile device of the plurality of mobile devices, the third image capture request received from a fourth mobile device of the plurality of mobile devices;

determining, by the UAV, in response to the third image capture request, a third position of the third mobile device relative to the UAV based at least in part on images of the physical environment received via the image capture device; and

autonomously maneuvering, by the UAV, such that the image capture device tracks the third position of the third mobile device.

18. The method of claim 14 , further comprising:

autonomously maneuvering, by the UAV, such that the image capture device is no longer tracking the first position of the first mobile device after a period of time has elapsed from when the first image capture request was received from the first mobile device.

19. The method of claim 14 , wherein each of the plurality of mobile devices are associated with a different one of a plurality of users attending an event at a physical venue.

20. The method of claim 19 , further comprising:

autonomously hovering, by the UAV, over the physical venue until an image captured request is received from one of the plurality of mobile devices.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY
To: SKYDIO, INC.
Reel/Frame 072107/0066 →
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2025
From: HERCULES CAPITAL, INC.
To: SKYDIO, INC.
Reel/Frame 072128/0698 →
SECURITY INTEREST Recorded Dec 5, 2024
From: SKYDIO, INC.
To: ACQUIOM AGENCY SERVICES LLC
Reel/Frame 069516/0452 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Nov 9, 2021
From: SKYDIO, INC.
To: HERCULES CAPITAL, INC., AS COLLATERAL AND ADMINISTRATIVE AGENT
Reel/Frame 058081/0677 →
SECURITY INTEREST Recorded Nov 8, 2021
From: SKYDIO, INC.
To: SILICON VALLEY BANK
Reel/Frame 058053/0768 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2018
From: BACHRACH, ABRAHAM; BRY, ADAM; DONAHOE, MATTHEW
To: SKYDIO, INC.
Reel/Frame 046807/0940 →
Continuity (4)
Continuation 14689008 · Apr 16, 2015
Provisional Application 62140413 · Mar 30, 2015
Provisional Application 62014650 · Jun 19, 2014
Related Publication 20180095459A1 · Apr 5, 2018
Cited By (1)
US 12,197,236