IP Library Granted Patent US 12,344,409
Granted Patent B2
US 12,344,409 · App. 17/981,807 · Granted Jul 1, 2025

Fully autonomous drone flights

Inventors: Mathias Andreas Hintermann (Playa Vista, CA); Justin Huang (Los Angeles, CA); Russell Douglas Patton (Playa Vista, CA)
Assignee: Snap Inc.
B64U30/20G05D1/101G06V10/82G06V20/17G06V40/107G06V40/172B64U2101/30B64U2201/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,344,409
App. No.
17/981,807
Granted
Jul 1, 2025
Kind
B2
Abstract

Systems, computer readable medium and methods for fully autonomous drone flight are disclosed. Example methods include taking off, navigating in accordance with a flight plan, and navigating the autonomous drone to land. The autonomous drone performs flight plans with only an initial command for the autonomous drone to fly and, in some examples, an indication of a landing space such as an open hand presented under the autonomous drone. After an initial fly command, the autonomous drone is not controlled by a remote-control device and does not receive any additional commands to complete the flight plan. The autonomous drone enters a lower energy state while flying where the wireless connections are turned off since the autonomous drone does not respond to commands during flight.

Claims (63)

1. An apparatus of an autonomous drone comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, configure the at least one processor to perform operations comprising:

capturing an image using a first image capturing device of the autonomous drone;

processing the image to identify a face of a user;

taking off from a hand of the user;

capturing a second image using a second image capturing device, the second image capturing device being directed downward;

processing the second image to identify the hand;

navigating in accordance with a flight plan; and

navigating the autonomous drone to land on the hand.

2. The apparatus of claim 1 , wherein the first image capturing device is mounted horizontally relative to an axis of propellers of the autonomous drone and the second image capturing device is mounted vertically relative to the axis of the propellers.

3. The apparatus of claim 1 , wherein the navigating the autonomous drone to land further comprises:

causing autonomous drone to hover in a front of a person associated with the face;

identifying an open hand of the person; and

navigating the autonomous drone to land on the open hand.

4. The apparatus of claim 1 , wherein the operations further comprise:

capturing a video while navigating the flight plan, and wherein the navigating in accordance with the flight plan further comprises directing the image capturing device towards a person associated with the face.

5. The apparatus of claim 1 , wherein the operations further comprise:

before the capturing the image, receiving an indication of a selection of a fly instruction.

6. The apparatus of claim 5 wherein the operations further comprise:

determining that no additional instructions from a remote device are received prior to navigating the autonomous drone to land.

7. The apparatus of claim 6 , wherein the fly instruction is a selection of a flight plan via an input device on the autonomous drone to select a flight plan, wherein the input device is a knob or a flight button.

8. The apparatus of claim 1 wherein the operations further comprise:

determining the flight plan based on a position of the face relative to the autonomous drone.

9. The apparatus of claim 1 wherein the operations further comprise:

after the processing the image, taking off in a space between a palm of a hand associated with the face and the face.

10. The apparatus of claim 1 wherein the operations further comprise:

determining the flight plan based on a setting of a user interface item accessible on an outside of the autonomous drone.

11. The apparatus of claim 1 , wherein the flight plan is based on identifying the face or another object in subsequent images captured by the first image capturing device or the second image capturing device, or is based on determining a location of the autonomous drone in space.

12. The apparatus of claim 1 wherein the operations further comprise:

in response to the navigating in accordance with the flight plan, turning off all wireless connections to the autonomous drone.

13. The apparatus of claim 1 wherein the operations further comprise:

after the processing the image, determining that no additional instructions are received prior to navigating the autonomous drone to land.

14. The apparatus of claim 1 wherein the flight plan indicates a target distance from a person associated with the face and a target height of the autonomous drone above a ground, and wherein the operations further comprise:

identifying the face using a neural network;

uploading an image of the face to a server; and

downloading weights from the server for the neural network to identify the face.

15. A method performed on an apparatus of an autonomous drone, the method comprising:

capturing an image using a first image capturing device of the autonomous drone;

processing the image to identify a face of a user;

taking off from a hand of the user;

capturing a second image using a second image capturing device, the second image capturing device being directed downward;

processing the second image to identify the hand;

navigating in accordance with a flight plan; and

navigating the autonomous drone to land on the hand.

16. The method of claim 15 further comprising:

before the capturing the image, receiving an indication of a selection of a fly instruction; and

determining that no additional instructions from a remote device are received prior to navigating the autonomous drone to land.

17. The method of claim 15 wherein the method further comprises:

in response to the navigating in accordance with the flight plan, turning off all wireless connections to the autonomous drone.

18. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of an apparatus of an autonomous drone, cause the at least one processor to perform operations comprising:

capturing an image using a first image capturing device of the autonomous drone;

processing the image to identify a face of a user;

taking off from a hand of the user;

capturing a second image using a second image capturing device, the second image capturing device being directed downward;

processing the second image to identify the hand;

navigating in accordance with a flight plan; and

navigating the autonomous drone to land on the hand.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

before the capturing the image, receiving an indication of a selection of a fly instruction; and

determining that no additional instructions are received prior to navigating the autonomous drone to land.

20. The non-transitory computer-readable storage medium of claim 18 wherein the operations further comprise:

in response to the navigating in accordance with the flight plan, turning off all wireless connections to the autonomous drone.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2022
From: HINTERMANN, MATHIAS ANDREAS; HUANG, JUSTIN; PATTON, RUSSELL DOUGLAS
To: SNAP INC.
Reel/Frame 061675/0327 →
Continuity (2)
Provisional Application 63335429 · Apr 27, 2022
Related Publication 20230350431A1 · Nov 2, 2023
References Cited (145)
US 6038295A · Mattes · 2000 [cited by applicant]
US 6819982B2 · Doane · 2004 [cited by applicant]
US 6980909B2 · Root et al. · 2005 [cited by applicant]
US 7173651B1 · Knowles · 2007 [cited by applicant]
US 7411493B2 · Smith · 2008 [cited by applicant]
US 7535890B2 · Rojas · 2009 [cited by applicant]
US 7542073B2 · Li et al. · 2009 [cited by applicant]
US 7971156B2 · Albertson et al. · 2011 [cited by applicant]
US 8131597B2 · Hudetz · 2012 [cited by applicant]
US 8174562B2 · Hartman · 2012 [cited by applicant]
US 8199747B2 · Rojas et al. · 2012 [cited by applicant]
US 8274550B2 · Steuart, III · 2012 [cited by applicant]
US 8332475B2 · Rosen et al. · 2012 [cited by applicant]
US 8718333B2 · Wolf et al. · 2014 [cited by applicant]
US 8724622B2 · Rojas · 2014 [cited by applicant]
US 8874677B2 · Rosen et al. · 2014 [cited by applicant]
US 8909679B2 · Root et al. · 2014 [cited by applicant]
US 8995433B2 · Rojas · 2015 [cited by applicant]
US 9040574B2 · Wang et al. · 2015 [cited by applicant]
US 9055416B2 · Rosen et al. · 2015 [cited by applicant]
US 9100806B2 · Rosen et al. · 2015 [cited by applicant]
US 9100807B2 · Rosen et al. · 2015 [cited by applicant]
US 9191776B2 · Root et al. · 2015 [cited by applicant]
US 9204252B2 · Root · 2015 [cited by applicant]
US 9225897B1 · Sehn et al. · 2015 [cited by applicant]
US 9276886B1 · Samaranayake · 2016 [cited by applicant]
US 9344642B2 · Niemi et al. · 2016 [cited by applicant]
US 9345711B2 · Friedhoff · 2016 [cited by applicant]
US 9443227B2 · Evans et al. · 2016 [cited by applicant]
US 9471059B1 · Wilkins · 2016 [cited by applicant]
US 9489661B2 · Evans et al. · 2016 [cited by applicant]
US 9489937B1 · Beard et al. · 2016 [cited by applicant]
US 9491134B2 · Rosen et al. · 2016 [cited by applicant]
US 9576369B2 · Venkataraman et al. · 2017 [cited by applicant]
US 9589448B1 · Schneider et al. · 2017 [cited by applicant]
US 9681046B2 · Adsumilli et al. · 2017 [cited by applicant]
US 9705831B2 · Spiegel · 2017 [cited by applicant]
US 9723272B2 · Lu et al. · 2017 [cited by applicant]
US 9742713B2 · Spiegel et al. · 2017 [cited by applicant]
US 9747901B1 · Gentry · 2017 [cited by applicant]
US 9922659B2 · Bradlow et al. · 2018 [cited by applicant]
US 9989965B2 · Cuban et al. · 2018 [cited by applicant]
US 10061328B2 · Canoy et al. · 2018 [cited by applicant]
US 10102423B2 · Shaburov et al. · 2018 [cited by applicant]
US 10140987B2 · Erickson et al. · 2018 [cited by applicant]
US 10168700B2 · Gordon et al. · 2019 [cited by applicant]
US 10284508B1 · Allen et al. · 2019 [cited by applicant]
US 10439972B1 · Spiegel et al. · 2019 [cited by applicant]
US 10501180B2 · Yu · 2019 [cited by applicant]
US 10509466B1 · Miller et al. · 2019 [cited by applicant]
US 10514876B2 · Sehn · 2019 [cited by applicant]
US 10614855B2 · Huang · 2020 [cited by applicant]
US 10748347B1 · Li et al. · 2020 [cited by applicant]
US 10768639B1 · Meisenholder et al. · 2020 [cited by applicant]
US 10958608B1 · Allen et al. · 2021 [cited by applicant]
US 10962809B1 · Castañeda · 2021 [cited by applicant]
US 10996846B2 · Robertson et al. · 2021 [cited by applicant]
US 10997787B2 · Ge et al. · 2021 [cited by applicant]
US 11012390B1 · Al Majid et al. · 2021 [cited by applicant]
US 11030454B1 · Xiong et al. · 2021 [cited by applicant]
US 11036368B1 · Al Majid et al. · 2021 [cited by applicant]
US 11062498B1 · Voss et al. · 2021 [cited by applicant]
US 11087728B1 · Canberk et al. · 2021 [cited by applicant]
US 11092998B1 · Castañeda et al. · 2021 [cited by applicant]
US 11106342B1 · Al Majid et al. · 2021 [cited by applicant]
US 11126206B2 · Meisenholder · 2021 [cited by examiner]
US 11143867B2 · Rodriguez, II · 2021 [cited by applicant]
US 11169600B1 · Canberk et al. · 2021 [cited by applicant]
US 11227626B1 · Krishnan Gorumkonda et al. · 2022 [cited by applicant]
US 11348265B1 · Nielsen · 2022 [cited by examiner]
US 11404056B1 · Meisenholder · 2022 [cited by examiner]
US 11435743B2 · Sun · 2022 [cited by examiner]
US 11531357B1 · Nielsen · 2022 [cited by examiner]
US 11822346B1 · Huang · 2023 [cited by examiner]
US 11892859B2 · Meisenholder · 2024 [cited by examiner]
US 12002168B2 · Bajana · 2024 [cited by examiner]
US 12071228B1 · Nielsen · 2024 [cited by examiner]
US 12073577B2 · Nielsen · 2024 [cited by examiner]
US 12093443B1 · Krishnan Gorumkonda · 2024 [cited by examiner]
US 20070250526A1 · Hanna · 2007 [cited by applicant]
US 20080255842A1 · Simhi · 2008 [cited by applicant]
US 20090122133A1 · Hartman · 2009 [cited by applicant]
US 20110202598A1 · Evans et al. · 2011 [cited by applicant]
US 20120194420A1 · Osterhout et al. · 2012 [cited by applicant]
US 20120209924A1 · Evans et al. · 2012 [cited by applicant]
US 20120281885A1 · Syrdal et al. · 2012 [cited by applicant]
US 20120287274A1 · Bevirt · 2012 [cited by applicant]
US 20130056581A1 · Sparks · 2013 [cited by applicant]
US 20130238168A1 · Reyes · 2013 [cited by applicant]
US 20140254896A1 · Zhou et al. · 2014 [cited by applicant]
US 20150022432A1 · Stewart et al. · 2015 [cited by applicant]
US 20150070272A1 · Kim et al. · 2015 [cited by applicant]
US 20150175263A1 · Reyes · 2015 [cited by applicant]
US 20150199022A1 · Gottesman et al. · 2015 [cited by applicant]
US 20150287246A1 · Huston et al. · 2015 [cited by applicant]
US 20150331490A1 · Yamada · 2015 [cited by applicant]
US 20150362917A1 · Wang et al. · 2015 [cited by applicant]
US 20160063987A1 · Xu et al. · 2016 [cited by applicant]
US 20160161946A1 · Wuth Sepulveda et al. · 2016 [cited by applicant]
US 20160179096A1 · Bradlow et al. · 2016 [cited by applicant]
US 20160292886A1 · Erad et al. · 2016 [cited by applicant]
US 20160307573A1 · Wobrock · 2016 [cited by applicant]
US 20160336020A1 · Bradlow et al. · 2016 [cited by applicant]
US 20170031369A1 · Liu et al. · 2017 [cited by applicant]
US 20170094259A1 · Kouperman et al. · 2017 [cited by applicant]
US 20170099424A1 · Jones · 2017 [cited by applicant]
US 20170102699A1 · Anderson · 2017 [cited by applicant]
US 20170177925A1 · Volkart · 2017 [cited by applicant]
US 20170225796A1 · Sun et al. · 2017 [cited by applicant]
US 20170228690A1 · Kohli · 2017 [cited by applicant]
US 20170244937A1 · Meier et al. · 2017 [cited by applicant]
US 20170337791A1 · Gordon-Carroll · 2017 [cited by applicant]
US 20170371353A1 · Millinger, III · 2017 [cited by applicant]
US 20180082682A1 · Erickson et al. · 2018 [cited by applicant]
US 20180173220A1 · Wang · 2018 [cited by examiner]
US 20190011921A1 · Wang et al. · 2019 [cited by applicant]
US 20200241575A1 · Meisenholder · 2020 [cited by examiner]
US 20210001810A1 · Rivard · 2021 [cited by examiner]
US 20210011612A1 · Dancie et al. · 2021 [cited by applicant]
US 20210074016A1 · Li et al. · 2021 [cited by applicant]
US 20210166732A1 · Shaburova et al. · 2021 [cited by applicant]
US 20210241529A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210303075A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210303077A1 · Anvaripour et al. · 2021 [cited by applicant]
US 20210303140A1 · Mourkogiannis · 2021 [cited by applicant]
US 20210357631A1 · Takahashi · 2021 [cited by examiner]
US 20210382503A1 · Meisenholder et al. · 2021 [cited by applicant]
US 20210382564A1 · Blachly et al. · 2021 [cited by applicant]
US 20210397000A1 · Rodriguez, II · 2021 [cited by applicant]
US 20210405661A1 · Liu · 2021 [cited by examiner]
US 20230162533A1 · Matsuzawa · 2023 [cited by examiner]
US 20230350434A1 · Fliszar · 2023 [cited by examiner]
US 20240250540A1 · Patton · 2024 [cited by examiner]
CA 2887596A1 · 2015 [cited by applicant]
WO WO2018195883A1 · 2018 [cited by applicant]
WO WO2021217430A1 · 2021 [cited by applicant]
WO WO2023211655A1 · 2023 [cited by applicant]
US 10,656,660 B1, 05/2020, Meisenholder et al. (withdrawn) [cited by applicant]
Laput, Gierad, et al., “PixelTone: A Multimodal Interface for Image Editing”, ACM, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, FR, (2013), 10 pgs. [cited by applicant]
Leyden, John, “This SMS will self-destruct in 40 seconds”, [Online] Retrieved from the Internet: <URL: http://www.theregister.co.uk/2005/12/12/stealthtext/>, (Dec. 12, 2005), 1 pg. [cited by applicant]
Meisenholder, David, et al., “Remoteless Control of Drone Behavior”, U.S. Appl. No. 15/640,143, filed Jun. 30, 2017, 108 pgs. [cited by applicant]
Pourmehr, Shokoofeh, et al., ‘“You two! Take off!’: Creating, Modifying, and Commanding Groups of Robots Using Face Engagement and Indirect Speech in Voice Commands”, IEEE/RSJ International Conference on Intelligent Rob… [cited by applicant]
“International Application Serial No. PCT/US2023/017601, International Preliminary Report on Patentability mailed Nov. 7, 2024”, 11 pgs. [cited by applicant]
“International Application Serial No. PCT/US2023/017601, International Search Report mailed Jun. 5, 2023”, 9 pgs. [cited by applicant]
“International Application Serial No. PCT/US2023/017601, Written Opinion mailed Jun. 5, 2023”, 9 pgs. [cited by applicant]