IP Library Granted Patent US 12,190,739
Granted Patent B2
US 12,190,739 · App. 17/730,569 · Granted Jan 7, 2025

Navigation correction for excessive wind

Inventors: David G. Fliszar (Manhattan Beach, CA); Russell Douglas Patton (Playa Vista, CA)
Assignee: Snap Inc.
G08G5/0039B64C39/024G08G5/006G08G5/0069G08G5/0091B64U50/19B64U2101/30B64U2201/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,190,739
App. No.
17/730,569
Granted
Jan 7, 2025
Kind
B2
Abstract

Systems, computer readable medium and methods for navigation correction for excessive wind in an autonomous drone are disclosed. Excessive winds can be a particular problem for small autonomous drones as safety and retrieval of the autonomous drones is important and the autonomous drones often have limited thrust and batteries. Autonomous drones are disclosed that detect and correct flight plans when excessive winds are detected. The autonomous drone determines based on the severity of the excessive winds whether to return to a home position which is typically a position of a user of the autonomous drone or to land in place. If the excessive winds subside, then the autonomous drone returns to its original flight plan at the point where the autonomous drone was blown off course by the excessive winds. The autonomous drone detects excessive winds either directly by sensor data or inferentially by unanticipated movement of the autonomous drone.

Claims (56)

1. A navigation system of an autonomous drone comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, configure the a least one processor to perform operations comprising:

causing an autonomous drone to take off from a home position based on a flight plan;

navigating the autonomous drone in accordance with the flight plan;

capturing images while navigating the autonomous drone in accordance with the flight plan;

streaming, in near real time, the captured images to a host device;

in response to detecting a wind speed exceeding a nominal threshold, determining an updated flight plan to return to the home position, navigating the autonomous drone in accordance with the updated flight plan, and suspending streaming the captured images until the autonomous drone returns to a position on the flight plan while being navigated in accordance with the updated flight plan; and

in response to detecting the wind speed exceeding a critical threshold, navigating the autonomous drone to land in place.

2. The navigation system of claim 1 wherein the operations further comprise:

in response to detecting the wind speed dropping below the critical threshold, determining a second updated flight plan to return to the home position, and navigating the autonomous drone in accordance with the second updated flight plan.

3. The navigation system of claim 1 wherein the operations further comprise:

in response to detecting the wind speed dropping below the nominal threshold, determining a second updated flight plan to resume the flight plan, and navigating the autonomous drone in accordance with the second updated flight plan.

4. The navigation system of claim 3 wherein the second updated flight plan comprises:

returning to a position of the flight plan from before the wind speed exceeded the nominal threshold.

5. The navigation system of claim 4 wherein the operations further comprise:

capturing images while navigating the autonomous drone in accordance with the flight plan; and

capturing images while navigating the autonomous drone in accordance with the second updated flight plan.

6. The navigation system of claim 1 wherein the operations further comprise:

determining a first height of the autonomous drone during navigating the autonomous drone in accordance with the flight plan, and wherein navigating the autonomous drone to land in place further comprises:

during navigating the autonomous drone to land in place, determining a second height of the autonomous drone; and

in response to the second height exceeding a height threshold, determining an updated flight plan to a location not exceeding the height threshold.

7. The navigation system of claim 1 wherein the nominal threshold and the critical threshold are adjusted based on a situation of the autonomous drone.

8. The navigation system of claim 7 wherein the situation comprises one or more of: whether people other than a user of the autonomous drone are present, whether the autonomous drone is in an outside location or an inside location, whether there are restricted areas within a second threshold of the flight plan, whether the autonomous drone is in a city situation, suburb situation, or field situation, whether there are dangerous areas within a third threshold of the flight plan, or a safety setting of the autonomous drone.

9. The navigation system of claim 1 wherein the operations further comprise:

in response to a determination that a level of a battery indicates the autonomous drone cannot finish the flight plan, determining the updated flight plan to return to the home position, and navigating the autonomous drone in accordance with the updated flight plan.

10. The navigation system of claim 1 wherein the operations further comprise:

in response to sensor data indicating a motion exceeding a nominal motion threshold, determining the updated flight plan to return to the home position, and navigating the autonomous drone in accordance with the updated flight plan; and

in response to the sensor data indicating a motion exceeding a critical motion threshold, navigating the autonomous drone to land in place.

11. The navigation system of claim 1 wherein the in response to detecting a wind speed exceeding the nominal threshold further comprises:

reducing power to one or more components of the autonomous drone, wherein the one or more components comprise one or more of: photography camera, wireless connections, or input/output devices.

12. The navigation system of claim 1 wherein the operations further comprise:

determining a deviation from the flight plan;

in response to the deviation from the flight plan indicating a deviation exceeding a nominal deviation threshold, determining the updated flight plan to return to the home position, and navigating the autonomous drone in accordance with the updated flight plan; and

in response to the deviation from the flight plan indicating a deviation exceeding a critical deviation threshold, navigating the autonomous drone to land in place.

13. The navigation system of claim 1 wherein the operations further comprise:

accessing a selection of the flight plan based on a position of a control user interface device; and

determining the flight plan based on the selection of the flight plan.

14. The navigation system of claim 1 wherein the wind speed is determined based on data from one or more of: a wind speed sensor, a difference in an expected position and an actual position of the autonomous drone, an accelerometer, a gyroscope, a navigation camera, an inertial measurement unit, a magnetic sensor, a height detector, by a value of a motor thrust command or output power command to the electrical motors compared with an expected value for a given performance, by the motor thrust command or the output power command to the electrical motors that exceeds a first threshold above an expected value, by the motor thrust command or the output power command to the electrical motors that exceeds a second threshold above the expected value, by the motor thrust command or the output power command to the electrical motors being at a maximum value for a third threshold period of time, or by the motor thrust command or the output power command to the electrical motors being at the maximum value for a fourth threshold period of time.

15. The navigation system of claim 1 wherein the updated flight plan is determined based on whether the autonomous drone is capturing images.

16. A method performed on an autonomous drone, the method comprising:

taking off from a home position based on a flight plan;

navigating the autonomous drone in accordance with the flight plan;

capturing images while navigating the autonomous drone in accordance with the flight plan; streaming, in near real time, the captured images to a host device;

in response to detecting a wind speed exceeding a nominal threshold, determining an updated flight plan to return to the home position, navigating the autonomous drone in accordance with the updated flight plan, and suspending streaming the captured images until the autonomous drone returns to a position on the flight plan while being navigated in accordance with the updated flight plan; and

in response to detecting the wind speed exceeding a critical threshold, navigating the autonomous drone to land in place.

17. The method of claim 16 wherein the method further comprises:

in response to detecting the wind speed dropping below the critical threshold, determining a second updated flight plan to return to the home position, and navigating the autonomous drone in accordance with the second updated flight plan.

18. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a navigation system of an autonomous drone, cause the at least one processor to perform operations comprising:

taking off from a home position based on a flight plan;

navigating the autonomous drone in accordance with the flight plan;

capturing images while navigating the autonomous drone in accordance with the flight plan; streaming, in near real time, the captured images to a host device;

in response to detecting a wind speed exceeding a nominal threshold, determining an updated flight plan to return to the home position, navigating the autonomous drone in accordance with the updated flight plan, and suspending streaming the captured images until the autonomous drone returns to a position on the flight plan while being navigated in accordance with the updated flight plan; and

in response to detecting the wind speed exceeding a critical threshold, navigating the autonomous drone to land in place.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

in response to detecting the wind speed dropping below the critical threshold, determining a second updated flight plan to return to the home position, and navigating the autonomous drone in accordance with the second updated flight plan.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2024
From: FLISZAR, DAVID G.; PATTON, RUSSELL DOUGLAS
To: SNAP INC.
Reel/Frame 066193/0105 →
Continuity (1)
Related Publication 20230351899A1 · Nov 2, 2023
References Cited (140)
US 6038295A · Mattes · 2000 [cited by applicant]
US 6819982B2 · Doane · 2004 [cited by applicant]
US 6980909B2 · Root et al. · 2005 [cited by applicant]
US 7173651B1 · Knowles · 2007 [cited by applicant]
US 7411493B2 · Smith · 2008 [cited by applicant]
US 7535890B2 · Rojas · 2009 [cited by applicant]
US 7542073B2 · Li et al. · 2009 [cited by applicant]
US 7971156B2 · Albertson et al. · 2011 [cited by applicant]
US 8131597B2 · Hudetz · 2012 [cited by applicant]
US 8174562B2 · Hartman · 2012 [cited by applicant]
US 8199747B2 · Rojas et al. · 2012 [cited by applicant]
US 8274550B2 · Steuart, III · 2012 [cited by applicant]
US 8332475B2 · Rosen et al. · 2012 [cited by applicant]
US 8646720B2 · Shaw · 2014 [cited by applicant]
US 8718333B2 · Wolf et al. · 2014 [cited by applicant]
US 8724622B2 · Rojas · 2014 [cited by applicant]
US 8874677B2 · Rosen et al. · 2014 [cited by applicant]
US 8909679B2 · Root et al. · 2014 [cited by applicant]
US 8995433B2 · Rojas · 2015 [cited by applicant]
US 9040574B2 · Wang et al. · 2015 [cited by applicant]
US 9055416B2 · Rosen et al. · 2015 [cited by applicant]
US 9100806B2 · Rosen et al. · 2015 [cited by applicant]
US 9100807B2 · Rosen et al. · 2015 [cited by applicant]
US 9191776B2 · Root et al. · 2015 [cited by applicant]
US 9204252B2 · Root · 2015 [cited by applicant]
US 9225897B1 · Sehn et al. · 2015 [cited by applicant]
US 9276886B1 · Samaranayake · 2016 [cited by applicant]
US 9344642B2 · Niemi et al. · 2016 [cited by applicant]
US 9345711B2 · Friedhoff · 2016 [cited by applicant]
US 9443227B2 · Evans et al. · 2016 [cited by applicant]
US 9471059B1 · Wilkins · 2016 [cited by applicant]
US 9489661B2 · Evans et al. · 2016 [cited by applicant]
US 9489937B1 · Beard et al. · 2016 [cited by applicant]
US 9491134B2 · Rosen et al. · 2016 [cited by applicant]
US 9576369B2 · Venkataraman et al. · 2017 [cited by applicant]
US 9589448B1 · Schneider et al. · 2017 [cited by applicant]
US 9681046B2 · Adsumilli et al. · 2017 [cited by applicant]
US 9705831B2 · Spiegel · 2017 [cited by applicant]
US 9723272B2 · Lu et al. · 2017 [cited by applicant]
US 9742713B2 · Spiegel et al. · 2017 [cited by applicant]
US 9747901B1 · Gentry · 2017 [cited by applicant]
US 9922659B2 · Bradlow et al. · 2018 [cited by applicant]
US 9989965B2 · Cuban et al. · 2018 [cited by applicant]
US 10061328B2 · Canoy et al. · 2018 [cited by applicant]
US 10102423B2 · Shaburov et al. · 2018 [cited by applicant]
US 10109224B1 · Ratti et al. · 2018 [cited by applicant]
US 10140987B2 · Erickson et al. · 2018 [cited by applicant]
US 10168700B2 · Gordon et al. · 2019 [cited by applicant]
US 10284508B1 · Allen et al. · 2019 [cited by applicant]
US 10370118B1 · Nielsen et al. · 2019 [cited by applicant]
US 10439972B1 · Spiegel et al. · 2019 [cited by applicant]
US 10501180B2 · Yu · 2019 [cited by applicant]
US 10509466B1 · Miller et al. · 2019 [cited by applicant]
US 10514876B2 · Sehn · 2019 [cited by applicant]
US 10614855B2 · Huang · 2020 [cited by applicant]
US 10748347B1 · Li et al. · 2020 [cited by applicant]
US 10768639B1 · Meisenholder et al. · 2020 [cited by applicant]
US 10958608B1 · Allen et al. · 2021 [cited by applicant]
US 10962809B1 · Castañeda · 2021 [cited by applicant]
US 10996846B2 · Robertson et al. · 2021 [cited by applicant]
US 10997787B2 · Ge et al. · 2021 [cited by applicant]
US 11012390B1 · Al Majid et al. · 2021 [cited by applicant]
US 11030454B1 · Xiong et al. · 2021 [cited by applicant]
US 11036368B1 · Al Majid et al. · 2021 [cited by applicant]
US 11062498B1 · Voss et al. · 2021 [cited by applicant]
US 11087728B1 · Canberk et al. · 2021 [cited by applicant]
US 11092998B1 · Castañeda et al. · 2021 [cited by applicant]
US 11106342B1 · Al Majid et al. · 2021 [cited by applicant]
US 11126206B2 · Meisenholder et al. · 2021 [cited by applicant]
US 11143867B2 · Rodriguez, II · 2021 [cited by applicant]
US 11169600B1 · Canberk et al. · 2021 [cited by applicant]
US 11227626B1 · Krishnan Gorumkonda et al. · 2022 [cited by applicant]
US 20070250526A1 · Hanna · 2007 [cited by applicant]
US 20080255842A1 · Simhi · 2008 [cited by applicant]
US 20090122133A1 · Hartman · 2009 [cited by applicant]
US 20110202598A1 · Evans et al. · 2011 [cited by applicant]
US 20120194420A1 · Osterhout et al. · 2012 [cited by applicant]
US 20120209924A1 · Evans et al. · 2012 [cited by applicant]
US 20120281885A1 · Syrdal et al. · 2012 [cited by applicant]
US 20120287274A1 · Bevirt · 2012 [cited by applicant]
US 20130056581A1 · Sparks · 2013 [cited by applicant]
US 20130238168A1 · Reyes · 2013 [cited by applicant]
US 20140254896A1 · Zhou et al. · 2014 [cited by applicant]
US 20150022432A1 · Stewart et al. · 2015 [cited by applicant]
US 20150070272A1 · Kim et al. · 2015 [cited by applicant]
US 20150175263A1 · Reyes · 2015 [cited by applicant]
US 20150199022A1 · Gottesman et al. · 2015 [cited by applicant]
US 20150287246A1 · Huston et al. · 2015 [cited by applicant]
US 20150331490A1 · Yamada · 2015 [cited by applicant]
US 20150362917A1 · Wang et al. · 2015 [cited by applicant]
US 20160063987A1 · Xu et al. · 2016 [cited by applicant]
US 20160161946A1 · Wuth Sepulveda et al. · 2016 [cited by applicant]
US 20160179096A1 · Bradlow et al. · 2016 [cited by applicant]
US 20160292886A1 · Erad et al. · 2016 [cited by applicant]
US 20160307573A1 · Wobrock · 2016 [cited by applicant]
US 20160336020A1 · Bradlow et al. · 2016 [cited by applicant]
US 20170031369A1 · Liu et al. · 2017 [cited by applicant]
US 20170094259A1 · Kouperman et al. · 2017 [cited by applicant]
US 20170099424A1 · Jones · 2017 [cited by applicant]
US 20170102699A1 · Anderson · 2017 [cited by applicant]
US 20170177925A1 · Volkart · 2017 [cited by applicant]
US 20170225796A1 · Sun et al. · 2017 [cited by applicant]
US 20170228690A1 · Kohli · 2017 [cited by applicant]
US 20170244937A1 · Meier et al. · 2017 [cited by applicant]
US 20170320564A1 · Kuzikov · 2017 [cited by applicant]
US 20170337791A1 · Gordon-carroll · 2017 [cited by applicant]
US 20170371353A1 · Millinger, III · 2017 [cited by applicant]
US 20180082682A1 · Erickson et al. · 2018 [cited by applicant]
US 20180233007A1 · Williams · 2018 [cited by examiner]
US 20180246529A1 · Hu et al. · 2018 [cited by applicant]
US 20180321676A1 · Matuszeski · 2018 [cited by examiner]
US 20190011921A1 · Wang et al. · 2019 [cited by applicant]
US 20200241575A1 · Meisenholder et al. · 2020 [cited by applicant]
US 20200334993A1 · Demri · 2020 [cited by examiner]
US 20210011612A1 · Dancie et al. · 2021 [cited by applicant]
US 20210065565A1 · Dow · 2021 [cited by examiner]
US 20210074016A1 · Li et al. · 2021 [cited by applicant]
US 20210166732A1 · Shaburova et al. · 2021 [cited by applicant]
US 20210241529A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210303075A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210303077A1 · Anvaripour et al. · 2021 [cited by applicant]
US 20210303140A1 · Mourkogiannis · 2021 [cited by applicant]
US 20210319705A1 · Furumoto · 2021 [cited by applicant]
US 20210362848A1 · Spencer · 2021 [cited by applicant]
US 20210382503A1 · Meisenholder et al. · 2021 [cited by applicant]
US 20210382564A1 · Blachly et al. · 2021 [cited by applicant]
US 20210397000A1 · Rodriguez, II · 2021 [cited by applicant]
CA 2887596 · 2015 [cited by applicant]
CN 111226184 · 2020 [cited by applicant]
CN 112925335 · 2021 [cited by applicant]
CN 109164827 · 2021 [cited by applicant]
WO 2023211638 · 2023 [cited by applicant]
US 10,656,660 B1, 05/2020, Meisenholder et al. (withdrawn) [cited by applicant]
Laput, Gierad, “PixelTone: A Multimodal Interface for Image Editing”, ACM, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, FR, (2013), 10 pgs. [cited by applicant]
Leyden, John, “This SMS will self-destruct in 40 seconds”, [Online] Retrieved from the Internet: <URL: http://www.theregister.co.uk/2005/12/12/stealthtext/>, (Dec. 12, 2005), 1 pg. [cited by applicant]
Meisenholder, David, “Remoteless Control of Drone Behavior”, U.S. Appl. No. 15/640,143, filed Jun. 30, 2017, 108 pgs. [cited by applicant]
Pourmehr, Shokoofeh, “‘You two! Take off!’: Creating, Modifying, and Commanding Groups of Robots Using Face Engagement and Indirect Speech in Voice Commands”, IEEE RSJ International Conference on Intelligent Robots and … [cited by applicant]
Yamada, Wataru, “iSphere: Self-Luminous Spherical Drone Display”, Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST), Quebec City, CA, (Oct. 22-25, 2017), 635-343. [cited by applicant]
“International Application Serial No. PCT/US2023/017251, International Search Report mailed Jun. 14, 2023”, 4 pgs. [cited by applicant]
“International Application Serial No. PCT/US2023/017251, Written Opinion mailed Jun. 14, 2023”, 8 pgs. [cited by applicant]