Granted Patent B2

US 12,287,647 · App. 18/072,982 · Granted Apr 29, 2025

Inventors: David G. Fliszar (Manhattan Beach, CA); Russell Douglas Patton (Playa Vista, CA)

Assignee: Snap Inc.

G05D1/1062B64C39/024B64D47/08B64U70/10G06T7/579G06V10/751G06V20/17G06V40/161B64U2101/30B64U2201/10G06T2207/10032G06T2207/30201

Abstract

Claims (59)

1. An apparatus of an autonomous drone comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, configure the at least one processor to perform operations comprising:

capturing an image using an image capturing device of the autonomous drone;

processing the image to identify an object;

navigating the autonomous drone based on a first type of navigation for a period of time, wherein the first type of navigation is based on a position of the autonomous drone relative to the object;

capturing images using the image capturing device during the period of time;

processing the images to generate structural information of a real-world environment depicted in the images; and

navigating the autonomous drone based on a second type of navigation after the period of time, the period of time being based on a time to generate the structural information, and the second type of navigation being based on the structural information.

2. The apparatus of claim 1 , wherein the second type of navigation is based on one or more of: visual inertial odometry, optical flow, or Simultaneous Localization and Mapping (SLAM).

3. The apparatus of claim 1 , wherein the second type of navigation is simultaneous localization and mapping.

4. The apparatus of claim 1 , wherein navigating the autonomous drone based on the first type of navigation further comprises:

navigating the autonomous drone relative to the object to one or more waypoints.

5. The apparatus of claim 1 , wherein the object is a person or a face.

6. The apparatus of claim 1 , wherein the operations further comprise:

determining a height above a ground based on sensor data, and wherein the navigating the autonomous drone based on the first type of navigation is further based on the height above the ground.

7. The apparatus of claim 1 , wherein the image capturing device is a first image capturing device, wherein the first image capturing device is mounted horizontally relative to an axis of propellers of the autonomous drone, and a second image capturing device is mounted vertically relative to the axis of the propellers and is directed downward, and wherein the operations further comprise:

capturing images in the first type of navigation using the first image capturing device.

8. The apparatus of claim 1 , wherein the operations further comprise:

after the processing the image to identify the object, taking off from a hand.

9. The apparatus of claim 1 , wherein the operations further comprise:

hovering at a constant location relative to the object;

determining a windspeed based on an electrical power applied to electrical motors that operate propellers of the autonomous drone; and

in response to determining the windspeed is above a threshold value, landing the autonomous drone.

10. The apparatus of claim 1 , wherein the navigating the autonomous drone based on the first type of navigation is based on identifying the object in subsequent images captured by the image capturing device and orienting the autonomous drone in space relative to the object.

11. The apparatus of claim 1 , wherein the operations further comprise:

processing a second image to determine one or more additional objects, and wherein the navigating the autonomous drone based on the first type of navigation is further based on the one or more additional objects.

12. The apparatus of claim 1 , wherein the image is a first image and wherein the navigating the autonomous drone based on the first type of navigation further comprises:

navigating to a first waypoint based on a first position of the object determined from the first image;

capturing a second image;

navigating to a second waypoint based on a second position of the object determined from the second image; and

capturing a third image.

13. The apparatus of claim 1 , wherein the operations further comprise:

determining whether the generated structural information is sufficient to switch to the second type of navigation; and

in response to a determination that the generated structural information is sufficient to switch to the second type of navigation, determine to end the period of time.

14. The apparatus of claim 1 , wherein the operations further comprise:

determining a distance of the autonomous drone from the object.

15. The apparatus of claim 14 , wherein the determining is based on a number of pixels of an image sensor the object occupies in the captured image and an estimated size of the object.

16. A method performed on an apparatus of an autonomous drone, the method comprising:

capturing an image using an image capturing device of the autonomous drone;

processing the image to identify an object;

navigating the autonomous drone based on a first type of navigation for a period of time, wherein the first type of navigation is based on a position of the autonomous drone relative to the object;

capturing images using the image capturing device during the period of time;

processing the images to generate structural information of a real-world environment depicted in the images; and

navigating the autonomous drone based on a second type of navigation after the period of time, the period of time being based on a time to generate the structural information, and the second type of navigation being based on the structural information.

17. The method of claim 16 , wherein the second type of navigation is based on one or more of: structure from motion navigation, visual inertial odometry, optical flow, or Simultaneous Localization and Mapping (SLAM).

18. The method of claim 16 , wherein the method further comprises:

determining whether the generated structural information is sufficient to switch to the second type of navigation; and

in response to a determination that the generated structural information is sufficient to switch to the second type of navigation, determine to end the period of time.

19. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of an apparatus of an autonomous drone, cause the at least one processor to perform operations comprising:

capturing an image using an image capturing device of the autonomous drone;

processing the image to identify an object;

navigating the autonomous drone based on a first type of navigation for a period of time, wherein the first type of navigation is based on a position of the autonomous drone relative to the object;

capturing images using the image capturing device during the period of time;

processing the images to generate structural information of a real-world environment depicted in the images; and

navigating the autonomous drone based on a second type of navigation after the period of time, the period of time being based on a time to generate the structural information, and the second type of navigation being based on the structural information.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the operations further comprise:

determining whether the generated structural information is sufficient to switch to the second type of navigation; and

in response to a determination that the generated structural information is sufficient to switch to the second type of navigation, determine to end the period of time.

Assignments (1)

Continuity (2)

References Cited (145)