IP Library › Granted Patent US 10,303,185
Granted Patent B2
US 10,303,185 · App. 16/052,494 · Granted May 28, 2019

Multi-camera system and method of use

Inventors: Tong Zhang (Beijing, CN); Wei Li (Beijing, CN); Xiang Li (Beijing, CN); Jia Lu (Beijing, CN)
Assignee: Hangzhou Zero Zero Technology Co., Ltd.
G05D1/102G06T7/593G06T7/74H04N5/2252H04N5/2256H04N5/2258H04N5/2354H04N5/23203H04N5/23229H04N5/247H04N5/332H04N7/181H04N7/185H04N13/243B64C39/024B64C2201/141B64C2201/146G06T2207/10012G06T2207/10024G06T2207/10032G06T2207/30241G06T2207/30261
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,303,185
App. No.
16/052,494
Granted
May 28, 2019
Kind
B2
Abstract

A method for operating a system including a plurality of cameras, the method including: selecting a subset of the cameras, determining a subset of pixels captured by the camera subset, determining a pixel depth associated with each pixel of the pixel subset, and controlling system operation based on the pixel depth.

Claims (86)

1. A method for aircraft operation comprising, at an aircraft comprising a set of cameras:

selecting a first camera group from the set of cameras, the first camera group comprising a lateral camera and a first downward-facing camera, wherein a lateral camera view region and a first downward-facing camera view region each comprise a first overlapped subset;

selecting a second camera group from the set of cameras, the second camera group comprising the first downward-facing camera and a second downward-facing camera, wherein the first downward-facing camera view region and a second downward-facing camera view region each comprise a second overlapped subset;

sampling a set of images, comprising:

at the first downward-facing camera, sampling a first image during a first time period;

at the second downward-facing camera, sampling a second image during the first time period; and

at the lateral camera, sampling a third image during the first time period;

based on the first camera group, determining a first dataset associated with a first region of the first image and a region of the third image, wherein the first region of the first image and the region of the third image are each representative of a first obstacle within the first overlapped subset;

based on the first dataset, determining a lateral clearance between the aircraft and the first obstacle;

based on the second camera group, determining a second dataset associated with a second region of the first image and a region of the second image, wherein the second region of the first image and the region of the second image are each representative of a second obstacle within the second overlapped subset;

based on the second dataset, determining an downward clearance between the aircraft and the second obstacle; and

based on the lateral clearance and the downward clearance, controlling flight of the aircraft.

2. The method of claim 1 , wherein the aircraft defines a lateral plane, the aircraft further comprising:

a set of rotors comprising a first rotor, a second rotor, a third rotor, and a fourth rotor, each rotor of the set defining a respective rotor rotation axis and a respective rotor plane normal the respective rotor rotation axis, wherein each respective rotor plane is coplanar with the lateral plane; and

a housing defining a perimeter, the perimeter surrounding the set of rotors within the lateral plane.

3. The method of claim 2 , wherein:

the housing comprises a central portion arranged between the first and second rotors; and

the first and second downward-facing cameras are mounted to the central portion.

4. The method of claim 3 , wherein the lateral camera is arranged on the perimeter.

5. The method of claim 1 , further comprising, based on the lateral clearance and an aircraft velocity, determining a potential collision with the first obstacle, wherein controlling flight of the aircraft comprises changing the aircraft velocity to avoid the potential collision.

6. The method of claim 1 , wherein selecting the first camera group comprises selecting the lateral camera based on an aircraft velocity vector, wherein a lateral camera view vector within the lateral camera view region is parallel the aircraft velocity vector.

7. The method of claim 6 , further comprising, after the first time period:

controlling the aircraft to change velocity to a second aircraft velocity vector non-parallel the aircraft velocity vector;

selecting a third camera group from the set of cameras based on the second aircraft velocity vector, the third camera group comprising a second lateral camera and the second downward-facing camera, wherein a second lateral camera view region and the second downward-facing camera view region each comprise a third overlapped subset;

sampling a second set of images, comprising:

at the second downward-facing camera, sampling a fourth image during a second time period; and

at the second lateral camera, sampling a fifth image during the second time period;

based on the third camera group, selecting a third dataset comprising a region of the fourth image and a region of the fifth image, wherein the region of the fourth image and the region of the fifth image are each representative of a third obstacle within the third overlapped subset;

based on the third dataset, determining a second lateral clearance between the aircraft and the third obstacle; and

based on the second lateral clearance, controlling flight of the aircraft.

8. The aircraft of claim 1 , wherein the first image, the second image, and the third image are sampled at a first time.

9. A method for aircraft operation comprising, at an aircraft comprising a first camera, a second camera, and a third camera:

sampling a set of images, comprising:

at the first camera, sampling a first image;

concurrent with sampling the first image, at the second camera, sampling a second image, wherein a first camera view region and a second camera view region both comprise a first overlapped subset; and

concurrent with sampling the first image, at the third camera, sampling a third image, wherein a third camera view region and the second camera view region both comprise a second overlapped subset;

determining a first dataset associated with a region of the first image and a first region of the second image, wherein the region of the first image and the first region of the second image are both representative of a first obstacle within the first overlapped subset;

based on the first dataset, determining a first obstacle clearance between the aircraft and the first obstacle;

determining a second dataset associated with a second region of the second image and a region of the third image, wherein the region of the third image and the second region of the second image are both representative of a second obstacle within the second overlapped subset and outside the first overlapped subset;

based on the second dataset, determining a second obstacle clearance between the aircraft and the second obstacle; and

based on the first obstacle clearance and the second obstacle clearance, controlling flight of the aircraft.

10. The method of claim 9 , wherein the aircraft defines a lateral plane, the aircraft further comprising:

a set of rotors comprising a first rotor, a second rotor, a third rotor, and a fourth rotor, each rotor of the set defining a respective rotor rotation axis and a respective rotor plane normal the respective rotor rotation axis, wherein each respective rotor plane is coplanar with the lateral plane; and

a housing defining a perimeter, the perimeter surrounding the set of rotors within the lateral plane.

11. The method of claim 10 , wherein the first and second cameras are arranged on the perimeter.

12. The method of claim 11 , wherein:

the aircraft defines a central axis normal to the lateral plane, wherein the central axis is arranged within a convex hull of the rotor rotation axes of the set of rotors;

the perimeter comprises:

a first side arranged proximal the first and second rotors with respect to the central axis; and

a second side arranged proximal the third rotor and distal the first rotor with respect to the central axis;

the first and second cameras are arranged along the first side; and

the third camera is arranged along the second side.

13. The method of claim 12 , wherein the second side is arranged proximal the fourth rotor with respect to the central axis.

14. The method of claim 11 , wherein:

the housing comprises a central portion arranged between the first and second rotors; and

the third camera is mounted to the central portion.

15. The method of claim 9 , wherein:

a first angle between a first camera central view axis and a second camera central view axis is greater than a threshold angle; and

a second angle between a second camera central view axis and the third camera central view axis is greater than the threshold angle.

16. The method of claim 15 , wherein the threshold angle is 60 degrees.

17. The method of claim 9 , wherein:

the third camera view region further comprises the first overlapped subset; and

the first dataset further comprises a second region of the third image, the second region of the third image representative of the first obstacle.

18. The method of claim 9 , wherein a first camera view vector within the first camera view region is parallel an aircraft velocity vector and a second camera view vector within the second camera view region is parallel the aircraft velocity vector.

19. An aircraft comprising:

a housing;

a set of cameras mechanically coupled to the housing, the set of cameras comprising a first camera, a second camera, and a third camera;

a motive mechanism mechanically coupled to the housing; and

a processor mechanically coupled to the housing;

wherein the processor is configured to:

receive a set of images from the set of cameras, the set of images comprising:

a first image sampled by the first camera, the first image representative of a first obstacle;

a second image sampled by the second camera, the second image representative of the first obstacle and a second obstacle; and

a third image sampled by the third camera, the third image representative of the second obstacle;

based on the first and second images, determine a first obstacle clearance between the aircraft and the first obstacle;

based on the third image, determine a second obstacle clearance between the aircraft and the second obstacle; and

based on the first obstacle clearance and the second obstacle clearance, control operation of the motive mechanism to control flight of the aircraft.

20. The aircraft of claim 19 , wherein:

the aircraft defines a lateral plane;

the aircraft further comprises a set of rotors comprising a first rotor, a second rotor, a third rotor, and a fourth rotor, each rotor of the set defining a respective rotor rotation axis and a respective rotor plane normal the respective rotor rotation axis, wherein each respective rotor plane is coplanar with the lateral plane;

the housing defines a perimeter, the perimeter surrounding the set of rotors within the lateral plane;

the perimeter defines a first side arranged proximal the first and second rotors; and

the first and second cameras are arranged along the first side.

21. The aircraft of claim 19 , wherein:

the housing comprises a central portion arranged between the first and second rotors; and

the second and third cameras are mounted to the central portion.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2018
From: ZHANG, TONG; LI, WEI; LI, XIANG; LU, JIA
To: HANGZHOU ZERO ZERO TECHNOLOGY CO., LTD
Reel/Frame 046699/0706 →
Continuity (3)
Continuation 15875967 · Jan 19, 2018
Provisional Application 62449340 · Jan 23, 2017
Related Publication 20180341278A1 · Nov 29, 2018
Cited By (2)
US 12,231,785 US 12,648,829