IP Library › Granted Patent US 9,866,752
Granted Patent B2
US 9,866,752 · App. 14/954,357 · Granted Jan 9, 2018

Systems and methods for producing a combined view from fisheye cameras

Inventors: Tao Zhang (Campbell, CA); Feng Guo (San Diego, CA); Ning Bi (San Diego, CA); Bijan Forutanpour (San Diego, CA); Phi Hung Nguyen (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N5/23238B60R1/00G06T3/0062G06T15/205H04N5/23261H04N5/265H04N13/0239B60R2300/105B60R2300/303
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,866,752
App. No.
14/954,357
Filed
Nov 30, 2015
Granted
Jan 9, 2018
Kind
B2
Examiner
PHAM, NAM D
Art Unit
2487
USPC
348/148
Abstract

An electronic device is described. The electronic device includes a processor. The processor is configured to obtain images from a plurality of cameras. The processor is also configured to project each image to a respective 3-dimensional (3D) shape for each camera. The processor is further configured to generate a combined view from the images.

Claims (49)

1. An electronic device, comprising:

a processor configured to:

obtain images from a plurality of cameras;

project each image to a respective 3-dimensional (3D) shape used to model each camera; and

generate a combined view based on the projected images.

2. The electronic device of claim 1 , wherein the processor is configured to rotate at least one of the 3D shapes to approximately align a corresponding image to a direction.

3. The electronic device of claim 1 , wherein the processor is configured to rotate at least one of the 3D shapes to compensate for a tilt of a corresponding camera.

4. The electronic device of claim 1 , wherein the processor is configured to project at least a portion of each 3D shape to a common 3D shape enclosing all 3D shapes associated with the cameras.

5. The electronic device of claim 4 , wherein the processor is configured to project at least a portion of the common 3D shape to a common plane.

6. The electronic device of claim 5 , wherein the processor is configured to project the common 3D shape from a north pole of the common 3D shape to the common plane, wherein the common plane is approximately parallel to ground.

7. The electronic device of claim 5 , wherein the processor is configured to align the images projected to the common plane.

8. The electronic device of claim 7 , wherein the processor is configured to determine whether an alignment error exceeds a predetermined threshold.

9. The electronic device of claim 7 , wherein the processor is configured to determine at least one seam between adjacent images and trim the images to produce a combined image.

10. The electronic device of claim 9 , wherein the processor is configured to project the combined image to a shape.

11. The electronic device of claim 9 , wherein the cameras are installed in a vehicle and the processor is configured to present the combined view in the vehicle.

12. The electronic device of claim 1 , wherein the images are wide-angle images and the cameras are wide-angle cameras.

13. A method, comprising:

obtaining images from a plurality of cameras;

projecting each image to a respective 3-dimensional (3D) shape used to model each camera; and

generating a combined view based on the projected images.

14. The method of claim 13 , further comprising:

rotating at least one of the 3D shapes to approximately align a corresponding image to a direction.

15. The method of claim 13 , further comprising:

rotating at least one of the 3D shapes to compensate for a tilt of a corresponding camera.

16. The method of claim 13 , further comprising:

projecting at least a portion of each 3D shape to a common 3D shape enclosing all 3D shapes associated with the cameras.

17. The method of claim 16 , further comprising:

projecting at least a portion of the common 3D shape to a common plane.

18. The method of claim 17 , wherein the common 3D shape is projected from a north pole of the common 3D shape to the common plane, wherein the common plane is approximately parallel to ground.

19. The method of claim 17 , wherein generating the combined view comprises aligning the images projected to the common plane.

20. The method of claim 19 , wherein generating the combined view comprises:

determining at least one seam between adjacent images; and

trimming the images to produce a combined image.

21. The method of claim 20 , wherein generating the combined view comprises projecting the combined image to a shape.

22. The method of claim 20 , wherein the cameras are installed in a vehicle and the combined view is presented in the vehicle.

23. A computer-program product, comprising a non-transitory tangible computer-readable medium having instructions thereon, the instructions comprising:

code for causing an electronic device to obtain images from a plurality of cameras;

code for causing the electronic device to project each image to a respective 3-dimensional (3D) shape used to model each camera; and

code for causing the electronic device to generate a combined view based on the projected images.

24. The computer-program product of claim 23 , further comprising code for causing the electronic device to rotate at least one of the 3D shapes to approximately align a corresponding image to a direction.

25. The computer-program product of claim 23 , further comprising code for causing the electronic device to rotate at least one of the 3D shapes to compensate for a tilt of a corresponding camera.

26. The computer-program product of claim 23 , further comprising code for causing the electronic device to project at least a portion of each 3D shape to a common 3D shape enclosing all 3D shapes associated with the cameras.

27. An apparatus, comprising:

means for obtaining images from a plurality of cameras;

means for projecting each image to a respective 3-dimensional (3D) shape used to model each camera; and

means for generating a combined view based on the projected images.

28. The apparatus of claim 27 , further comprising means for rotating at least one of the 3D shapes to approximately align a corresponding image to a direction.

29. The apparatus of claim 27 , further comprising means for rotating at least one of the 3D shapes to compensate for a tilt of a corresponding camera.

30. The apparatus of claim 27 , further comprising means for projecting at least a portion of each 3D shape to a common 3D shape enclosing all 3D shapes associated with the cameras.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2016
From: ZHANG, TAO; GUO, FENG; BI, NING; FORUTANPOUR, BIJAN; NGUYEN, PHI HUNG
To: QUALCOMM INCORPORATED
Reel/Frame 037738/0518 →
Continuity (2)
Provisional Application 62170075 · Jun 2, 2015
Related Publication 20160360104A1 · Dec 8, 2016