IP Library › Granted Patent US 8,896,668
Granted Patent B2
US 8,896,668 · App. 13/079,640 · Granted Nov 25, 2014

Combining data from multiple image sensors

Inventors: Sergiu R. Goma (San Diego, CA); Milivoje Aleksic (San Diego, CA); Hau Hwang (San Diego, CA); Joseph Cheung (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N13/0239H04N13/0048H04N5/247H04N13/0296
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,896,668
App. No.
13/079,640
Filed
Apr 4, 2011
Granted
Nov 25, 2014
Kind
B2
Art Unit
2487
USPC
348/47
Abstract

A method of combining data from multiple sensors is disclosed. The method includes receiving lines of image data at an image processor having an input for a single camera. Each line of the image data includes first line data from a first image captured by a first camera and second line data from a second image captured by a second camera. The method also includes generating an output frame having a first section corresponding to line data of the first image and having a second section corresponding to line data of the second image. The first section and the second section are configured to be used to generate a three-dimensional (3D) image format or a 3D video format.

Claims (43)

1. A method comprising:

receiving lines of image data at an image processor having an input for a single camera, wherein each line of the image data is a combined line including a first data line from a first image captured by a first camera and a second data line from a second image captured by a second camera, wherein a combiner coupled to the first camera and to the second camera generates each line of the image data by receiving and aligning one first data line with one second data line to form one combined line, and wherein the combiner and the image processor are included in a system-on-chip; and

generating an output frame based on the combined lines, the output frame having a first section corresponding to the first data lines of the combined lines and having a second section corresponding to the second data lines of the combined lines, wherein the first section and the second section are configured to be used to generate data in a three-dimensional (3D) image format or in a 3D video format.

2. The method of claim 1 , further comprising:

synchronizing, using the combiner, line by line readout of first image data from the first camera and second image data from the second camera to generate each line of the image data; and

storing each combined line as one line of the image data in a separate row of a memory.

3. The method of claim 1 , further comprising:

storing each combined line as one line of the image data in a separate row of a memory.

4. The method of claim 1 , further comprising processing the output frame to generate 3D image data and sending the 3D image data to a display device, wherein the display device is a component of at least one of a communication device, a third camera, a personal digital assistant, and a computer.

5. The method of claim 1 , wherein each line of the image data includes a gap section between the first data line and the second data line, wherein the gap section is between the first section and the second section in the output frame, and wherein the gap section comprises non-image data.

6. The method of claim 1 , wherein the input for the single camera comprises a single camera parallel interface or serial interface and an inter-integrated circuit interface.

7. The method of claim 1 , wherein a control signal is provided by the image processor to the first camera and to the second camera.

8. The method of claim 7 , wherein the control signal is provided by the image processor via a sensor synchronizer, and wherein the image processor corresponds to a video front end.

9. An apparatus comprising:

an image processor having an input for a single camera, wherein the image processor is configured to receive lines of image data via the input, each line of the image data being a combined line including a first data line from a first image captured by a first camera and a second data line from a second image captured by a second camera,

wherein the image processor is configured to generate an output frame based on the combined lines, the output frame having a first section corresponding to the first data lines of the combined lines and having a second section corresponding to the second data lines of the combined lines, and wherein the first section and the second section are configured to be used to generate data with a three-dimensional (3D) image format or to generate data with a 3D video format; and

a combiner coupled to the first camera and to the second camera, wherein the combiner is configured to generate each line of the image data by receiving and aligning one first data line with one second data line to form one combined line, and wherein the image processor and the combiner are included in a system-on-chip.

10. The apparatus of claim 9 , wherein the combiner is configured to align line by line readout of first image data from the first camera and second image data from the second camera to generate each line of the image data, and wherein a synchronization between data streams from the first and second cameras received at the combiner occurs within a single line of the image data.

11. The apparatus of claim 9 , further comprising a graphics processing unit configured to process the output frame to generate 3D image data or 3D video data.

12. The apparatus of claim 9 , further comprising;

a graphics processing unit configured to process the output frame to generate 3D image data or 3D video data and

a display device coupled to display the 3D image data or the 3D video data, wherein the display device is a component of at least one of a communication device, a third camera, a personal digital assistant, and a computer.

13. The apparatus of claim 9 , further comprising a graphics processing unit configured to process the output frame to generate 3D image data or 3D video data, wherein the input for the single camera comprises a single camera parallel interface or serial interface and an inter-integrated circuit interface.

14. The apparatus of claim 9 , further comprising:

a sensor synchronizer coupled to the first camera and to the second camera, wherein the sensor synchronizer generates a common control clock signal, a common control data signal, a common camera clock signal and a common camera reset signal; and

a memory configured to store each combined line as one line of the image data in a separate row of the memory.

15. The apparatus of claim 14 , wherein the image processor is configured to provide the common camera clock signal via the sensor synchronizer to the first camera and to the second camera, and wherein the image processor corresponds to a video front end.

16. A non-transitory computer readable-medium comprising instructions, which when executed by a processor cause the processor to:

receive lines of image data via an input for a single camera, wherein each line of the image data is a combined line including a first data line from a first image captured by a first camera and a second data line from a second image captured by a second camera, wherein a combiner coupled to the first camera and to the second camera generates each line of the image data by receiving and aligning one first data line with one second data line to form one combined line, and wherein the combiner and the processor are included in a system-on-chip; and

generate an output frame based on the combined lines, the output frame having a first section corresponding to the first data lines of the combined lines and having a second section corresponding to the second data lines of the combined lines, wherein the first section and the second section are configured to be used to generate data in a three-dimensional (3D) image format or in a 3D video format.

17. The non-transitory computer readable-medium of claim 16 , wherein a synchronization between data streams from the first and second cameras received at the combiner occurs within a single line of the image data, and wherein a graphics processor is caused to:

process the output frame to generate 3D image data; and

send the 3D image data to a display device.

18. The non-transitory computer readable-medium of claim 16 , further comprising instructions that, when executed by the processor, cause the processor to store each combined line as one line of the image data in a separate row of a memory.

19. The non-transitory computer readable-medium of claim 18 , wherein instructions further cause the processor to provide a control signal to the first camera and to the second camera, and wherein the processor corresponds to a video front end.

20. An apparatus comprising:

means for receiving lines of image data at an input for a single camera, wherein each line of the image data is a combined line including a first data line from a first image captured by a first camera and a second data line from a second image captured by a second camera,

wherein the means for receiving lines is configured to generate an output frame based on the combined lines, the output frame having a first section corresponding to the first data lines of the combined lines and having a second section corresponding to the second data lines of the combined lines, wherein the first section and the second section are configured to be used to generate data in a three-dimensional (3D) image format or in a 3D video format; and

means for combining the first data lines and the second data lines, wherein the means for combining is coupled to the first camera and to the second camera, wherein the means for combining is configured to generate each line of the image data by receiving and aligning one first data line with one second data line to form one combined line, and wherein the means for combining and the means for receiving lines are included in a system-on-chip.

21. The apparatus of claim 20 , wherein the means for receiving is configured to provide a control signal to the first and second cameras, and wherein the apparatus further comprises:

means for processing the output frame to generate 3D image data; and

means for sending the 3D image data to a display device.

22. The apparatus of claim 20 , further comprising means for synchronizing sensors coupled to the first camera and to the second camera, wherein the first camera and the second camera are coupled to receive signals including a common startup or reset signal and a common clock input, and wherein the signals are provided to the first camera and to the second camera from the means for synchronizing sensors.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE RE-RECORD TO ADD U.S. PROVISIONAL APPLICATION 61/412,755 FILED NOVEMBER 11, 2010 AS PRIORITY INFORMATION. PREVIOUSLY RECORDED ON REEL 026144 FRAME 0687. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 17, 2011
From: GOMA, SERGIU R.; ALEKSIC, MILIVOJE; HWANG, HAU; CHEUNG, JOSEPH
To: QUALCOMM INCORPORATED
Reel/Frame 026460/0396 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 18, 2011
From: GOMA, SERGIU R.; ALEKSIC, MILIVOJE; HWANG, HAU; CHEUNG, JOSEPH
To: QUALCOMM INCORPORATED
Reel/Frame 026144/0687 →
Continuity (5)
Provisional Application 61320940 · Apr 5, 2010
Provisional Application 61324259 · Apr 14, 2010
Provisional Application 61359312 · Jun 28, 2010
Provisional Application 61412755 · Nov 11, 2010
Related Publication 20120081519A1 · Apr 5, 2012