Granted Patent B2

US 12,520,045 · App. 17/810,213 · Granted Jan 6, 2026

Inventors: Hamid R. Sheikh (Allen, TX); Youngjun Yoo (Plano, TX); John Seokjun Lee (Allen, TX); Michael O. Polley (Garland, TX)

Assignee: Samsung Electronics Co., Ltd.

H04N23/698H04N5/2621H04N23/45H04N23/667H04N23/695

Abstract

Claims (70)

1 . A method for operating an electronic device, the method comprising:

capturing, by an image sensor of each of multiple camera modules, a stream from a pixel array of the image sensor, each of the camera modules comprising at least one image signal processing (ISP) circuit and positioned on a camera printed circuit board (PCB) that is electrically coupled to a main PCB of the electronic device, wherein the camera modules are configured to be synchronized by a synchronization signal and capture time-synchronized streams, one of the camera modules configured to generate the synchronization signal;

performing, by each of the camera modules, remosaicing of the stream to generate a converted stream;

applying, by each of the camera modules, scaling to the converted stream in order to produce a preview stream;

using, by each of the camera modules, the converted stream as a full frame stream;

compressing, by each of the camera modules, the preview stream using a first compression and the full frame stream directly after the remosaicing using a second compression;

outputting, by each of the camera modules, the compressed preview stream and the compressed full frame stream;

receiving and decompressing, by an application processor of the electronic device, the compressed preview streams and the compressed full frame streams, the application processor external to the camera modules; and

generating, from the decompressed preview streams, auto-focus, auto-exposure, and auto-white-balance (3A) outputs and multi-frame high dynamic range outputs; wherein the preview streams and the full frame streams are continuously processed, compressed, and output; and

wherein each camera module includes an image sensor array comprising a first tiltable image sensor module hingedly connected to a second tiltable image sensor module, the image sensor array configured to operate in:

a first mode in which (i) a first optical axis of the first tiltable image sensor module and a second optical axis of the second tiltable image sensor module are substantially perpendicular to a surface of the electronic device and (ii) the first and second tiltable image sensor modules are within a thickness profile of the electronic device; and

a second mode in which (i) the first optical axis of the first tiltable image sensor module and the second optical axis of the second tiltable image sensor module are not perpendicular to the surface of the electronic device and (ii) at least part of the first and second tiltable image sensor modules are no longer within the thickness profile of the electronic device.

2 . The method of claim 1 , wherein the preview streams have a lower resolution than the full frame streams.

3 . The method of claim 1 , wherein the full frame streams have a lower frame rate than the preview streams.

4 . The method of claim 1 , wherein the preview streams and the full frame streams are compressed simultaneously.

5 . The method of claim 1 , wherein the compressed preview streams and the compressed full frame streams are output on separate virtual channels.

6 . The method of claim 1 , further comprising:

generating video streams from the converted streams.

7 . The method of claim 1 , further comprising:

converting the decompressed preview streams into output preview streams; and

converting the decompressed full frame streams into output full frame streams.

8 . An apparatus comprising:

a camera printed circuit board (PCB) configured to be electrically coupled to a main PCB;

multiple camera modules positioned on the camera PCB, the camera modules configured to be synchronized by a synchronization signal generated by one of the camera modules and capture time-synchronized streams, each of the camera modules comprising:

an image sensor including a pixel array configured to generate a stream of image data; and

at least one image signal processing (ISP) circuit;

wherein each camera module is configured to:

capture the stream using the pixel array;

perform remosaicing of the stream to generate a converted stream;

apply scaling to the converted stream in order to produce a preview stream;

use the converted stream as a full frame stream;

compress the preview stream using a first compression and the full frame stream directly after the remosaicing using a second compression; and

output the compressed preview stream and the compressed full frame stream; and

an application processor external to the camera modules and configured to: receive and decompress the compressed preview streams and the compressed full frame streams; and

generate, from the decompressed preview streams, auto-focus, auto-exposure, and auto-white-balance (3A) outputs and multi-frame high dynamic range outputs;

wherein the camera modules are configured to continuously process, compress, and output the preview streams and the full frame streams; and

wherein each camera module includes an image sensor array comprising a first tiltable image sensor module hingedly connected to a second tiltable image sensor module, the image sensor array configured to operate in:

a first mode in which (i) a first optical axis of the first tiltable image sensor module and a second optical axis of the second tiltable image sensor module are substantially perpendicular to a surface of the apparatus and (ii) the first and second tiltable image sensor modules are within a thickness profile of the apparatus; and

a second mode in which (i) the first optical axis of the first tiltable image sensor module and the second optical axis of the second tiltable image sensor module are not perpendicular to the surface of the apparatus and (ii) at least part of the first and second tiltable image sensor modules are no longer within the thickness profile of the apparatus.

9 . The apparatus of claim 8 , wherein the preview streams have a lower resolution than the full frame streams.

10 . The apparatus of claim 8 , wherein the full frame streams have a lower frame rate than the preview streams.

11 . The apparatus of claim 8 , wherein the camera modules are configured to compress the preview streams and the full frame stream simultaneously.

12 . The apparatus of claim 8 , wherein the camera modules are configured to output the compressed preview streams and the compressed full frame streams on separate virtual channels.

13 . The apparatus of claim 8 , wherein the camera modules are further configured to generate video streams from the converted streams.

14 . The apparatus of claim 8 , further comprising:

a virtual channel demultiplexer configured to separate the compressed preview streams and the compressed full frame streams from respective streams of still images or image frames; and

an image signal processor configured to:

convert the decompressed preview streams into output preview streams; and

convert the decompressed full frame streams into output full frame streams.

15 . A non-transitory computer readable medium containing instructions that when executed cause an electronic device to:

obtain, using an image sensor of each of multiple camera modules, a stream of image data from a pixel array of the image sensor, each of the camera modules comprising at least one image signal processing (ISP) circuit and positioned on a camera printed circuit board (PCB) that is electrically coupled to a main PCB of the electronic device, wherein the camera modules are configured to be synchronized by a synchronization signal and capture time-synchronized streams, one of the camera modules configured to generate the synchronization signal;

perform, using each of the camera modules, remosaicing of the stream to generate a converted stream;

apply, using each of the camera modules, scaling to the converted stream in order to produce a preview stream;

use, by each of the camera modules, the converted stream as a full frame stream;

compress, using each of the camera modules, the preview stream using a first compression and the full frame stream directly after the remosaicing using a second compression;

output, from each of the camera modules, the compressed preview stream and the compressed full frame stream;

receive and decompress, using an application processor of the electronic device, the compressed preview streams and the compressed full frame streams, the application processor external to the camera modules; and

generate, from the decompressed preview streams, auto-focus, auto-exposure, and auto-white-balance (3A) outputs and multi-frame high dynamic range outputs;

wherein the instructions when executed cause the electronic device to continuously process, compress, and output the preview streams and the full frame streams; and

wherein each camera module includes an image sensor array comprising a first tiltable image sensor module hingedly connected to a second tiltable image sensor module, the image sensor array configured to operate in:

a first mode in which (i) a first optical axis of the first tiltable image sensor module and a second optical axis of the second tiltable image sensor module are substantially perpendicular to a surface of the electronic device and (ii) the first and second tiltable image sensor modules are within a thickness profile of the electronic device; and

a second mode in which (i) the first optical axis of the first tiltable image sensor module and the second optical axis of the second tiltable image sensor module are not perpendicular to the surface of the electronic device and (ii) at least part of the first and second tiltable image sensor modules are no longer within the thickness profile of the electronic device.

16 . The non-transitory computer readable medium of claim 15 , wherein the preview streams have a lower resolution than the full frame streams.

17 . The non-transitory computer readable medium of claim 15 , wherein the full frame streams have a lower frame rate than the preview streams.

18 . The non-transitory computer readable medium of claim 15 , wherein the instructions that when executed cause the electronic device to compress the preview streams and the full frame streams comprise:

instructions that when executed cause the electronic device to compress the preview streams and the full frame streams simultaneously.

19 . The non-transitory computer readable medium of claim 15 , wherein the instructions that when executed cause the electronic device to output the compressed preview streams and the compressed full frame streams comprise:

instructions that when executed cause the electronic device to output the compressed preview streams and the compressed full frame streams on separate virtual channels.

20 . The non-transitory computer readable medium of claim 15 , further containing instructions that when executed cause the electronic device to:

generate video streams from converted streams.

Assignments (1)

Continuity (3)

References Cited (52)