Camera system dual-encoder architecture
View Patent ↗An image capture accelerator performs accelerated processing of image data. In one embodiment, the image capture accelerator includes accelerator circuitry including a pre-processing engine and a compression engine. The pre-processing engine is configured to perform accelerated processing on received image data, and the compression engine is configured to compress processed image data received from the pre-processing engine. In one embodiment, the image capture accelerator further includes a demultiplexer configured to receive image data captured by an image sensor array implemented within, for example, an image sensor chip. The demultiplexer may output the received image data to an image signal processor when the image data is captured by the image sensor array in a standard capture mode, and may output the received image data to the accelerator circuitry when the image data is captured by the image sensor array in an accelerated capture mode.
1. A camera system, comprising:
an image sensor chip configured to produce image data representative of light incident upon the image sensor chip;
an image signal processor chip (“ISP”) configured to process the image data; and
an image capture accelerator chip (“ICA”) coupled between the image sensor chip and the ISP, the image capture accelerator comprising:
an input configured to receive the image data from the image sensor chip;
a decimator configured to decimate the received image data into a plurality of image sub-band components;
a wavelet encoder configured to encode a first subset of the plurality of image sub-band components;
an H.264 encoder configured to encode a second subset of the plurality of image sub-band components;
a concatenator configured to concatenate the encoded first subset of image sub-band components and the second subset of image sub-band components to produce concatenated encoded image sub-band components; and
an output configured to output the received image data when the ICA is configured to operate in a normal mode and to output the concatenated encoded image sub-band components when the ICA is configured to operate in an accelerated mode.
2. The camera system of claim 1 , wherein the wavelet encoder comprises an entropy encoder.
3. The camera system of claim 1 , wherein the first subset of image sub-band components comprises a high/low sub-band component, a low/high sub-band component, and a high/high sub-band component.
4. The camera system of claim 1 , wherein the second subset of image sub-band components comprises a low/low sub-band component.
5. The camera system of claim 1 , wherein the output is configured to output the received image data to the ISP when the ICA is configured to operate in the normal mode.
6. The camera system of claim 1 , wherein the output is configured to output the concatenated encoded image sub-band components to a memory for storage when the ICA is configured to operate in an accelerated mode.
7. The camera system of claim 6 , wherein the ICA further comprises:
one or more decoders configured to access the stored concatenated encoded image sub-band components from the memory, split the concatenated encoded image sub-band components into the encoded first subset of image sub-band components and the second subset of image sub-band components, and decode the encoded first subset of image sub-band components and the second subset of image sub-band components to produce the plurality of image sub-band components;
wherein the output is further configured to output the plurality of image sub-band components to the ISP after the one or more decoders produce the plurality of image sub-band components.
8. A method for capturing images by a camera system, comprising:
capturing, by an image sensor chip, light incident upon the image sensor to produce image data representative of the captured light;
decimating, by an image capture accelerator chip (“ICA”) coupled to the image sensor chip, the received image data into a plurality of image sub-band components;
encoding, by a wavelet encoder of the ICA, a first subset of the plurality of image sub-band components;
encoding, by an H.264 encoder of the ICA, a second subset of the plurality of image sub-band components;
concatenating the encoding first subset of image sub-band components and the second subset of image sub-band components to produce concatenated encoded image sub-band components;
when the ICA is configured to operate in a normal mode, outputting the received image data; and
when the ICA is configured to operate in an accelerated mode, outputting the concatenated encoded image sub-band components.
9. The method of claim 8 , wherein the wavelet encoder comprises an entropy encoder.
10. The method of claim 8 , wherein the first subset of image sub-band components comprises a high/low sub-band component, a low/high sub-band component, and a high/high sub-band component.
11. The method of claim 8 , wherein the second subset of image sub-band components comprises a low/low sub-band component.
12. The method of claim 8 , wherein outputting the received image data comprises outputting the received image data to an image signal processing chip (“ISP”) coupled to the ICA.
13. The method of claim 8 , wherein outputting the concatenated encoded image sub-band components comprises outputting the concatenated encoded image sub-band components to a memory for storage.
14. The method of claim 13 , further comprising:
accessing the stored concatenated encoded image sub-band components from the memory;
splitting the concatenated encoded image sub-band components into the encoded first subset of image sub-band components and the second subset of image sub-band components;
decoding the encoded first subset of image sub-band components and the second subset of image sub-band components to produce the plurality of image sub-band components; and
outputting the plurality of image sub-band components to the ISP decoding the encoded first subset of image sub-band components and the second subset of image sub-band components.
15. An image capture accelerator integrated circuit (“ICA”), comprising:
an input configured to receive image data captured by an image sensor chip;
a decimator configured to decimate the received image data into a plurality of image sub-band components;
a wavelet encoder configured to encode a first subset of the plurality of image sub-band components;
an H.264 encoder configured to encode a second subset of the plurality of image sub-band components;
a concatenator configured to concatenate the encoded first subset of image sub-band components and the second subset of image sub-band components to produce concatenated encoded image sub-band components; and
an output configured to output the received image data when the ICA is configured to operate in a normal mode and to output the concatenated encoded image sub-band components when the ICA is configured to operate in an accelerated mode.
16. The ICA of claim 15 , wherein the wavelet encoder comprises an entropy encoder.
17. The ICA of claim 15 , wherein the first subset of image sub-band components comprises a high/low sub-band component, a low/high sub-band component, and a high/high sub-band component.
18. The ICA of claim 15 , wherein the second subset of image sub-band components comprises a low/low sub-band component.
19. The ICA of claim 15 , wherein the output is configured to output the received image data to an image signal processor chip (“ISP”) when the ICA is configured to operate in the normal mode.
20. The ICA of claim 15 , wherein the output is configured to output the concatenated encoded image sub-band components to a memory for storage when the ICA is configured to operate in an accelerated mode, and further comprising:
one or more decoders configured to access the stored concatenated encoded image sub-band components from the memory, split the concatenated encoded image sub-band components into the encoded first subset of image sub-band components and the second subset of image sub-band components, and decode the encoded first subset of image sub-band components and the second subset of image sub-band components to produce the plurality of image sub-band components;
wherein the output is further configured to output the plurality of image sub-band components to an ISP after the one or more decoders produce the plurality of image sub-band components.