Camera system transmission in bandwidth constrained environments
View Patent ↗An image capture accelerator performs accelerated processing of image data. In one embodiment, the image capture accelerator includes accelerator circuitry including a pre-processing engine and a compression engine. The pre-processing engine is configured to perform accelerated processing on received image data, and the compression engine is configured to compress processed image data received from the pre-processing engine. In one embodiment, the image capture accelerator further includes a demultiplexer configured to receive image data captured by an image sensor array implemented within, for example, an image sensor chip. The demultiplexer may output the received image data to an image signal processor when the image data is captured by the image sensor array in a standard capture mode, and may output the received image data to the accelerator circuitry when the image data is captured by the image sensor array in an accelerated capture mode.
1. A camera system, comprising:
an image sensor chip configured to produce image data representative of light incident upon the image sensor chip;
an image capture accelerator chip (“ICA”) comprising:
an input configured to receive the image data from the image sensor chip;
a compression engine configured to decimate the received image data into a plurality of image sub-band components; and
one or more outputs configured to output the image data and a set of the plurality of image sub-band components; and
an image signal processor chip (“ISP”) with a first input and a second input, the ICA coupled between the image sensor chip and the ISP, the ISP configured to detect an amount of output bandwidth available to the camera system, the ISP comprising:
one or more inputs configured to receive the image data and the set of image sub-band components;
an encoder configured to encode the received image data and the set of image sub-band components; and
an output configured to select and output either the encoded image data or one or more encoded image sub-band components based on the detected amount of output bandwidth.
2. The camera system of claim 1 , wherein the output selects the encoded image data in response to the detected amount of output bandwidth exceeding a threshold.
3. The camera system of claim 1 , wherein the output selects one or more of the encoded image sub-band components in response to the detected amount of output bandwidth falling below a threshold.
4. The camera system of claim 3 , wherein the selected one or more of the encoded image sub-band components comprises a low/low sub-band component.
5. The camera system of claim 4 , wherein the low/low sub-band component is selected from a set of low/low sub-band components within the set of image sub-band components comprising a first low/low sub-band component of a first resolution and a second low/low sub-band component of a second resolution lower than the first resolution, wherein the low/low sub-band component comprises the first low/low sub-band component in response to the detected amount of output bandwidth comprising a first bandwidth, and wherein the low/low sub-band component comprises the second low/low sub-band component in response to the detected amount of output bandwidth comprising a second bandwidth lower than the first bandwidth.
6. The camera system of claim 1 , wherein the output outputs an encoded low/low image sub-band component, and subsequently outputs additional image sub-band components in response to additional output bandwidth being available to the camera system.
7. The camera system of claim 1 , wherein the detected amount of output bandwidth available to the camera system comprises a first bandwidth in response to determining that a wireless connection is available to the camera system, and comprises a second bandwidth greater than the first bandwidth in response to determining that a wired connection is available to the camera system.
8. A method for capturing image data by a camera system, comprising:
capturing, by an image sensor chip, light incident upon the image sensor chip to produce image data representative of the captured light;
receiving, by an image capture accelerator chip (“ICA”), the image data from the image sensor chip;
decimating, by the ICA, the received image data into a plurality of image sub-band components;
outputting, by the ICA, the received image data and the plurality of image sub-band components;
encoding, by an image signal processor chip (“ISP”), the received image data and a set of image sub-band components; and
outputting, by the ISP, either the encoded image data or one or more encoded image sub-band components based on a detected amount of output bandwidth available to the camera system.
9. The method of claim 8 , wherein the encoded image data is outputted in response to the detected amount of output bandwidth exceeding a threshold.
10. The method of claim 8 , wherein one or more of the encoded image sub-band components are outputted in response to the detected amount of output bandwidth falling below a threshold.
11. The method of claim 10 , wherein the outputted one or more of the encoded image sub-band components comprises a low/low sub-band component.
12. The method of claim 11 , wherein the low/low sub-band component is selected from a set of low/low sub-band components within the set of image sub-band components comprising a first low/low sub-band component of a first resolution and a second low/low sub-band component of a second resolution lower than the first resolution, wherein the low/low sub-band component comprises the first low/low sub-band component in response to the detected amount of output bandwidth comprising a first bandwidth, and wherein the low/low sub-band component comprises the second low/low sub-band component in response to the detected amount of output bandwidth comprising a second bandwidth lower than the first bandwidth.
13. The method of claim 8 , wherein an encoded low/low image sub-band component is outputted, and wherein additional image sub-band components are subsequently outputted in response to additional output bandwidth being available to the camera system.
14. The method of claim 8 , wherein the detected amount of output bandwidth available to the camera system comprises a first bandwidth in response to determining that a wireless connection is available to the camera system, and comprises a second bandwidth greater than the first bandwidth in response to determining that a wired connection is available to the camera system.
15. An image signal processor integrated circuit (“ISP”), comprising:
a bandwidth detector configured to determine an amount of output bandwidth available to the ISP;
a first input configured to receive image data representative of light incident upon an image sensor chip;
a second input configured to receive, from an image capture accelerator chip (“ICA”) a set of image sub-band components comprising a decimation of the received image data;
an encoder configured to encode the received image data and the set of image sub-band components; and
an output configured to select and output either the encoded image data or one or more encoded image sub-band components based on the determined amount of output bandwidth.
16. The ISP of claim 15 , wherein the output selects the encoded image data in response to the determined amount of output bandwidth exceeding a threshold.
17. The ISP of claim 15 , wherein the output selects one or more of the encoded image sub-band components in response to the determined amount of output bandwidth falling below a threshold.
18. The ISP of claim 17 , wherein the selected one or more of the encoded image sub-band components comprises a low/low sub-band component.
19. The ISP of claim 18 , wherein the low/low sub-band component is selected from a set of low/low sub-band components within the set of image sub-band components comprising a first low/low sub-band component of a first resolution and a second low/low sub-band component of a second resolution lower than the first resolution, wherein the low/low sub-band component comprises the first low/low sub-band component in response to the determined amount of output bandwidth comprising a first bandwidth, and wherein the low/low sub-band component comprises the second low/low sub-band component in response to the determined amount of output bandwidth comprising a second bandwidth lower than the first bandwidth.
20. The ISP of claim 15 , wherein the output outputs an encoded low/low image sub-band component, and subsequently outputs additional image sub-band components in response to additional output bandwidth being available to the ISP.