Sparse scanout for image sensors
To support sparse scanout of an image sensor, an image data protocol such as the MIPI CSI protocol is extended with support for pixel coordinates in long packets. The receiver uses these to compute where in memory these pixels should be stored or where on a display they should be displayed. Truly sparse scanout is supported for any arbitrarily shaped image areas including for example elliptical readout for fisheye lenses. These techniques save MIPI and serializer/deserializer bandwidth in automotive and other applications, allowing for more cameras per vehicle or other host. Such techniques can be implemented in a way that is compatible with prior MIPI standardized approaches.
1. Image data transmission method comprising:
defining, within a readout image area, a scanout area that spans plural overlapping regions of interest;
determining a position of the scanout area within the readout image area; and
reading out less than all pixels of the readout image area, including transmitting, within a long packet of a serial image data transmission, (a) image data from the scanout area, the image data spanning the plural overlapping regions of interest, and (b) a pixel coordinate applying to all of the plural overlapping regions of interest, the pixel coordinate indicating the determined position of the scanout area within the readout image area.
2. The image data transmission method of claim 1 wherein the long packet further comprises a further coordinate indicating readout image area line number.
3. The image data transmission method of claim 1 further including embedding, in the long packet image data, region of interest information for the plural overlapping regions of interest corresponding to the defined scanout area.
4. The image data transmission method of claim 1 further including not transmitting portions of the readout image area outside the defined scanout area.
5. The image data transmission method of claim 1 wherein the defined scanout area is elliptical.
6. The image data transmission method of claim 1 wherein the defined scanout area is of arbitrary shape.
7. The image data transmission method of claim 1 further including subsequently transmitting a further long packet different from the first-mentioned long packet, the further long packet not including any pixel coordinate and containing image data to be located by reference to the pixel coordinate indicating the determined position included in the first-mentioned long packet.
8. A system for sparse scanout transmission comprising:
an image sensor comprising a set of pixels covering a readout image area, the image sensor being configured to select a non-rectangular scanout area smaller than the readout image area, —and to determine a starting position of a line corresponding to a portion of the selected non-rectangular scanout area within the readout image data that spans plural overlapping regions of interest; and
an interface configured to include, within a long packet of a serialized image data interface transmission, a coordinate indicating the starting position of the line, the coordinate applying to all of the plural overlapping regions of interest.
9. The system of claim 8 wherein the coordinate is paired with a further coordinate indicating a number of the line.
10. The system of claim 8 wherein the interface is configured to embed, within the long packet, region of interest information corresponding to the selected scanout area.
11. The system of claim 8 wherein the interface is configured to not transmit portions of the readout image area outside the selected scanout area.
12. The system of claim 8 wherein the selected scanout area is elliptical.
13. The system of claim 8 wherein the selected scanout area is of arbitrary shape.
14. The system of claim 8 wherein the interface is configured to subsequently transmit a further long packet not including a coordinate, the further long packet containing image data to be located by referencing a previously-transmitted coordinate.
15. A sparse scanout receiver comprising:
an interface configured to receive a long packet including a coordinate applying to plural overlapping regions of interest, the coordinate indicating a starting position of a line within a readout image array corresponding to scanout image data contained in the long packet, the scanout image data including pixels from each of the plural overlapping regions of interest;
the interface being further configured to receive a further long packet not including a coordinate indicating a starting position of a line corresponding to further scanout image data in the readout image array contained in the further long packet; and
a processor operatively coupled to the interface, the processor configured to apply the coordinate to a horizontal starting position of the further scanout image data while incrementing a line designator for the first-mentioned scanout image data to obtain a line designator for the further scanout image data.
16. In a MIPI CSI-2 protocol of a type including short packets and long packets, the long packets including a packet header (PH) indicating the length of a payload containing serialized image data,
an improvement comprising:
(a) including a pair of coordinates in at least some long packet payloads or in an extended packet header, the coordinate pair indicating an upper lefthand corner position in a planar coordinate space associated with serialized a scanout image data within one or more payloads; and
(b) including within the long packet headers an indicator that the packet contains a coordinate pair specifying a variable position of the scanout image data within a larger sensor readout image array, the coordinate pair applying in common to plural regions of interest contained within the serialized scanout image data.
17. The improvement of claim 16 further comprising embedding region of interest information in further long packets.
18. The improvement of claim 16 wherein only some long packet payloads contain coordinate pairs.