Image synthesis method with DSP and GPU
This patent application focuses on partitioning the computation to perform the image processing on the DSP, communicate the results to the GPU in an optimized manner, and use the GPU to render 3D information to the display. This invention solves the problem of seamlessly merging live streams from plural automotive cameras into a 3D display that can be rotated to see different view points. This purpose is achieved by partitioning the computation task between a digital signal processor (DSP) for image processing and a graphics processing unit (GPU) for rendering and display. This invention also includes meta data structures for communication of results from the DSP to the GPU.
1. A method of data processing comprising steps of:
receiving a video image feed from each of a plurality of cameras having differing views of a scene;
dewarping each video image feed from a corresponding received video image to a predetermined display surface;
adjusting lighting of each dewarped video image to a uniform lighting level;
wherein said steps of receiving, dewarping, adjusting and synthesizing are performed via a digital signal processor;
synthesizing a combined image from the lighting adjusted video images into a single combined video image;
vertex processing;
pixel processing;
blending;
forming a birds eye surround view;
said steps of vertex processing, pixel processing, blending, shading and color conversion, and forming a birds eye surround view are performed on a graphics processing unit; and
further comprising a step of forming an index buffer data structure to transmit data from the digital signal processor to the graphics processing unit mapping per-pixel for the whole buffer as defined below:
typedef struct _gridpoint
{
short x;
/1. X coordinate, signed 16bit
short y;
/2. Y coordinate, signed 16bit
q12_4_t z;
/3. Z coordinate (Q12.4
format)
unsigned short texelY;
/4. Input image gridpoint
location vertical
unsigned short texelX;
/5. Input image gridpoint
location horizontal
unsigned short imageindex1;/6. Input image index for
blend layer1 (i.e 1
- 4)
unsigned char alpha1;
/ Alpha for layer 1
unsigned short imageindex2;/6. Input image index for
blend layer2 (i.e 1
- 4)
unsigned char alpha2;
/ Alpha for layer 2 (including
priority)
unsigned char coloradjust;
/ Color adjustment factor
}gridpoint.
2. The method of claim 1 , wherein:
said steps of receiving, dewarping, adjusting and synthesizing performed via a digital signal processor and said steps of vertex processing, pixel processing, blending, and forming a birds eye surround view on a graphics processing unit are controlled by varying a mesh count depending on digital signal processor and graphics processing unit computational capacities.
3. The method of claim 1 , further comprising:
generating attribute arrays from per-pixel information having an index array, texture coordinates and image ID for a given camera view point and mesh.