Display device and system
An image processing engine and method of forming a hologram of a target image for projection using data streaming. An input or primary image is sub-sampled using a kernel and the secondary image output used to generate a hologram of the target image. A technique of kernel sub-sampling using a plurality of two or more data streams provides improvements in efficiency, including reduced data storage requirements and increased processing speed.
1. An image processor arranged to generate a secondary image by under-sampling a primary image using a kernel having m rows and n columns of kernel values, wherein the kernel has a plurality of kernel sampling positions for each row of the primary image, each kernel sampling position for a row separated by a stride distance of x pixels, wherein the image processing engine comprises a data streaming engine configured to:
form a first data stream of pixel values, wherein the first data stream is formed by reading image pixel values of the primary image row by row, each pixel value corresponding to a row position and a column position within the kernel;
form a second data stream of kernel values, each kernel value corresponding to a row position and a column position within the kernel, and
synchronise the pixel values of the first data stream with the kernel values of the second data stream so that each pixel value is paired with a respective kernel value corresponding to the same row position and column position within the kernel.
2. The image processor as claimed in claim 1 wherein there is a one-to-many correlation between the pixel values of the primary image in the first data stream and the kernel values of the second data stream.
3. The image processor as claimed in claim 1 wherein the data streaming engine is configured to form the second data stream using the steps:
(i) repeatedly reading the kernel values of a first row of the kernel the plurality of times;
(ii) repeatedly reading the kernel values of a next row of the kernel the plurality of times;
(iii) iteratively repeating step (ii) (m-2) times;
(iv) returning to step (i); and
(v) stopping steps (i) to (iv) when there are no more pixel values in the first data stream.
4. The image processor as claimed in claim 1 wherein each row of kernel values of the kernel in the second data stream is paired with a plurality of rows of image pixels of the primary image in the first data stream.
5. The image processor as claimed in claim 1 further comprising a buffer, wherein the image processing engine is further configured to:
receive, in sequence, synchronized pairs of image pixel values and kernel values of the first and second data streams from the data streaming engine;
process each pixel value of the first data stream with its paired kernel value of the second data stream, and
accumulate the processed pixel values for each kernel sampling position for storage in the buffer.
6. The image processor as claimed in claim 5 configured to:
process the pixel values of a first row of the primary image in the first data stream using the steps:
(a) multiplying each pixel value with its paired kernel value of the second data stream to determine a sequence of corresponding weighted pixel values,
(b) summing the n weighted pixel values for each kernel sampling position of a first plurality of kernel sampling positions,
(c) determining the accumulated weighted pixel values for each of the first plurality of kernel sampling positions, and
(d) storing the accumulated weighted pixel values for each of the first plurality of kernel sampling positions in consecutive storage locations in the buffer so as to form a sequence of partial pixel values of a secondary image in the buffer, and
iteratively repeat steps (a) to (d) to process the pixel values for each subsequent row of the primary image in the first data stream.
7. The image processor as claimed in claim 6 configured to:
iteratively repeat steps (a) to (d) to process the pixel values of (m-1) subsequent rows of the primary image in the first data stream to determine an accumulated weighted complete pixel value for each of the first plurality of kernel sampling positions, and
processing subsequent consecutive sets of m rows of pixel values of the primary image in the first data stream using steps (a) to (d) for each kernel sampling position of a further pluralities of kernel sampling positions.
8. The image processor as claimed in claim 6 further configured to process the pixel values of each row of the primary image in the first data stream using the step:
(e) feeding-back, from the buffer, a third data stream comprising the sequence of partial pixel values of the secondary image for use in processing the pixel values of the next row of the primary image in the first data stream.
9. The image processor as claimed in claim 8 configured to determine the accumulated weighted pixel values for each kernel sampling position in (c) by determining the sum of:
the n weighted pixel values determined in (b) for the kernel sampling position, and
the corresponding partial secondary image pixel value of the third data stream for the kernel sampling position contained in the feedback in (e).
10. The image processor as claimed in claim 6 further configured to:
(f)output from the buffer a final secondary image pixel value corresponding to each kernel sampling window position of each of the pluralities of kernel sampling window positions.
11. The image processor as claimed in claim 1 wherein the stride distance in the x direction is n pixels, and wherein the kernel is moved in a raster scan path in which the stride distance in the y direction is m pixels so that the kernel sampling sub-samples contiguous arrays of m×n pixels of the primary image.
12. A method for generating a secondary image by under-sampling a primary image using a kernel having m rows and n columns of kernel values, wherein the kernel has a plurality of kernel sampling positions for each row of the primary image, each kernel sampling position for a row separated by a stride distance of x pixels, the method comprising:
forming a first data stream of pixel values, wherein the first data stream is formed by reading image pixel values of the primary image row-by-row, each pixel value corresponding to a row position and a column position within the kernel;
forming a second data stream of kernel values, each kernel value corresponding to a row position and a column position within the kernel, and
synchronizing the pixel values of the first data stream with the kernel values of the second data stream so that each pixel value is paired with a respective kernel value corresponding to the same row position and column position within the kernel.
13. The method as claimed in claim 12 wherein forming the second data stream comprises:
(i) repeatedly reading the kernel values of a first row of the kernel the plurality of times;
(ii) repeatedly reading the kernel values of a next row of the kernel the plurality of times;
(iii) iteratively repeating step (ii) (m-2) times;
(iv) returning to step (i), and
(v) stopping steps (i) to (iv) when there are no more pixel values in the first data stream.
14. The method as claimed in claim 12 further comprising pairing each row of kernel values of the kernel in the second data stream with a plurality of rows of image pixels of the primary image in the first data stream.
15. The method as claimed in claim 12 further comprising:
receiving, in sequence, synchronized pairs of image pixel values and kernel values of the first and second data streams from the data streaming engine;
processing each pixel value of the first data stream with its paired kernel value of the second data stream;
accumulating the processed pixel values for each kernel sampling position, and
storing each of the accumulated values at a corresponding storage location in a buffer.
16. The method as claimed in claim 15 further comprising:
processing the pixel values of a first row of the primary image in the first data stream using the steps:
(a) multiplying each pixel value with its paired kernel value of the second data stream to determine a sequence of corresponding weighted pixel values,
(b) summing the n weighted pixel values for each kernel sampling position of a first plurality of kernel sampling positions,
(c) determining the accumulated weighted pixel values for each of the first plurality of kernel sampling positions, and
(d) storing the accumulated weighted pixel values for each of the first plurality of kernel sampling positions in consecutive storage locations in the buffer so as to form a sequence of partial pixel values of a secondary image in the buffer, and
iteratively repeating steps (a) to (d) to process the pixel values for each subsequent row of the primary image in the first data stream.
17. The method as claimed in claim 16 further comprising:
iteratively repeating steps (a) to (d) to process the pixel values of (m-1) subsequent rows of the primary image in the first data stream to determine an accumulated weighted complete pixel value for each of the first plurality of kernel sampling positions, and
processing subsequent consecutive sets of m rows of pixel values of the primary image in the first data stream using steps (a) to (d) for each kernel sampling position of a further pluralities of kernel sampling positions.
18. The method as claimed in claim 16 wherein processing the pixel values of each row of the primary image in the first data stream comprises:
(e) feeding-back, from the buffer, a third data stream comprising the sequence of partial pixel values of the secondary image for use in processing the pixel values of the next row of the primary image in the first data stream.
19. The method as claimed in claim 18 wherein determining the accumulated weighted pixel values for each kernel sampling position in (c) comprises determining the sum of:
the n weighted pixel values determined in (b) for the kernel sampling position, and
the corresponding partial secondary image pixel value of the third data stream for the kernel sampling position contained in the feedback in (e).
20. The method as claimed in claim 16 further comprising:
(e) outputing from the buffer a final secondary image pixel value corresponding to each sampling window position of each of the pluralities of sampling window positions.
21. The method as claimed in claim 12 , wherein each kernel value is a weighting value.