Generation of a disparity result with low latency
A powerful, scaleable, and reconfigurable image processing system and method of processing data therein is described. This general purpose, reconfigurable engine with toroidal topology, distributed memory, and wide bandwidth I/O are capable of solving real applications at real-time speeds. The reconfigurable image processing system can be optimized to efficiently perform specialized computations, such as real-time video and audio processing. This reconfigurable image processing system provides high performance via high computational density, high memory bandwidth, and high I/O bandwidth. Generally, the reconfigurable image processing system and its control structure include a homogeneous array of 16 field programmable gate arrays (FPGA) and 16 static random access memories (SRAM) arranged in a partial torus configuration. The reconfigurable image processing system also includes a PCI bus interface chip, a clock control chip, and a datapath chip. It can be implemented in a single board. It receives data from its external environment, computes correspondence, and uses the results of the correspondence computations for various post-processing industrial applications. The reconfigurable image processing system determines correspondence by using non-parametric local transforms followed by correlation. These non-parametric local transforms include the census and rank transforms. Other embodiments involve a combination of correspondence, rectification, a left-right consistency check, and the application of an interest operator.
1. A method of generating disparity results with low latency in an image processing system that processes elements of a first set of image data and a second set of image data, each having a plurality of elements, comprising:
generating, using a processor, a disparity result for a first element before all elements of the first data set and the second data set have been received.
2. The method of claim 1 , further comprising recognizing a hand gesture in order to control an application by tracking a location and an orientation of a user's hand based at least in part on the disparity result.
3. The method of claim 2 , wherein the application includes providing instructions for a visual display.
4. The method of claim 1 , further comprising alerting a person entering a controlled zone to a danger or redirecting hazardous activity around the person based at least in part on the disparity result.
5. The method of claim 1 , further comprising detecting a position of one or more vehicles and/or one or more pedestrians for a traffic monitoring application based at least in part on the disparity result.
6. The method of claim 1 , further comprising determining whether an object associated with the first element is moving by calculating a motion associated with the object as the object appears in a first image and in a second image taken at different times.
7. A system for generating disparity results with low latency in an image processing system that processes elements of a first set of image data and a second set of image data, each having a plurality of elements, comprising:
a processor configured to generate a disparity result for a first element before all elements of the first data set and the second data set have been received; and
an input interface coupled to the processor and configured to receive the first set of image data and the second set of image data; and
an output interface coupled to the processor and configured to provide the disparity result.
8. A method of generating a disparity result with low latency in a data processing system that processes elements of a first set of data and a second set of data, each having a plurality of elements, comprising:
performing a transformation on a first subset of one or more elements and a second subset of one or more elements of a first data set;
performing a transformation on a third subset of one or more elements and a fourth subset of one or more elements of a second data set;
generating a first correlation result by correlating a transformed first subset of one or more elements with a transformed third subset of one or more elements; and
generating a second correlation result, concurrently with generating the first correlation result, by correlating a transformed second set of one or more elements with a transformed fourth subset of one or more elements, and
generating a disparity result based at least in part on the first correlation result and the second correlation result.
9. The method of claim 8 , wherein the first subset and the second subset contain the same elements.
10. The method of claim 8 , further comprising recognizing a hand gesture in order to control an application by tracking a location and an orientation of a user's hand based at least in part on the correlation result.
11. The method of claim 10 , wherein the application includes providing instructions for a visual display.
12. The method of claim 8 , further comprising alerting a person entering a controlled zone to a danger or redirecting hazardous activity around the person based at least in part on the correlation result.
13. The method of claim 8 , further comprising detecting a position of one or more vehicles and/or one or more pedestrians for a traffic monitoring application based at least in part on the correlation result.
14. The method of claim 8 , further comprising determining whether an object associated with the transformed first subset of one or more elements is moving by calculating a motion associated with the object as the object appears in a first image and in a second image taken at different times.
15. A system for generating a disparity result with low latency in a data processing system that processes elements of a first set of data and a second set of data, each having a plurality of elements, comprising:
a processor configured to:
perform a transformation on a first subset of one or more elements and a second subset of one or more elements of a first data set;
perform a transformation on a third subset of one or more elements and a fourth subset of one or more elements of a second data set;
generate a first correlation result by correlating a transformed first subset of one or more elements with a transformed third subset of one or more elements; and
generate a second correlation result, concurrently with generating the first correlation result, by correlating a transformed second set of one or more elements with a transformed fourth subset of one or more elements;
generate a disparity result based at least in part on the first correlation result and the second correlation result; and
an input interface coupled to the processor and configured to receive the first set of data and the second set of data; and
an output interface coupled to the processor and configured to provide the disparity result.
16. The system of claim 15 , wherein an application processor is configured to recognize a hand gesture in order to control an application by tracking a location and an orientation of a user's hand based at least in part on the disparity result.
17. The system of claim 16 , wherein the application includes providing instructions for a visual display.
18. The system of claim 15 , wherein an application processor is configured to alert a person entering a controlled zone to a danger or redirecting hazardous activity around the person based at least in part on the disparity result.
19. The system of claim 15 , wherein an application processor is configured to detect a position of one or more vehicles and/or one or more pedestrians for a traffic monitoring application based at least in part on the disparity result.
20. The system of claim 15 , wherein an application processor is configured to determine whether an object associated with the first element is moving by calculating a motion associated with the object as the object appears in a first image and in a second image taken at different times.
21. The system of claim 15 , wherein the first subset and the second subset contain the same elements.
22. The system of claim 15 , wherein an application processor is configured to recognize a hand gesture in order to control an application by tracking a location and an orientation of a user's hand based at least in part on the correlation result.
23. The system of claim 22 , wherein the application includes providing instructions for a visual display.
24. The system of claim 15 , wherein an application processor is configured to alert a person entering a controlled zone to a danger or redirecting hazardous activity around the person based at least in part on the correlation result.
25. The system of claim 15 , wherein an application processor is configured to detect a position of one or more vehicles and/or one or more pedestrians for a traffic monitoring application based at least in part on the correlation result.
26. The system of claim 15 , wherein an application processor is configured to determine whether an object associated with the transformed first subset of one or more elements is moving by calculating a motion associated with the object as the object appears in a first image and in a second image taken at different times.