Images for perception modules of autonomous vehicles
Disclosed are devices, systems and methods for processing an image. In one aspect a method includes receiving an image from a sensor array including an x-y array of pixels, each pixel in the x-y array of pixels having a value selected from one of three primary colors, based on a corresponding x-y value in a mask pattern. The method may further include generating a preprocessed image by performing preprocessing on the image. The method may further include performing perception on the preprocessed image to determine one or more outlines of physical objects.
1. An apparatus comprising:
at least one processor and memory including executable instructions that when executed cause the at least one processor to perform operations comprising:
receiving an image including an array of pixels from a sensor array, each pixel in the array of pixels having a value corresponding to one of three primary colors selected based on a mask pattern;
generating a preprocessed image by performing preprocessing on the image, wherein the preprocessing excludes reconstruction of a full color image from incomplete color samples output from the sensor array overlaid with a color filter array for de-mosaicing; and
performing perception on the preprocessed image.
2. The apparatus of claim 1 , wherein the mask pattern is a Bayer pattern.
3. The apparatus of claim 2 , wherein the Bayer pattern is a red-green-green-blue pattern assigned to a 2-pixel by 2-pixel array repeated across a raw image.
4. The apparatus of claim 1 , wherein the mask pattern includes a greater number of green pixel values than both red and blue.
5. The apparatus of claim 1 , wherein the mask pattern is selected such that a value of every other pixel along a row in a reduced image corresponds to a green value of a raw image.
6. The apparatus of claim 1 , wherein the mask pattern is selected such that a value of every other pixel along a column in a reduced image corresponds to a green value of a raw image.
7. The apparatus of claim 1 , wherein the one of the three primary colors is selected based on a corresponding position value in the mask pattern, and wherein each position value in the mask pattern is from one of three possible values.
8. The apparatus of claim 1 , wherein the image is generated using one or more color-selective filters.
9. The apparatus of claim 1 , wherein the preprocessing is performed on the image from the sensor array without human perception image enhancement.
10. The apparatus of claim 9 , wherein the human perception image enhancement includes one or more of de-mosaicing, white balancing, and noise reduction.
11. The apparatus of claim 1 , wherein the preprocessing excludes scaling for white balancing.
12. The apparatus of claim 1 , wherein the preprocessing excludes a reduction of salt and pepper noise, or another reduction of Gaussian noise.
13. The apparatus of claim 1 , wherein the performing the perception on the preprocessed image determines one or more outlines of objects.
14. A non-transitory computer readable medium having code stored thereon, the code, when executed by a processor, causing the processor to implement a method comprising:
receiving an image including an array of pixels from a sensor array, each pixel in the array of pixels having a value corresponding to one of three primary colors selected based on a mask pattern;
generating a preprocessed image by performing preprocessing on the image, wherein the preprocessing excludes reconstruction of a full color image from incomplete color samples output from the sensor array overlaid with a color filter array for de-mosaicing; and
performing perception on the preprocessed image.
15. The non-transitory computer readable medium of claim 14 , wherein the mask pattern is a Bayer pattern, and wherein the Bayer pattern is a red-green-green-blue pattern assigned to a 2-pixel by 2-pixel array repeated across a raw image.
16. The non-transitory computer readable medium of claim 14 , wherein the one of the three primary colors is selected based on a corresponding position value in the mask pattern, and wherein each position value in the mask pattern is from one of three possible values.
17. A method for processing an image, comprising:
receiving an image including an array of pixels from a sensor array, each pixel in the array of pixels having a value corresponding to one of three primary colors selected based on a mask pattern;
generating a preprocessed image by performing preprocessing on the image, wherein the preprocessing excludes reconstruction of a full color image from incomplete color samples output from the sensor array overlaid with a color filter array for de-mosaicing; and
performing perception on the preprocessed image.
18. The method of claim 17 , wherein the mask pattern is a Bayer pattern.
19. The method of claim 18 , wherein the Bayer pattern is a red-green-green-blue pattern assigned to a 2-pixel by 2-pixel array repeated across a raw image.
20. The method of claim 17 , wherein the one of the three primary colors is selected based on a corresponding position value in the mask pattern, and wherein each position value in the mask pattern is from one of three possible values.