Systems and methods for performing direct conversion of image sensor data to image analytics
Systems and methods for performing direct conversion of image sensor data to image analytics are provided. One such system for directly processing sensor image data includes a sensor configured to capture an image and generate corresponding image data in a raw Bayer format, and a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw Bayer format. Systems and methods for training the CNN are provided, and may include a generative model that is configured to convert RGB images into estimated images in the raw Bayer format.
1. A system for directly processing sensor image data, the system comprising:
a sensor configured to capture an image and generate corresponding image data in a raw Bayer format; and
a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw Bayer format,
wherein the CNN is configured to perform at least one of image classification or objection detection directly from the image data in the raw Bayer format.
2. The system of claim 1 :
wherein the CNN was trained using a generative model configured to convert an RGB image into an estimated image in the raw Bayer format using a ground truth image in the raw Bayer format; and
wherein the generative model was trained using image data in the raw Bayer format and without using labels.
3. The system of claim 2 , wherein the generative model is configured to generate a labeled image dataset in the raw Bayer format.
4. The system of claim 3 , wherein the CNN was trained using the labeled image dataset in the raw Bayer format.
5. A method for directly processing sensor image data, the method comprising:
receiving image data in a raw Bayer format;
generating image analytics directly from the image data in the raw Bayer format; and
performing at least one of image classification or objection detection directly from the image data in the raw Bayer format.
6. An apparatus for directly processing sensor image data, the apparatus comprising:
a means for receiving image data in a raw Bayer format;
a means for generating image analytics directly from the image data in the raw Bayer format; and
means for performing at least one of image classification or objection detection directly from the image data in the raw Bayer format.
7. A system for directly processing sensor image data, the system comprising:
a sensor configured to capture an image and generate corresponding image data in a raw RGB format;
a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw RGB format; and
performing at least one of image classification or objection detection directly from the image data in the raw RGB format.
8. A system for directly processing sensor image data, the system comprising:
a sensor configured to capture an image and generate corresponding image data in a raw Bayer format; and
a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw Bayer format,
wherein the CNN was trained using a generative model configured to convert an RGB image into an estimated image in the raw Bayer format using a ground truth image in the raw Bayer format.
9. A system for directly processing sensor image data, the system comprising:
a sensor configured to capture an image and generate corresponding image data in a raw Bayer format; and
a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw Bayer format and without performing image signal pre-processing on the image data in the raw Bayer format.
10. A system for directly processing sensor image data, the system comprising:
a sensor configured to capture an image and generate corresponding image data in a raw Bayer format; and
a convolution neural network (CNN) coupled to the sensor and configured to generate image analytics directly from the image data in the raw Bayer format,
wherein the CNN is configured to generate inferences directly from the image data in the raw Bayer format.
11. The system of claim 1 , wherein the CNN was trained using a generative model configured to convert an RGB image into an estimated image in the raw Bayer format using a ground truth image in the raw Bayer format.
12. The system of claim 1 , wherein the CNN was trained using images in the raw Bayer format.
13. The system of claim 1 , wherein the CNN was trained using a labeled image dataset in the raw Bayer format.
14. The system of claim 10 , wherein the CNN was trained using a generative model configured to convert an RGB image into an estimated image in the raw Bayer format using a ground truth image in the raw Bayer format.
15. The system of claim 10 , wherein the CNN was trained using images in the raw Bayer format.
16. The system of claim 10 , wherein the CNN was trained using a labeled image dataset in the raw Bayer format.
17. A method for directly processing sensor image data, the method comprising:
receiving image data in a raw Bayer format;
generating, using a convolutional neural network (CNN), image analytics directly from the image data in the raw Bayer format; and
generating, using the CNN, inferences directly from the image data in the raw Bayer format.
18. The method of claim 17 , wherein the generating image analytics and the generating inferences is performed by the CNN trained using a generative model configured to convert an RGB image into an estimated image in the raw Bayer format using a ground truth image in the raw Bayer format.
19. The method of claim 17 , wherein the generating image analytics and the generating inferences is performed by the CNN trained using images in the raw Bayer format.
20. The method of claim 17 , wherein the generating image analytics and the generating inferences is performed by the CNN trained using a labeled image dataset in the raw Bayer format.