Image sensor module and method of operating the same
An image sensor module includes an image sensor configured to generate image data; a memory including a processor in memory (PIM) circuit, of the PIM circuit including banks configured to store the image data and a plurality of processing elements; and a signal processor, wherein the memory is configured to read the image data from the banks, perform a first image processing operation on the image data by using the processing elements corresponding to the banks, and store image-processed image data on which the first image processing operation is performed in the banks or output the image-processed image data.
1. An image sensor module comprising:
an image sensor configured to generate image data;
a memory including a processor in memory (PIM) circuit and banks configured to store the image data, the PIM circuit including a plurality of processing elements; and
a signal processor,
wherein the memory is configured to
read the image data from the banks,
perform a first image processing operation on the image data by using the processing elements corresponding to the banks, and
store image-processed image data on which the first image processing operation is performed in the banks or output the image-processed image data,
wherein at least some of the plurality of processing elements are connected through a data line.
2. The image sensor module of claim 1 , wherein, from among the banks, a first bank is connected to a first processing element from among the processing elements, and
the first processing element performs calculation processing corresponding to the first image processing operation based on data read out from the first bank.
3. The image sensor module of claim 2 , wherein, from among the banks, a second bank is adjacent to the first bank and connected to a second processing element from among the processing elements, and
the first processing element is connected to the second processing element through a data line.
4. The image sensor module of claim 3 , wherein the first processing element performs calculation processing corresponding to the first image processing operation based on data read out from the first bank and at least a portion of data read out from the second bank.
5. The image sensor module of claim 4 , wherein the first processing element includes arithmetic logic units (ALUs) corresponding to bit lines of the first bank, and
the second processing element comprises ALUs corresponding to bit lines of the second bank.
6. The image sensor module of claim 5 , wherein, from among the ALUs of the first processing element, a first ALU adjacent to the second bank is connected to a second ALU adjacent to the first bank from among the ALUs of the second processing element through a data line.
7. The image sensor module of claim 3 , wherein the memory includes a local bus for transmission and reception of data between the banks and the processing elements.
8. The image sensor module of claim 7 , wherein, from among the banks, a third bank stores weight information respectively corresponding to the processing elements and transmits the weight information to at least one of the processing elements through the local bus.
9. The image sensor module of claim 7 , wherein the memory reads out data stored in a fourth bank from among the banks and transmits read out data to the first processing element or the second processing element through the local bus.
10. The image sensor module of claim 1 , wherein the first image processing operation is neural network-based calculation processing.
11. The image sensor module of claim 10 , wherein the processing elements perform at least one of a convolution operation and a pooling operation from among neural network-based calculation processings.
12. The image sensor module of claim 10 , wherein the first image processing operation includes at least one of denoising, demosaicing, or remosaicing.
13. A method of operating an image sensor module comprising an image sensor and a memory, the method comprising:
obtaining image data by using the image sensor;
storing the image data in banks included in the memory;
performing an image processing operation on the image data by using processing elements included in the memory and corresponding to the banks; and
storing image-processed image data in the memory or outputting the image-processed image data from the memory,
wherein at least some of the processing elements are connected through a data line.
14. The method of claim 13 , wherein the storing of the image data in the banks includes
dividing the image data into a plurality of image regions; and
storing the image regions in the banks.
15. The method of claim 14 , wherein the performing of the image processing operation includes
reading out the image regions stored in the banks; and
performing an image processing operation on the image regions by using the processing elements respectively connected to the banks.
16. The method of claim 15 , wherein the performing of the image processing operation on the image regions includes
exchanging data corresponding to edges of the image regions through a data line connected between the processing elements; and
performing an image processing operation on the image regions based on the image regions and exchanged data by using the processing elements respectively connected to the banks.
17. An image sensor module comprising:
an image sensor configured to generate image data; and
a memory configured to store the image data,
wherein the memory includes
a memory bank comprising a first bank and a second bank adjacent to each other;
a processor in memory (PIM) circuit comprising a first processing element connected to the first bank and a second processing element connected to the second bank; and
a local bus for transmission and reception of data between the memory bank and the PIM circuit,
wherein the first processing element and the second processing element are connected through a data line.
18. The image sensor module of claim 17 , wherein
the first processing element includes arithmetic logic units (ALUs) corresponding to bit lines of the first bank, and
the second processing element comprises ALUs corresponding to bit lines of the second bank.
19. The image sensor module of claim 18 , wherein, from among the ALUs of the first processing element, a first ALU adjacent to the second bank is connected to a second ALU adjacent to the first bank from among the ALUs of the second processing element through a data line.