Robust frame size error detection and recovery mechanism to minimize frame loss for camera input sub-systems
An image data frame is received from an external source. An error concealment operation is performed on the received image data frame in response to determining that a first frame size of the received image data frame is erroneous. The first frame size of the image data frame is determined to be erroneous based on at least one frame synchronization signal associated with the image data frame. An image processing operation is performed on the received image data frame on which the error concealment operation has been performed, thereby enabling an image processing module to perform the image processing operation without entering into a deadlock state and thereby prevent a host processor from having to execute hardware resets of deadlocked modules.
1. An image processing device, comprising:
an error handling circuit configured to:
receive an image dataframe;
determine that a size of the image data frame is erroneous based on a first marker indicating a start of the image data frame and further based on a second marker indicating an end of the image data frame; and
perform an error concealment operation on the image data frame in response to determining that the size of the image data frame is erroneous, the error concealment operation including discarding data of a remainder of the image data frame being received by the error handling circuit, and
when the image data frame is determined to be undersized, determine that pixel data of at least one of a plurality of lines of the image data frame is missing, and in response, generate dummy data for the missing pixel data, and
when the image data frame is determined to be oversized, generate dummy data to complete the image data frame; and
an image processing circuit that is communicatively coupled to receive, from the error handling circuit, and perform an image processing operation on, the image data frame on which the error concealment operation has been performed.
2. The image processing device according to claim 1 , wherein, in performing the error concealment operation when the image data frame is determined to be undersized and pixel data of at least one of a plurality of lines of the image data frame is determined to be missing, the error handling circuit is further configured to:
generate the dummy data for the missing pixel data so that the size of the image data frame becomes equal to a reference size.
3. The image processing device according to claim 2 , wherein the error handling circuit generates the dummy data by repeating data of a last received good pixel of the image data frame.
4. The image processing device according to claim 1 , wherein to determine that the image data frame is oversized, the error handling circuit is further configured to determine that the size of the image frame is larger than a reference size, and in response to determining that the size of the image frame is larger than the reference size, the error handling circuit is further configured to:
generate the dummy data to complete the image data frame so that the size of the completed image data frame becomes equal to the reference size.
5. The image processing device according to claim 4 , wherein to determine that the size of the image data frame is larger than the reference size, the error handling circuit is configured to determine that a number of pixels for which data is received by the error handling circuit for at least one of a plurality of lines of the image data frame is greater than a reference number of pixels per line for the image data frame.
6. The image processing device according to claim 4 , wherein to determine that the size of the image data frame is larger than the reference size, the error handling circuit is configured to determine that a number of lines for which data is received by the error handling circuit for the image data frame is greater than a reference number of lines of the image data frame.
7. The image processing device according to claim 1 , wherein the error handling circuit is further configured to detect a frame sync signal protocol violation for the image data frame based on at least one of the first marker, the second marker, or a third marker,
wherein the first marker comprises a vertical-start signal that is active while receiving data of a first pixel of the image data frame by the error handling circuit,
wherein the second marker comprises a vertical-end signal that is active while receiving data of a last pixel of the image data frame by the error handling circuit, and
wherein the third marker comprises:
a horizontal-start signal that is active while receiving data of a first pixel of each of a plurality of lines of the image data frame by the error handling circuit; or
a horizontal-end signal that is active while receiving data of a last pixel of each of the plurality of lines of the image data frame by the error handling circuit.
8. The image processing device according to claim 7 , wherein the error handling circuit is further configured to interrupt a host processor in response to detecting the frame sync signal protocol violation for the image data frame.
9. The image processing device according to claim 1 , further comprising an image data receiver circuit communicatively coupled to the error handling circuit, wherein the image data receiver circuit is configured to receive a continuous stream of image data frames, and wherein the error handling circuit is configured to perform the error concealment operation for each image data frame of the continuous stream in response to determining that a size of the corresponding image data frame is erroneous.
10. The image processing device according to claim 1 , further comprising an error detection circuit that is communicatively coupled to the error handling circuit,
wherein the size of the image data frame is a first size, and
wherein the error detection circuit is configured to determine whether a second fame-size of the image data frame is erroneous based on a reference size.
11. The image processing device according to claim 10 , wherein the error detection circuit is further configured to:
determine whether the second size of the image data frame is larger than the reference size; and
responsive to determining that the second size is larger than the reference size, discard extra pixel data of one or more lines of the image data frame so that the second size becomes equal to the reference size.
12. The image processing device according to claim 1 , wherein the error handling circuit is further configured to interrupt a host processor in response to determining that the size of the image data frame is erroneous.
13. The image processing device according to claim 12 , wherein the error handling circuit is further configured to perform the error concealment operation for each of a plurality of lines of the image data frame, one line at a time.
14. The image processing device according to claim 13 , wherein the error handling circuit is further configured to report error information to the host processor, wherein the error information includes at least one of error location information and frame confidence value information.
15. An image processing method, comprising:
receiving an image data frame;
determining that a size of the image data frame is erroneous based on a first marker indicating a start of the image data frame and further based on a second marker indicating an end of the image data frame;
performing an error concealment operation on the image data frame in response to determining that the size of the image data frame is erroneous, wherein the error concealment operation includes:
discarding data of a remainder of the image data frame being received by the error handling circuit, and
when the image frame is determined to be undersized and pixel data of at least one of a plurality of lines of the image data frame is determined to be missing, generating dummy data for the missing pixel data, and
when the image data frame is determined to be oversized, generating dummy data to complete the image data frame; and
performing an image processing operation on the image data frame on which the error concealment operation has been performed.
16. The image processing method according to claim 15 , wherein generating the dummy data for the missing pixel data includes:
generating the dummy data for the missing pixel data so that the size of the image data frame becomes equal to a reference size.
17. The image processing method according to claim 15 , wherein generating the dummy data to complete the image data frame includes:
generating the dummy data to complete the image data frame so that the size of the completed frame becomes equal to the reference size.
18. The image processing method according to claim 15 , further comprising:
detecting a frame sync signal protocol violation for the image data frame based on at least one of the first marker, the second marker, or a third marker,
wherein the first marker comprises a vertical-start signal that is active while receiving data of a first pixel of the image data frame,
wherein the second marker comprises a vertical-end signal that is active while receiving data of a last pixel of the image data frame,
wherein the third marker comprises:
a horizontal-start signal that is active while receiving data of a first pixel of each of a plurality of lines of the image data frame; or
a horizontal-end signal that is active while receiving data of a last pixel of each of the plurality of lines of the image data frame.
19. The image processing method according to claim 1 , wherein the size of the image data frame is a first size, the method further comprising determining whether a second size of the image data frame is erroneous based on a reference size,
wherein the second size of the image data frame is measured by a first module of an image processing device that is upstream of a second module of the image processing device that performs the error concealment operation.
20. The image processing method according to claim 19 , further comprising:
determining whether the second size of the image data frame is larger than the reference sine;
responsive to determining that the second size is larger than the reference size, discarding, by the first module, extra pixel data of one or more lines of the image data frame so that the second size becomes equal to the reference size; and
transmitting the image data frame with the discarded extra pixel data to the second module.
21. The image processing method according to claim 15 , further comprising interrupting a host processor in response to determining that the size of the image data frame is erroneous.
22. A non-transitory computer readable medium comprising instructions stored thereon that, when executed by one or more processors, cause the one or more processors to:
receive an image data frame;
determine that a size of the image data frame is erroneous based on a first marker indicating a start of the image data frame and further based on a second marker indicating an end of the image data frame;
perform an error concealment operation on the image data frame in response to determining that the size of the image data frame is erroneous, wherein, in performing the error concealment operation, the one or more processors:
discard data of a remainder of the image data frame being received by the error handling circuit, and
when the image frame is determined to be undersized and pixel data of at least one of a plurality of lines of the image data frame is determined to be missing, generate dummy data for the missing pixel data, and
when the image data frame is determined to be oversized, generate dummy data to complete the image data frame, and
perform an image processing operation on the image data frame on which the error concealment operation has been performed.