Detecting keypoints in image data
Methods and systems for detecting keypoints in image data may include an image sensor interface receiving pixel data from an image sensor. A front-end pixel data processing circuit may receive pixel data and convert the pixel data to a different color space format. A back-end pixel data processing circuit may perform one or more operations on the pixel data. An output circuit may receive pixel data and output the pixel data to a system memory. A keypoint detection circuit may receive pixel data from the image sensor interface in the image sensor pixel data format or receive pixel data after processing by the front-end or the back-end pixel data processing circuits. The keypoint detection circuit may perform a keypoint detection operation on the pixel data to detect one or more keypoints in the image frame and output to the system memory a description of the one or more keypoints.
1. An image signal processor, comprising:
a keypoint detection circuit configured to:
receive a first pixel data of a source image frame;
detect a first keypoint based at least in part on a portion of the first pixel data of the source image frame having one or more magnitude values exceeding a keypoint sensitivity threshold value; and
output to a memory a description based on the detected first keypoint; and
an image processing circuit configured to:
identify a second keypoint in a second image frame different from the source image frame according to the description accessed from the memory; and
perform pixel processing operations according to the identified second keypoint to modify a second pixel data of the second image frame.
2. The image signal processor of claim 1 , wherein the source image frame comprises an image resolution different from an image resolution of the second image frame.
3. The image signal processor of claim 1 , wherein the first pixel data for the source image frame comprises luminance channel image data.
4. The image signal processor of claim 1 , wherein the source image frame is encoded in a first color space format, and wherein the second image frame is encoded in a second color space format different from the first color space format.
5. The image signal processor of claim 4 , wherein the second color space format comprises a luminance channel and a plurality of chroma channels.
6. The image signal processor of claim 1 , wherein to modify the second pixel data of the second image frame the image processing circuit is configured to perform one or more noise filtering or color processing operations on respective pixel data of the one or more image frames.
7. The image signal processor of claim 1 , wherein to output the description, the keypoint detection circuit is configured to selectively output a description of one or more keypoints detected in a first set of respective regions of the source image frame responsive to respective magnitude values of the one or more keypoints detected in the first set of respective regions exceeding one of a plurality of keypoint sensitivity threshold values corresponding to the first set of respective regions of the source image frame.
8. A method, comprising:
performing, by a keypoint detection circuit:
receiving a first pixel data of a source image frame;
detecting a first keypoint based at least in part on a portion of the first pixel data of the source image frame having one or more magnitude values exceeding a keypoint sensitivity threshold value; and
outputting to a memory a description based on the detected first keypoint; and
performing, by an image processing circuit:
identifying a second keypoint in a second image frame different from the source image frame according to the description accessed from the memory; and
performing pixel processing operations according to the identified second keypoint to modify a second pixel data of the second image frame.
9. The method of claim 8 , wherein the source image frame comprises an image resolution different from an image resolution of the second image frame.
10. The method of claim 8 , wherein the first pixel data for the source image frame comprises luminance channel image data.
11. The method of claim 8 , wherein the source image frame is encoded in a first color space format, and wherein the second image frame is encoded in a second color space format different from the first color space format.
12. The method of claim 11 , wherein the second color space format comprises a luminance channel and a plurality of chroma channels.
13. The method of claim 8 , wherein modifying the second pixel data of the second image frame comprises performing one or more noise filtering or color processing operations on respective pixel data of the one or more image frames.
14. The method of claim 8 , wherein outputting the description comprises selectively outputting a description of one or more keypoints detected in a first set of respective regions of the source image frame responsive to respective magnitude values of the one or more keypoints detected in the first set of respective regions exceeding one of a plurality of keypoint sensitivity threshold values corresponding to the first set of respective regions of the source image frame.
15. A device, comprising:
a central processing unit;
a system memory; and
an image signal processor comprising:
a keypoint detection circuit configured to:
receive a first pixel data of a source image frame;
detect a first keypoint based at least in part on a portion of the first pixel data of the source image frame having one or more magnitude values exceeding a keypoint sensitivity threshold value; and
output to a memory a description based on the detected first keypoint; and
an image processing circuit configured to:
identify a second keypoint in a second image frame different from the source image frame according to the description accessed from the memory; and
perform pixel processing operations according to the identified second keypoint to modify a second pixel data of the second image frame.
16. The device of claim 15 , further comprising:
a multiplexer circuit configured to:
select pixel data for an image frame from:
an image sensor interface in an image sensor pixel data format,
a back-end pixel data processing circuit in a color space format, or
a system memory in another format; and
provide the pixel data to a pre-processing circuit; and
the pre-processing circuit configured to:
convert the image frame to a processing format comprising one or more data channels including a luminance channel to generate the source image frame; and
output the source image frame in the processing format to the keypoint detection circuit.
17. The device of claim 15 , wherein the source image frame comprises an image resolution different from an image resolution of the second image frame.
18. The device of claim 15 , wherein the source image frame is encoded in a first color space format, and wherein the second image frame is encoded in a second color space format different from the first color space format.
19. The device of claim 18 , wherein the second color space format comprises a plurality of color channels.
20. The device of claim 15 , wherein to modify the second pixel data of the second image frame the image processing circuit is configured to perform one or more noise filtering or color processing operations on respective pixel data of the one or more image frames.