De-interlacing and frame rate upconversion for high definition video
View Patent ↗Technologies and implementations for de-interlacing an interlaced digital video and up-converting a frame rate of a digital video are generally disclosed.
1. A method to de-interlace an interlaced video, the method comprising:
estimating, on a computing device, a first progressive scan video frame based at least, in part, on an interlaced video frame from the interlaced video, wherein the estimating includes:
estimating, for each interlaced pixel of a plurality of interlaced pixels in the interlaced video frame, a progressive scan pixel value based at least, in part, on a value of the interlaced pixel;
determining, for each interlaced pixel of the plurality of interlaced pixels, an interpolation direction based at least, in part, on the estimated progressive scan pixel value; and
re-estimating, for each interlaced pixel of the plurality of interlaced pixels, the progressive scan pixel value based at least, in part, on the value of the interlaced pixel and the determined interpolation direction; and
generating a second progressive scan video frame based at least, in part, on application of a nonlocal means filter to the interlaced video frame.
2. A method to de-interlace an interlaced video, the method comprising:
estimating, on a computing device, a first progressive scan video frame based at least, in part, on an interlaced video frame from the interlaced video; and
generating a second progressive scan video frame based at least, in part, on application of a nonlocal means filter to the interlaced video frame, wherein the generating include:
identifying, for each interlaced pixel of a plurality of interlaced pixels in the interlaced video frame, a first similarity window of pixels in the estimated first progressive scan video frame; and
determining a value of a pixel in the second progressive scan video frame based at least, in part, on a weighted average of values of the pixels in the first similarity window.
3. The method of claim 2 , wherein the interlaced video frame includes a first interlaced video frame, and wherein the generating the second progressive scan video frame based at least, in part, on the application of the nonlocal means filter to the interlaced video frame further includes:
identifying, for each interlaced pixel of the plurality of interlaced pixels, a second similarity window of pixels in a second interlaced video frame from the interlaced video, wherein the determining the value of the pixel in the second progressive scan video frame is further based at least, in part, on a weighted average of values of the pixels in the second similarity window.
4. The method of claim 2 , wherein the determining the value of the pixel in the second progressive scan video frame based at least, in part, on the weighted average of values of the pixels in the first similarity window includes:
determining a kernel matrix based at least, in part, on a local gradient of values of the pixels in the first similarity window; and
determining weight values based at least, in part, on the kernel matrix.
5. The method of claim 3 , wherein the first similarity window includes even lines of pixels from the first interlaced video frame, and wherein the second similarity window includes odd lines of pixels from the second interlaced video frame.
6. The method of claim 3 , wherein the first similarity window includes odd lines of pixels from the first interlaced video frame, and wherein the second similarity window includes even lines of pixels from the second interlaced video frame.
7. A non-transitory machine readable medium having stored therein instructions that, in response to execution by one or more processors, operatively enable a de-interlacer tool to:
estimate a first progressive scan video frame based at least, in part, on an interlaced video frame from an interlaced video, wherein the estimation includes:
estimate, for each interlaced pixel of a plurality of interlaced pixels in the interlaced video frame, a progressive scan pixel value based at least, in part, on a value of the interlaced pixel;
determine, for each interlaced pixel of the plurality of interlaced pixels, an interpolation direction based at least, in part, on the estimated progressive scan pixel value; and
re-estimate, for each interlaced pixel of the plurality of interlaced pixels, the progressive scan pixel value based at least, in part, on the value of the interlaced pixel and the determined interpolation direction; and
generate a second progressive scan video frame based at least, in part, on application of a nonlocal means filter to the interlaced video frame.
8. A non-transitory machine readable medium having stored therein instructions that, in response to execution by one or more processors, operatively enable a de-interlacer tool to:
estimate a first progressive scan video frame based at least, in part, on an interlaced video frame from an interlaced video; and
generate a second progressive scan video frame based at least, in part, on application of a nonlocal means filter to the interlaced video frame, wherein the generation includes:
identify, for each interlaced pixel of a plurality of interlaced pixels in the interlaced video frame, a first similarity window of pixels in the estimated first progressive scan video frame; and
determine a value of a pixel in the second progressive scan video frame based at least, in part, on a weighted average of values of the pixels in the first similarity window.
9. The non-transitory machine readable medium of claim 8 , wherein the interlaced video frame includes a first interlaced video frame, and wherein the stored instructions that operatively enable the de-interlacer tool to generate the second progressive scan video frame include instructions that, in response to execution by the one or more processors, operatively enable the de-interlacer tool to:
identify, for each interlaced pixel of the plurality of interlaced pixels, a second similarity window of pixels in a second interlaced video frame from the interlaced video, wherein determination of the value of the pixel in the second progressive scan video frame is further based at least, in part, on a weighted average of values of the pixels in the second similarity window.
10. The non-transitory machine readable medium of claim 8 , wherein the stored instructions that operatively enable the de-interlacer tool to determine the value of the pixel in the second progressive scan video frame include instructions that, in response to execution by the one or more processors, operatively enable the de-interlacer tool to:
determine a kernel matrix based at least, in part, on a local gradient of values of the pixels in the first similarity window; and
determine weight values based at least, in part, on the kernel matrix.
11. The non-transitory machine readable medium of claim 9 , wherein the first similarity window includes even lines of pixels from the first interlaced video frame, and wherein the second similarity window includes odd lines of pixels from the second interlaced video frame.
12. The non-transitory machine readable medium of claim 9 , wherein the first similarity window includes odd lines of pixels from the first interlaced video frame, and wherein the second similarity window includes even lines of pixels from the second interlaced video frame.
13. A system to de-interlace an interlaced video, the system comprising:
a processor; and
a de-interlacer tool communicatively coupled to the processor, wherein the de-interlacer tool includes:
a frame estimation module configured to estimate a first progressive scan video frame based at least, in part, on an interlaced video frame from the interlaced video,
wherein to estimate the first progressive scan video frame, the frame estimation module of the de-interlacer tool is configured to:
estimate, for each interlaced pixel of a plurality of interlaced pixels in the interlaced video frame, a progressive scan pixel value based at least, in part, on a value of the interlaced pixel;
determine, for each interlaced pixel of the plurality of interlaced pixels, an interpolation direction based at least, in part, on the estimated progressive scan pixel value; and
re-estimate, for each interlaced pixel of the plurality of interlaced pixels, the progressive scan pixel value based at least, in part, on the value of the interlaced pixel and the determined interpolation direction; and
a filter module coupled to the frame estimation module and configured to generate a second progressive scan video frame based at least, in part, on application of a nonlocal means filter to the interlaced video frame.
14. A system to de-interlace an interlaced video, the system comprising:
a processor; and
a de-interlacer tool communicatively coupled to the processor, wherein the de-interlacer tool includes:
a frame estimation module configured to estimate a first progressive scan video frame based at least, in part, on an interlaced video frame from the interlaced video; and
a filter module coupled to the frame estimation module and configured to generate a second progressive scan video frame based at least, in part, on application of a nonlocal means filter to the interlaced video frame,
wherein to generate the second progressive scan video frame, the filter module of the de-interlacer tool is configured to:
identify, for each interlaced pixel of a plurality of interlaced pixels in the interlaced video frame, a first similarity window of pixels in the estimated first progressive scan video frame; and
determine a value of a pixel in the second progressive scan video frame based at least, in part, on a weighted average of values of the pixels in the first similarity window.
15. The system of claim 14 , wherein the interlaced video frame includes a first interlaced video frame, and wherein to generate the second progressive scan video frame, the filter module of the de-interlacer tool is configured to:
identify, for each interlaced pixel of the plurality of interlaced pixels, a second similarity window of pixels in a second interlaced video frame from the interlaced video, wherein the determination of the value of the pixel in the second progressive scan video frame is further based at least, in part, on a weighted average of values of the pixels in the second similarity window.
16. The system of claim 14 , wherein to determine the value of the pixel in the second progressive scan video frame, the filter module of the de-interlacing tool is configured to:
determine a kernel matrix based at least, in part, on a local gradient of values of the pixels in the first similarity window; and
determine weight values based at least, in part, on the kernel matrix.
17. The system of claim 15 , wherein the first similarity window includes even lines of pixels from the first interlaced video frame, and wherein the second similarity window includes odd lines of pixels from the second interlaced video frame.
18. The system of claim 15 , wherein the first similarity window includes odd lines of pixels from the first interlaced video frame, and wherein the second similarity window includes even lines of pixels from the second interlaced video frame.