Edge enhancement for temporal scaling with metadata

1. A method to post-process a non-key frame of a video sequence through a neighboring key frame of the video sequence, comprising:

downsampling a reconstructed key frame to form a downsampled reconstructed key frame;

upsampling the downsampled reconstructed key frame to form an upsampled reconstructed key frame;

forming a difference between the reconstructed key frame and the upsampled reconstructed key frame;

adding the difference to a reconstructed non-key frame to compensate the reconstructed non-key frame to form an edge enhanced non-key frame; and

updating the edge enhanced non-key frame to form an updated non-key frame based on a difference between a downsampled edge enhanced non-key frame and a downsampled non-key frame,

wherein the downsampled edge enhanced non-key frame is formed by downsampling the edge enhanced non-key frame and the reconstructed non-key frame is formed by upsampling the downsampled non-key frame.

2. The method of claim 1 , wherein the difference between the reconstructed key frame and the upsampled reconstructed key frame is filtered before being added to the reconstructed non-key frame.

3. The method of claim 1 , wherein the updating comprises:

evaluating if the difference between the downsampled edge enhanced non-key frame and the downsampled reconstructed non-key frame is less than a threshold value;

if the difference between the downsampled edge enhanced non-key frame and the downsampled non-key frame is less than the threshold value, setting the updated non-key frame to the edge enhanced non-key frame; and

if the difference between the downsampled edge enhanced non-key frame and the downsampled non-key frame is not less than the threshold value, setting the updated non-key frame to the reconstructed non-key frame.

4. The method of claim 1 , wherein the updating comprises:

setting a first frame to the reconstructed non-key frame and a first downsampled frame to the downsampled non-key frame, wherein the first frame is formed by upsampling the first downsampled frame;

setting a second frame to the edge enhanced non-key frame and a second downsampled frame to the downsampled edge enhanced non-key frame, wherein the second downsampled frame is formed by downsampling the second frame;

evaluating if the difference between the first downsampled frame and the second downsampled frame is less than a threshold value;

if the difference between the first downsampled frame and the second downsampled frame is less than the threshold value, setting the updated non-key frame to the second frame;

if the difference between the first downsampled frame and the second downsampled frame is not less than the threshold value, setting the updated non-key frame to the reconstructed non-key frame;

updating the first frame to the second frame and the first downsampled frame to the second downsampled frame;

updating the second frame to the updated non-key frame and the second downsampled frame to a downsampled updated non-key frame, wherein the downsampled updated non-key frame is formed by downsampling the updated non-key frame; and

iterating the evaluating, the setting of the updated non-key frame, the updating of the first frame, and the updating of the second frame until one or more criteria are satisfied.

5. The method of claim 4 , wherein the threshold value is monotonically decreasing between successive iterations.

6. The method of claim 4 , wherein the iterating is performed a set number of times.

7. The method of claim 1 , wherein the updating comprises:

upsampling the difference between the downsampled edge enhanced non-key frame and the downsampled non-key frame to form an upsampled difference; and

adding the upsampled difference and the edge enhanced non-key frame to form the updated non-key frame.

8. The method of claim 1 , wherein the updating comprises:

setting a first frame to the edge enhanced non-key frame and a first downsampled frame to the downsampled edge enhanced non-key frame, wherein the first downsampled frame is formed by downsampling the first frame;

upsampling the difference between the downsampled first frame and the downsampled non-key frame to form an upsampled difference;

multiplying a step size with the upsampled difference to form a scaled upsampled difference;

adding the scaled upsampled difference to form the updated non-key frame;

setting the first frame to the updated non-key frame and the first downsampled frame to a downsampled updated non-key frame, wherein the downsampled updated non-key frame is formed by downsampling the updated non-key frame; and

iterating the upsampling of the difference, the multiplying, the adding, and the setting of the first frame.

9. The method of claim 8 , wherein the step size is monotonically decreasing between successive iterations.

10. A method to process a non-key frame of a video sequence through a key frame of the video sequence, comprising:

downsampling a reconstructed key frame to form a downsampled reconstructed key frame;

forming a difference between a downsampled non-key frame and the downsampled reconstructed key frame;

upsampling the difference to form an upsampled difference;

adding the upsampled difference to the reconstructed key frame to form an edge enhanced non-key frame; and

updating the edge enhanced non-key frame to form an updated non-key frame based on a difference between a downsampled edge enhanced non-key frame and the downsampled non-key frame,

wherein the downsampled edge enhanced non-key frame is formed by downsampling the edge enhanced non-key frame.

11. The method of claim 10 , wherein the key frame is a neighboring key frame of the non-key frame.

12. The method of claim 10 , wherein the steps of forming the difference, upsampling and adding occur at a decoding stage of the video sequence.

13. A method to process a non-key frame of a video sequence through a neighboring key frame of the video sequence, comprising:

forming a difference between the non-key frame and a reconstructed key frame;

downsampling the difference to form a downsampled difference;

upsampling the downsampled difference to form an upsampled difference;

adding the upsampled difference to the reconstructed key frame to generate an edge enhanced non-key frame; and

updating the edge enhanced non-key frame to form an updated non-key frame based on a difference between a downsampled edge enhanced non-key frame and a downsampled non-key frame,

wherein the downsampled edge enhanced non-key frame is formed by downsampling the edge enhanced non-key frame and the downsampled non-key frame is formed by downsampling the non-key frame.

14. The method of claim 13 , wherein the reconstructed key frame is a neighboring key frame of the non-key frame.

15. The method of claim 13 , wherein the steps of upsampling and adding occur at a decoding stage of the video sequence.

16. A method to post-process a non-key frame of a video sequence through key frames of the video sequence, comprising:

downsampling a plurality of reconstructed key frames to form downsampled reconstructed key frames;

upsampling the downsampled reconstructed key frames to form upsampled reconstructed key frames;

forming differences between the reconstructed key frames and the respective upsampled reconstructed key frames;

adding a sum of the differences to a reconstructed non-key frame to compensate the reconstructed non-key frame to form an edge enhanced non-key frame; and

updating the edge enhanced non-key frame to form an updated non-key frame based on a difference between a downsampled edge enhanced non-key frame and a downsampled non-key frame,

wherein the downsampled edge enhanced non-key frame is formed by downsampling the edge enhanced non-key frame and the reconstructed non-key frame is formed by upsampling the downsampled non-key frame.

17. The method of claim 16 , wherein the key frames comprise one or more key frames previous to the non-key frame and/or one or more key frames following the non-key frame.

18. The method of claim 16 , wherein the differences are weighted differences based on weighting parameters.

19. The method of claim 18 wherein the weighting parameters depend on a temporal distance between the key frame of each difference and the non-key frame.

20. The method of claim 18 , wherein the weighting parameters are fixed parameters.

21. The method of claim 18 , wherein the weighting parameters depend on a difference between a current non-key frame and the plurality of reconstructed key frames.

22. The method of claim 18 , wherein the weighting parameters are adaptive.

23. The method of claim 16 , wherein the adding the sum of the differences is limited to a sum of the differences formed with reference to an adaptively chosen subset of the key frames.

24. The method of claim 16 , wherein the reconstructed non-key frame is further compensated by taking into account previously post-processed non-key frames.

25. A method to post-process a non-key frame of a video sequence through key frames of the video sequence, comprising:

downsampling reconstructed key frames to form downsampled reconstructed key frames;

for each downsampled reconstructed key frame forming a difference between a downsampled non-key frame and the downsampled reconstructed key frame;

upsampling each difference to form upsampled differences;

adding each reconstructed key frame to a respective upsampled difference to form revised upsampled differences;

adding the revised upsampled differences together to form an edge enhanced non-key frame; and

updating the edge enhanced non-key frame to form an updated non-key frame based on a difference between a downsampled edge enhanced non-key frame and the downsampled non-key frame,

wherein the downsampled edge enhanced non-key frame is formed by downsampling the edge enhanced non-key frame.

26. The method of claim 25 , wherein the revised upsampled differences are weighted revised upsampled differences based on weighting parameters.

27. The method of claim 26 wherein the weighting parameters depend on a temporal distance between the key frame of each difference and the non-key frame.

28. The method of claim 26 , wherein the weighting parameters are fixed parameters.

29. The method of claim 26 , wherein the weighting parameters depend on a difference between a current non-key frame and reference key frames.

30. The method of claim 26 , wherein the weighting parameters are adaptive.