Data compression for video
The present invention provides a technique for performing one or more aspects of video coding such as quantization, intra prediction coding or inter prediction coding in dependence on a perceptual model taking into account human sensitivity to data in the video signal. The perceptual model may relate to spatial frequency, temporal frequency, contrast sensitivity, color sensitivity, a structural metric, and/or one or more parameters affecting perception such as motion in the video, the distance of a recipient user from the screen, and the size, aspect ratio or resolution of the screen of the recipient terminal.
1. A method comprising:
receiving a video signal comprising a plurality of frames, each frame comprising one or more portions of video data;
transforming each of said portions into a transform domain, so as for each portion to generate transformed video data comprising a set of coefficients, each coefficient representing a contribution to a property of the portion at a different spatial or temporal frequency;
quantizing the video data of each portion according to a quantization bin size distribution, the quantization bin size distribution varying dynamically during ongoing encoding such that a different quantization bin size distribution is applied to different transformed video data, the quantization bin size distribution being dependent on: a structural metric of the one or more portions of the video data and a measure of human sensitivity to the video data such that a portion of the video data to which a human viewer is more sensitive is quantized using a smaller quantization bin size and a portion of the video data to which the human viewer is less sensitive is quantized using a larger quantization bin size, the quantization bin size distribution varying dynamically according to information received automatically from a recipient terminal; and
encoding the video data of each portion based, at least in part, on the quantizing.
2. The method of claim 1 , wherein:
said quantization comprises quantizing the coefficients of each portion; and
the quantization bin size distribution uses different quantization bin sizes for different colour properties of the transformed video data.
3. The method of claim 1 , wherein the quantization bin size distribution is related to a measure of human sensitivity to said property at different spatial frequencies.
4. The method of claim 1 , wherein the quantization bin size distribution uses different quantization bin sizes for different ones of said coefficients.
5. The method of claim 1 , wherein the property comprises one of a luminance and a colour property.
6. The method of claim 5 , wherein some of said coefficients represent luminance and some of said coefficients represent at least one colour property, and the method further comprising using a different quantization bin size distribution for quantizing the luminance and for quantizing the at least one colour property.
7. The method of claim 5 , wherein some of said coefficients represent a first colour channel and some of said coefficients represent a second colour channel, and the method further comprising using different quantization bin size distributions for the first and second colour channels.
8. The method of claim 1 , further comprising adapting the quantization bin size distribution according to information identifying spatial frequencies in objects included in the video data.
9. The method of claim 1 , wherein the quantization bin size distribution is variable, during encoding, in dependence on a distance of the human viewer from a screen of the recipient terminal.
10. The method of claim 1 , wherein the quantization bin size distribution is dynamically varied during ongoing encoding in dependence on a parameter which affects human perception.
11. The method of claim 10 , wherein the bin size distribution is varied dynamically in dependence on motion in the video signal, the motion being a parameter which affects human perception.
12. The method of claim 11 , wherein the motion is determined by a motion estimation algorithm applied to the frames of the video signal.
13. The method of claim 11 , wherein the motion is detected by physical sensors in a mobile terminal housing a camera which generates said video signal.
14. The method of claim 9 , comprising transmitting an indication of the bin size distribution to a decoder for use in decoding the video signal.
15. The method of claim 1 , wherein the video data is input to the quantizer before inter or intra encoding such that the video data represents absolute values.
16. A method comprising:
receiving a video signal comprising a plurality of frames, each frame comprising one or more portions of video data;
transforming each of said portions into a transform domain, so as for each portion to generate transformed video data comprising a set of coefficients, each coefficient representing a contribution to a property of the portion at a different spatial or temporal frequency;
quantizing the video data of each portion, the quantizing including quantizing the coefficients of each portion;
encoding the video data of each portion according to a quantization bin size distribution, the quantization bin size distribution varying dynamically during ongoing encoding such that a different quantization bin size distribution is applied to different transformed video data, the quantization bin size distribution being dependent on: a measure of human sensitivity to the video data such that a portion of the video data to which a human viewer is more sensitive is quantized using a smaller quantization bin size and a portion of the video data to which the human viewer is less sensitive is quantized using a larger quantization bin size, the encoding including using different quantization bin size distributions for different colour channels; and
transmitting the encoded bitstream to a decoder, the encoded bitstream including side information indicating a prediction method to be used during decoding, the prediction method being dependent on the measure of human sensitivity to the video data.
17. An encoder comprising:
a signal processing apparatus configured to implement a method of encoding a video signal for transmission to a decoder of a receiving terminal, the method comprising:
receiving a video signal comprising a plurality of frames, each frame comprising one or more portions of video data;
transforming each of said portions into a transform domain, so as for each portion to generate transformed video data comprising a set of coefficients, each coefficient representing a contribution to a property of the portion at a different spatial or temporal frequency;
quantizing the video data of each portion according to a quantization bin size distribution, the quantizing including quantizing the coefficients of each portion, the quantization bin size distribution varying dynamically during ongoing encoding such that a different quantization bin size distribution is applied to different transformed video data, the quantization bin size distribution being dependent on: a structural metric of the one or more portions of the video data and a measure of human sensitivity to the video data such that a portion of the video data to which a human viewer is more sensitive is quantized using a smaller quantization bin size and a portion of the video data to which the human viewer is less sensitive is quantized using a larger quantization bin size, the quantization bin size distribution varying dynamically during ongoing encoding in dependence on information received automatically from a recipient terminal;
encoding the video data of each portion based, at least in part, on the quantizing.
18. The encoder of claim 17 , wherein the quantization bin size distribution is related to a measure of human sensitivity to said property at different spatial or temporal frequencies.
19. The encoder of claim 17 , wherein the quantization bin size distribution uses different quantization bin sizes for different ones of said coefficients.
20. The encoder of claim 17 , wherein the property comprises one of a luminance and a colour property.
21. The encoder of claim 20 , wherein some of said coefficients represent luminance and some of said coefficients represent at least one colour property, and the method further comprising using a different quantization bin size distribution for quantizing the luminance and for quantizing the at least one colour property.
22. The encoder of claim 20 , wherein some of said coefficients represent a first colour channel and some of said coefficients represent a second colour channel, and the method further comprising using different quantization bin size distributions for the first and second colour channels.
23. The encoder of claim 17 , wherein the quantization bin size distribution is further dependent on an aspect ratio, a size, and a resolution of a screen that is to present the video data after the quantizing.
24. The encoder of claim 17 , wherein the quantization bin size distribution is variable in dependence on a parameter which affects human perception and a distance of the human viewer from a screen of the receiving terminal being a parameter which affects human perception.
25. The encoder of claim 17 , wherein the quantization bin size distribution is dynamically varied during ongoing encoding in dependence on a parameter which affects human perception.
26. The encoder of claim 25 , wherein the bin size distribution is varied dynamically in dependence on motion in the video signal, the motion being a parameter which affects human perception.
27. The encoder of claim 26 , wherein the motion is determined by a motion estimation algorithm applied to the frames of the video signal.
28. The encoder of claim 26 , wherein the motion is detected by physical sensors in a mobile terminal housing a camera which generates said video signal.
29. The encoder of claim 24 , wherein the method further comprises transmitting an indication of the bin size distribution to the decoder for use in decoding the video signal.
30. The encoder of claim 17 , wherein the video data is input to the quantizer before inter or intra encoding such that the video data represents absolute values.
31. An encoder comprising:
a signal processing apparatus configured to implement a method of encoding a video signal for transmission to a decoder of a receiving terminal, the method comprising:
receiving a video signal comprising a plurality of frames, each frame comprising one or more portions of video data;
transforming each of said portions into a transform domain, so as for each portion to generate transformed video data comprising a set of coefficients, each coefficient representing a contribution to a property of the portion at a different spatial or temporal frequency,
quantizing the video data of each portion, the quantizing including quantizing the coefficients of each portion; and
encoding the video data of each portion according to a quantization bin size distribution, the quantization bin size distribution varying dynamically during ongoing encoding such that a different quantization bin size distribution is applied to different transformed video data, the quantization bin size distribution being dependent on: a structural metric of the one or more portions of the video data and a measure of human sensitivity to the video data such that a portion of the video data to which a human viewer is more sensitive is quantized using a smaller quantization bin size and a portion of the video data to which the human viewer is less sensitive is quantized using a larger quantization bin size, the quantization bin size distribution varying dynamically according to information received automatically from a recipient terminal.
32. The encoder of claim 31 , wherein the quantization bin size distribution is related to a measure of human sensitivity to said property at different spatial or temporal frequencies.
33. The encoder of claim 31 , wherein the quantization bin size distribution uses different quantization bin sizes for different ones of said coefficients.
34. The encoder of claim 31 , wherein the property comprises one of a luminance and a colour property.
35. The encoder of claim 34 , wherein some of said coefficients represent luminance and some of said coefficients represent at least one colour property, and the method further comprising using a different quantization bin size distribution for quantizing the luminance and for quantizing the at least one colour property.
36. The encoder of claim 34 , wherein some of said coefficients represent a first colour channel and some of said coefficients represent a second colour channel, and the method further comprising using different quantization bin size distributions for the first and second colour channels.
37. The encoder of claim 31 , wherein the quantization bin size distribution is further dependent on an aspect ratio, a size, and a resolution of a screen that is to present the video data after the quantizing.
38. The encoder of claim 31 , wherein the quantization bin size distribution is variable in dependence on a parameter which affects human perception and a distance of the human viewer from a screen of the recipient terminal being a parameter which affects human perception.
39. The encoder of claim 31 , wherein the quantization bin size distribution is dynamically varied during ongoing encoding in dependence on a parameter which affects human perception.
40. The encoder of claim 39 , wherein the bin size distribution is varied dynamically in dependence on motion in the video signal, the motion being a parameter which affects human perception.
41. The encoder of claim 40 , wherein the motion is determined by a motion estimation algorithm applied to the frames of the video signal.
42. The encoder of claim 40 , wherein the motion is detected by physical sensors in a mobile terminal housing a camera which generates said video signal.
43. The encoder of claim 38 , wherein the method further comprises transmitting an indication of the bin size distribution to the decoder for use in decoding the video signal.
44. The encoder of claim 31 , wherein the video data is input to the quantizer before inter or intra encoding such that the video data represents absolute values.
45. The method of claim 16 , wherein the property of the portion includes a luminance property or a colour property.
46. The method of claim 16 , wherein the quantization bin size distribution is variable in dependence on a distance of the human viewer from a screen of the recipient terminal.