Video coding using a saliency map
A video coder includes a processing resource and a non-transitory storage device containing instructions executable by the processing resource to compute a weighted Δ frame based on a saliency map and a Δ frame. The saliency map is to indicate the relative importance of each pixel in a current frame based on its perceptual significance. The Δ frame is to include differences between corresponding pixels in a current frame and a motion predicted frame.
1. A video coder, comprising:
a processing resource; and
a non-transitory storage device containing instructions executable by the processing resource to compute a compressed frame based on:
a saliency map generated to indicate relative importance of each pixel in a current frame based on a perceptual significance associated with each such pixel, wherein the saliency map is generated when a threshold number of delta color values between current pixels and corresponding pixels in a motion predicted frame of a video stream is exceeded;
a tone map function selected from a plurality of tone map functions based on the saliency map;
a weighted delta frame determined by multiplying a dynamic tone mapped color difference image and a delta frame, wherein the delta frame includes differences between corresponding pixels in the current frame and the motion predicted frame indicated by the saliency map; and
a preservation map generated by multiplying a variance map of the motion predicted frame by the difference between a weighted delta frame and a delta frame determined based on the saliency map.
2. The video coder of claim 1 , wherein for a first pixel in the current frame designated by the saliency map to have a higher relative importance than a second pixel, the non-transitory storage device contains instructions executable by the processing resource to select a tone map function for the first pixel that preserves smaller color differences than a tone map function selected for the second pixel.
3. The video coder of claim 2 , wherein the non-transitory storage device contains instructions executable by the processing resource to apply the selected tone map function to a color difference in a color difference image to generate a dynamic tone mapped color difference image.
4. The video coder of claim 1 , wherein the non-transitory storage device containing instructions executable by the processing resource to select, based on the saliency map, a tone map function out of at least four tone map functions stored on the non-transitory storage device.
5. The video coder of claim 4 , wherein each of the at least four tone map functions specifies a relationship between a color difference weighting and a color difference between corresponding pixels in the current frame and the motion predicted frame.
6. The video coder of claim 1 , wherein the non-transitory storage device containing instructions executable by the processing resource to determine a separate saliency map for a plurality of frames of an image sequence.
7. The video coder of claim 1 , wherein the non-transitory storage device containing instructions executable by the processing resource to select a different tone map function for two pixels indicated by the saliency map to have different levels of importance, the tone map function selected for a more important pixel of the two pixels preserves smaller color differences than the tone map function selected for a less important pixel of the two pixels.
8. The video coder of claim 1 , wherein the non-transitory storage device containing instructions executable by the processing resource to determine, on a pixel by pixel basis, which tone map function to select for each pixel.
9. The video coder of claim 1 , wherein the non-transitory storage device containing instructions executable by the processing resource to select one of the plurality of tone map functions on a pixel-by-pixel basis.
10. The video coder of claim 1 wherein each of the plurality of tone map functions color differences are normalized to an output from 0 to 1.