Saliency-based video compression systems and methods
Examples of systems and methods described herein may provide saliency-based video compression. A saliency map associated with a video may be generated and/or provided. A tile configuration may be selected for the video and quality settings assigned to each tile in accordance with the saliency map. The video may then be compressed (e.g., encoded) in tiles in accordance with the quality settings. Compressed videos may be stored together with saliency metadata, facilitating storage management and/or re-compression.
1. A method comprising:
generating a single saliency map associated with a video segment, comprising:
collecting a maximum saliency value for each of a plurality of pixels across a plurality of frames in the video segment; and
obtaining aggregated saliency values based on the collected maximum saliency value for each of the plurality of pixels across the plurality of frames in the video segment;
selecting a tiling configuration for the video segment from a plurality of candidate tiling configurations, said selecting comprising:
determining a quality setting for each tile in each of the candidate tiling configurations, the quality setting based on a saliency value at each tile in the saliency map, including assigning at least one of a target bitrate or constant rate factor to selected tiles associated with a highest subset of saliency values in the saliency map or a minimum bitrate to a lowest subset of saliency values in the saliency map;
compressing the tiles in accordance with the multiple candidate tiling configurations and the saliency map; and
selecting at least one of the multiple candidate tiling configurations as a selected tiling configuration based on evaluating metrics for the multiple candidate tiling configurations, the metrics including at least one of bitrate, video quality, or combinations thereof;
and
encoding the tiles in accordance with the selected tiling configuration.
2. The method of claim 1 , wherein generating the saliency map associated with the video segment comprises combining a plurality of intermediate saliency maps.
3. The method of claim 1 , wherein said determining comprises assigning a target bitrate or constant rate factor to selected tiles associated with a highest subset of saliency values in the saliency map.
4. The method of claim 1 , wherein said determining comprises assigning a minimum bitrate to a lowest subset of saliency values in the saliency map.
5. The method of claim 1 , further comprising generating a tiled saliency map bitstream based on the saliency map, wherein the tiled saliency map bitstream is configured to specify a location of a particular tile among the tiles.
6. A system comprising:
at least one processor;
at least one computer readable media encoded with instructions, which, when executed, cause the at least one processor to perform operations comprising:
generating a single saliency map based on an input video segment, comprising:
collecting saliency for each of a plurality of pixels across a plurality of frames in the video segment; and
obtaining aggregated saliency values based on the collected maximum saliency value for each of the plurality of pixels across the plurality of frames in the video segment;
selecting a tiling configuration for the video segment from a plurality of candidate tiling configurations, said selecting comprising:
determining a quality setting for each tile in each of the candidate tiling configurations, the quality setting based on a saliency value at each tile in the saliency map, including assigning at least one of a target bitrate or constant rate factor to selected tiles associated with a highest subset of saliency values in the saliency map or a minimum bitrate to a lowest subset of saliency values in the saliency map;
compressing the tiles in accordance with the multiple candidate tiling configurations and the saliency map; and
selecting at least one of the multiple candidate tiling configurations as a selected tiling configuration based on evaluating metrics for the multiple candidate tiling configurations, the metrics including at least one of bitrate, video quality, or combinations thereof;
a transcoder configured to transcode the input video segment in the tiles in accordance with the associated quality settings to provide a compressed video segment.
7. The system of claim 6 , wherein the operations further comprise storing the compressed video segment in a video container including saliency metadata comprising a saliency weight for each of the tiles.
8. The system of claim 6 , wherein the operations further comprise:
receiving a request to reduce a size of the compressed video segment; and
altering the quality settings responsive to the request to provide updated quality settings,
wherein the transcoder is further configured to re-transcode the compressed video segment in accordance with the updated quality settings.
9. The system of claim 6 , wherein the operations further comprise:
receiving additional perceptual information; and
updating the saliency map based on the additional perceptual information to provide an updated saliency map.
10. The system of claim 6 , wherein the system comprises a cloud-based service configured to receive the input video segment from an external source.
11. The system of claim 6 , wherein the operations further comprise generating a tiled saliency map bitstream based on the saliency map, wherein the tiled saliency map bitstream is configured to specify a location of a particular tile among the tiles.