Method and system for intelligent video adaptation

1. A method for efficient video adaptation of an input video with a sequence of frames, the method comprising:

generating a video trace from the input video by performing principal component analysis on each frame of the sequence of frames to determine eigenvectors for each frame, wherein the video trace is a compact representation for temporal and spatial distortions for the sequence of frames;

segmenting the input video into a plurality of video shots using the video trace, wherein each video shot has a consistent rate-distortion coding consistency;

selecting, using the video trace, a subset of frames for the video shots that minimizes spatial-temporal distortions of adapted video; and

selecting transcoding parameters for the subset of frames to produce an optimal video quality of the adapted video under constraints of frame rate, bit rate, frame quality, and viewing time constraint.

2. The method of claim 1 , wherein the step of selecting transcoding parameters includes:

querying spatio-temporal distortion models for quantization parameters at various operating points; and

adapting the transcoding parameters to network characteristics, device capacity, and user preference in accordance with the spatio-temporal distortion models,

wherein the adapting is a process of selecting an operating point that satisfies the user preferences, and meets system constraints while providing optimal video quality in the adapted video.

3. The method of claim 2 , wherein the step of selecting transcoding parameters further comprises looking up quantization parameters, QP, in a spatio-temporal model to achieve optimal video quality, wherein the spatio-temporal model is a function of time compression rate, r and a bit-rate, R.

4. The method of claim 2 , further comprising retrieving a pre-computed video trace and pre-computed spatio-temporal model from side information associated with the input video.

5. The method of claim 1 , further comprising building a spatio-temporal model from the input video.

6. The method of claim 5 , wherein the step of generating a video trace comprises:

performing principal component analysis on a frame to produce a trace value; and

placing the trace value in a projection space,

wherein a distance between two trace values in the projection space is a temporal distortion between two frames corresponding to the two trace values.

7. The method of claim 1 , wherein the step of performing principal component analysis, comprises:

reconstructing an image from a frame of the input video;

scaling the image to accommodate for a display width and height;

vectorizing the image to produce a vectorized image; and

projecting the vectorized image on a set of basis functions.

8. The method of claim 1 , wherein the step of selecting transcoding parameters further comprises:

receiving a view time constraint on the time compression rate, r, for transcoding the video shots, wherein the video time constraint establishes a video play back duration;

determining a data rate, R, that is desired for encoding the video shots at the video play back duration;

selecting a P-SNR curve that matches the view time constraint from a pre-computed Rate Distortion table PSNR(QP,r); and

looking up a closest QP for the P-SNR curve that results in the data rate, R, desired.

9. The method of claim 1 , wherein the step of selecting the transcoding parameters takes into account a viewing time for setting the time compression factor, r.

10. A method for generating side information to an input video with a sequence of frames, the method comprising:

generating a video trace from the input video by performing principal component analysis on each frame of the sequence of frames to determine eigenvectors for each frame, wherein the video trace is a compact representation for temporal and spatial distortions for the sequence of frames and the video trace is used to generate spatio-temporal rate distortion models;

segmenting the input video into a plurality of video shots using the video trace and the spatio-temporal rate distortion models, wherein each video shot has a consistent rate-distortion coding consistency;

identifying transcoding parameters for the plurality of video shots for various constraints of viewing time and bit rate; and

evaluating operating points on video shots as a function of viewing time and bit rate for the transcoding parameters.

11. The method of claim 10 , further comprising:

storing the spatio-temporal rate-distortion models as side information with the input video; and

storing the video trace as side information with the input video.

12. The method of claim 10 , wherein the step of identifying transcoding parameters for the plurality of video shots includes evaluating several operating points on plurality of video shots and interpolating between the operating points for identifying quantization parameters.

13. The method of claim 10 , wherein the step of generating a video trace comprises:

performing principal component analysis on frames of a video shot to produce a trace value; and

placing the trace value in a projection space of the video trace, wherein a distance between two trace values in the projection space is a temporal distortion between two frames corresponding to the two trace values.

14. A system for transcoding an input video comprising:

a compressed domain processor for

receiving an input video having a sequence of frames and

generating a video trace from the input video by performing principal component analysis on each frame of a sequence of frames to determine eigenvectors for each frame, wherein the video trace is a compact representation for temporal and spatial distortions for the sequence of frames in the input video; and

a video adaptation engine (VAE) operatively coupled to the compressed domain processor for using the video trace to

temporally segment an input video into video shots using the video trace, wherein each video shot has a consistent rate-distortion coding consistency,

select which frames to transcode in the video shots using the video trace, and

generate transcoding parameters for the frames that provide optimal adapted video quality in accordance with an adaptation request and operating points of a transcoder.

15. The system of claim 14 , further comprising:

a shot segmentation unit that receives the video trace from the compressed domain processor and segments the input video into video shots having rate-distortion coding consistency across frames; and

a transcoder that receives the video shots from the shot segmentation unit and receives transcoding parameters from the VAE for adapting the input video to produce adapted video in accordance with constraints of bit rate, frame rate, and distortion.

16. The system of claim 15 , wherein the VAE generates the transcoding parameters from the video trace in view of the adaptation request by

selecting frames in the video shots that minimize a temporal distortion of the adapted video using the video trace;

querying spatio-temporal distortion models for different operating points; and

repeating the steps of selecting frames and querying the spatio-temporal distortion models at the video shot level for identifying an operating point that provides optimal adapted video quality given the adaptation request.

17. The system of claim 14 , wherein the VAE further performs the steps of:

evaluating various operating points for adapted video quality using the spatio-temporal rate-distortion models; and

selecting operating points and transcoding parameters for the plurality of video shots that satisfy a user preference and meet system constraints while achieving an optimal quality of adapted video.

18. The system of claim 14 , wherein the VAE retrieves the video trace and spatio-temporal distortion models from side information associated with the input video.

19. The system of claim 14 , wherein the VAE builds spatio-temporal distortion models from the video trace for different operating points during video adaptation.

20. The system of claim 14 , wherein the VAE decides what frames in a video shot are selected for transcoding and decides the quantization parameters for the frames to achieve an optimal adapted video quality based on the spatio-temporal distortion models that characterize how the quantization parameters affect the output video quality.

21. The system of claim 14 , wherein, when a time compression factor is not provided, the VAE solves a utility function maximization problem by:

reconstructing spatio-temporal rate distortion models at pre-determined view time compression levels;

evaluating a utility at operating points in the spatio-temporal rate distortion models;

sorting operating points by an associated rate; and

performing a bi-section search on the utility to locate an optimal adaptation point.