System and method for reconstructing restored facial images from video

1. A system for producing restored facial images from frames of a video, comprising:

an active shape component for fitting an active shape model to a facial image found in each of a plurality of digital video frames;

a registration component for registering each location within a facial region of each said facial image with a corresponding location in any of the other facial images; and

a restoration component for producing a restored facial image from the facial images.

2. The system of claim 1 , wherein the active shape component comprises:

a video input module for providing a plurality of video frames;

a face detection module adapted to detect a face in each of the plurality of video frames; and

a fitting module for determining facial image model parameters such that the model closely resembles the face in each of the plurality of video frames.

3. The system of claim 1 , wherein said restoration component comprises a super-resolution algorithm.

4. The system of claim 3 , wherein the super-resolution algorithm comprises:

an image formation model (IFM) which uses an active shape model for each of the plurality of video frames for producing IFM facial images; and

a data fidelity model which uses actual facial images from each of the plurality of video frames to compare to the IFM facial images, said data fidelity model including a cost function.

5. The system of claim 4 , comprising a regularization model for inhibiting high spatial frequency distortion, said regularization model including a second cost function.

6. The system of claim 1 , wherein said restoration component comprises:

a warping module for taking an output from the registration component to warp each of the facial images of each of the plurality of video frames to a new frame of reference having a greater pixel resolution than the pixel resolution of the plurality of video frames;

an averaging module for averaging each of the warped facial images on a pixel by pixel basis; and

an image restoration algorithm for producing a restored facial image from the averaged facial images.

7. The system of claim 6 , wherein the image restoration algorithm comprises a Wiener filter.

8. The system of claim 6 , wherein the new frame of reference has a pixel resolution two times greater than the pixel resolution of the plurality of video frames.

9. A computerized method for producing restored facial images from frames of a video, comprising:

receiving a plurality of digital video frames containing facial images;

fitting an active shape model to a facial image found in each of the plurality of video frames;

registering each image location of each said facial image with the corresponding image location of each of the other facial images; and

producing a restored facial image from the facial images that has a resolution.

10. The method of claim 9 , wherein said fitting an active appearance model comprises:

accessing a database of facial images; and

constructing a generalized mathematical representation of the appearance and the shape of the facial images from the database.

11. The method of claim 10 , wherein the generalized mathematical representation is adjustable.

12. The method of claim 9 , wherein the fitting an active appearance model comprises:

detecting a face in each of the plurality of video frames; and

producing facial image model parameters such that the facial image model closely resembles the face in each of the plurality of video frames.

13. The method of claim 9 , wherein said producing a restored facial image comprises utilizing a super-resolution algorithm.

14. The method of claim 13 , wherein the super-resolution algorithm comprises:

an image formation model (IFM) which uses an active shape model for each of the plurality of video frames for producing IFM facial images; and

a data fidelity model which uses actual facial images from each of the plurality of video frames to compare to the IFM facial images, said data fidelity model including a cost function and a cost function gradient.

15. The method of claim 14 , comprising regularizing to inhibit high spatial frequency distortion.

16. The method of claim 9 , wherein said producing a restored facial image comprises:

a warping module for taking an output from the registration component to warp each of the facial images of each of the plurality of video frames to a new frame of reference having a greater pixel resolution than the pixel resolution of the plurality of video frames;

an averaging module for averaging each of the warped facial images on a pixel by pixel basis; and

an image restoration algorithm for producing a restored facial image from the averaged facial images.

17. The method of claim 16 , wherein the image restoration algorithm comprises a Wiener filter.

18. The method of claim 16 , wherein the new frame of reference has a pixel resolution two times greater than the pixel resolution of the plurality of video frames.