Virtual camera operator

1. A method of virtual camera operation, the method comprising:

identifying a face of a participant in an input video stream;

identifying a target region of the face in a first frame of the input video stream and in a second frame of the input video frame;

generating speech motion data based on an inter-frame motion value determined from a first histogram, associated with a first frame, and a second histogram, associated with a second frame, the first histogram and the second histogram being based upon a gradient associated with each pixel, of a plurality of pixels, in the target region of the face in each of a first frame of the input video stream and a second frame of the input video stream;

determining a speaking state of the participant in the input video stream based on the speech motion data;

identifying a subject of the input video stream based on the speaking state; and

generating, using a processor, an output video stream including a portion of the input video stream based on the subject.

2. The method of claim 1 , wherein the portion of the input video stream is a first portion of the input video stream and the generating the output video stream includes generating a transition from a second portion of the input video stream to the first portion of the input video stream.

3. The method of claim 1 , wherein the participant is a participant in a plurality of participants and the input video stream includes a scene that includes the plurality of participants, and wherein:

the identifying the face includes identifying a plurality of faces wherein each face in the

plurality of faces is associated with a respective participant in the plurality of participants;

the determining the speaking state includes determining a plurality of speaking states wherein each speaking state in the plurality of speaking states is associated with a respective participant in the plurality of participants and a respective face in the plurality of faces; and the identifying the active region is based on the plurality of speaking states.

4. The method of claim 1 , wherein the identifying the face includes generating a subsampled video stream based on the input video stream, the subsampled video stream having a resolution that is lower than a resolution of the input video stream.

5. The method of claim 1 , wherein the identifying the face includes generating identification data, and generating motion tracking data.

6. The method of claim 5 , wherein the generating the identification data includes generating landmarking data.

7. The method of claim 5 , wherein the input video stream includes a plurality of frames and has a frame rate, and wherein the generating the identification data is performed at a frequency that is lower than the frame rate.

8. The method of claim 5 , wherein the generating the motion tracking data includes using visual template tracking, optical-flow based feature tacking, or a combination of visual template tracking and optical-flow based feature tracking.

9. The method of claim 5 , wherein the identifying the face includes synchronizing the identification data with the motion tracking data.

10. The method of claim 1 , wherein the determining the speaking state includes:

identifying speech motion data associated with the face; and

generating a speaker model associated with the face.

11. The method of claim 10 , wherein the generating the speaker model includes identifying a speaking state probability based on the speech motion data.

12. An apparatus for use in virtual camera operation, the apparatus comprising:

a memory; and

a processor configured to execute instructions stored in the memory to:

identify a face of a participant in an input video stream;

identify a target region of the face in a first frame of the input video stream and in a second frame of the input video frame;

generate speech motion data based on an inter-frame motion value determined from a first histogram, associated with a first frame, and a second histogram, associated with a second frame, the first histogram and the second histogram being based upon a gradient associated with each pixel, of a plurality of pixels, in the target region of the face in each of a first frame of the input video stream and a second frame of the input video stream;

determine a speaking state of the participant in the input video stream based on the speech motion data;

identify a subject of the input video stream based on the speaking state; and

generate an output video stream including a portion of the input video stream based on the subject.

13. The apparatus of claim 12 , wherein the portion of the input video stream is a first portion of the input video stream and wherein the processor is configured to execute instructions stored in the memory to generate the output video stream by generating a transition from a second portion of the input video stream to the first portion of the input video stream.

14. The apparatus of claim 12 , wherein the subject is a first subject and wherein the processor is configured to execute instructions stored in the memory to generate the output video stream by generating a transition from a second subject to the first subject.

15. The apparatus of claim 12 , wherein the participant is a participant in a plurality of participants and the input video stream includes a scene that includes the plurality of participants, and wherein the processor is configured to execute instructions stored in the memory to:

identify the face by identifying a plurality of faces wherein each face in the plurality of faces is associated with a respective participant in the plurality of participants;

determine the speaking state by determining a plurality of speaking states wherein each speaking state in the plurality of speaking states is associated with a respective participant in the plurality of participants and a respective face in the plurality of faces; and

identify the active region based on the plurality of speaking states.

16. The apparatus of claim 12 , wherein the processor is configured to execute instructions stored in the memory to identify the face by generating a subsampled video stream based on the input video stream, the subsampled video stream having a resolution that is lower than a resolution of the input video stream.

17. The apparatus of claim 12 , wherein the processor is configured to execute instructions stored in the memory to identify the face by generating identification data, and generating motion tracking data.

18. The apparatus of claim 17 , wherein the processor is configured to execute instructions stored in the memory to generate the identification data by generating landmarking data.

19. The apparatus of claim 17 , wherein the input video stream includes a plurality of frames and has a frame rate, and wherein the processor is configured to execute instructions stored in the memory to generate the identification data at a frequency that is lower than the frame rate.

20. The apparatus of claim 17 , wherein the processor is configured to execute instructions stored in the memory to generate the motion tracking data using visual template tracking, optical-flow based feature tacking, or a combination of visual template tracking and optical-flow based feature tracking.

21. The apparatus of claim 17 , wherein the processor is configured to execute instructions stored in the memory to identify the face by synchronizing the identification data with the motion tracking data.

22. The apparatus of claim 12 , wherein the processor is configured to execute instructions stored in the memory to determine the speaking state by:

identifying speech motion data associated with the face; and

generating a speaker model associated with the face.

23. The apparatus of claim 22 , wherein the processor is configured to execute instructions stored in the memory to generate the speaker model by identifying a speaking state probability based on the speech motion data.

24. The method of claim 1 , wherein the subject is a first subject and the generating the output video stream includes generating a transition from a second subject to the first subject.