Matching active speaker pose between two cameras

1. A method for selecting a camera of a plurality of cameras, each with a different view of a group of individuals in an environment and providing a video stream, a primary camera of the plurality of cameras having a microphone array, to provide a video stream for provision to a far end, the method comprising:

utilizing sound source localization using the microphone array on the primary camera to determine direction information;

identifying a speaker in the group of individuals using the sound source localization direction information and an image from the video stream of the primary camera;

determining pose information of the speaker in the image from the video stream of the primary camera;

for each of the plurality of cameras other than the primary camera, determining pose information of each individual in the group of individuals in an image from the video stream of the primary camera;

for each of the plurality of cameras other than the primary camera, comparing the pose information of each individual in the group of individuals with the pose information of the identified speaker, determining a best pose of the identified speaker in the image from the video stream of the camera; and

selecting a camera from the plurality of cameras to provide a video stream for provision to a far end based on a determination of the camera having the best frontal view of the identified speaker using pose information after setting an individual as a speaker for each camera.

2. The method of claim 1 , wherein selecting a camera from the plurality of cameras is further based on:

selecting the camera providing the most facial views of attendees when there is not a speaker and there are attendees; and

selecting a default camera when there are no attendees.

3. The method of claim 1 further comprising machine learning based on neural networks to determine pose boundaries and keypoints.

4. The method of claim 1 , further comprising receiving audio from each microphone in the microphone array to perform a sound source localization algorithm to determine a particular individual which is speaking.

5. The method of claim 1 , wherein sound source localization and machine learning based on neural networks are implemented by the primary camera and no sound source localization and machine learning based on neural networks are implemented by secondary cameras.

6. The method of claim 1 , wherein sound source localization and machine learning based on neural networks are implemented by the primary camera and secondary cameras.

7. The method of claim 2 further comprising determining over a period of time if the selected camera continues to be the default camera.

8. A system for selecting a camera of a plurality of cameras, each with a different view of a group of individuals in an environment, to provide a video stream for provision to a far end, the system comprising:

a plurality of cameras, one of the plurality of cameras being a primary camera, each camera including:

an imager;

an output interface for providing data and a video stream;

RAM;

a processor coupled to the imager, the output interface, and the RAM for executing programs; and

memory coupled to the processor for storing programs executed by the processor, the memory storing programs executed by the processor to perform the operation of providing a video stream from the camera,

the primary camera further including a microphone array and the memory further storing programs to utilize sound source localization using the microphone array to determine direction information and provide the direction information; and

a codec coupled to the plurality of cameras, the codec including:

an input interface for coupling to the plurality of cameras to receive data and video streams;

a network interface for communicating with the far end;

RAM;

a processor coupled to the network interface, the input interface and the RAM for executing programs; and

memory coupled to the processor for storing programs executed by the processor, the memory storing programs executed by the processor to perform the operation of:

identifying a speaker in the group of individuals using the sound source localization direction information and an image from the video stream of the primary camera;

determining pose information of the speaker in the image from the video stream of the primary camera;

for each of the plurality of cameras other than the primary camera, determining pose information of each individual in the group of individuals in an image from the video stream of the camera;

for each of the plurality of cameras other than the primary camera, comparing the pose information of each individual in the group of individuals with the pose information of the identified speaker, determining a best pose of the identified speaker in the image from the video stream of the camera; and

selecting a camera from the plurality of cameras to provide a video stream for provision to a far end based on a determination of the camera having the best frontal view of the identified speaker using pose information after setting an individual as a speaker for each camera.

9. The system of claim 8 , wherein selecting a camera from the plurality of cameras is further based on:

selecting the camera providing the most facial views of attendees when there is not a speaker and there are attendees; and

selecting a default camera when there are no attendees.

10. The system of claim 8 , wherein the primary camera is built into the codec and the processor, RAM and memory are the processor and memory of the codec.

11. The system of claim 8 further comprising receiving audio from each microphone in the microphone array to perform a sound source localization algorithm to determine a particular individual which is speaking.

12. The system of claim 8 , wherein machine learning based on neural networks is implemented by the primary camera for individual face and pose detection.

13. The system of claim 8 , wherein machine learning based on neural networks is implemented by the primary camera and each of the other plurality of cameras for individual face and pose detection.

14. The system of claim 13 further comprising a production module bounding boxes of images, feature vectors, poses, and SSL for detected faces of individuals from the primary camera and secondary cameras.

15. The system of claim 14 further comprising room director components for the secondary cameras which send output of machine learning of the secondary cameras to the production module.

16. The system of claim 9 further comprising determining by the processor of the primary camera, over a period of time if the selected camera continues to be the default camera.

17. A camera comprising:

a microphone array on a primary camera utilizing sound source localization to determine direction information;

a processor identifying a speaker in a group of individuals using the sound source localization direction information and an image from a video stream of the primary camera;

a codec for determining pose information of the speaker in the image from the video stream of the primary camera;

for each of a plurality of cameras other than the primary camera, determining by the codec pose information of each individual in the group of individuals in an image from the video stream of the camera;

for each of the plurality of cameras other than the primary camera, comparing by the codec the pose information of each individual in the group of individuals with the pose information of the speaker, determining the best match and setting that individual as the speaker in the image from the video stream of the camera; and

selecting by the codec, a camera from the plurality of cameras to provide a video stream for provision to a far end based on a determination of the camera having the best frontal view of the speaker after setting an individual as a speaker for each camera.

18. The camera of claim 17 , wherein selecting a camera from the plurality of cameras is further based on:

selecting the camera providing the most facial views of attendees when there is not a speaker and there are attendees; and

selecting a default camera when there are no attendees.

19. The camera of claim 17 further comprising receiving audio from each microphone in the microphone array to perform when the sound source localization is performed on the primary camera.

20. The camera of claim 17 further comprising machine learning systems based on neural networks to determine pose boundaries and keypoints.