System and method for automatically framing conversations in a meeting or a video conference
Utilization of a state machine to determine participant framing. The states include empty room, group framing, any talker, conversation mode and unambiguous talker. In empty room state, the conference room is framed. In group framing state, any participants in the room are framed. In any talker state, the talking participant is framed. In conversation mode state, all talking participants are framed. In unambiguous talker state, the single talking participant is framed. Various framing conditions define transitions between the states. Conditions include, presence of participants, which and number of participants that are talking for how long, system mute and far site talking. The conversation states and conditions and framing decisions provide a fully automated framing mechanism to provide pleasant framing of the individuals in the near site or end for any of the conditions relating to number of talkers, participants and the like.
1. A method of framing a view of a near site provided to a far site in a videoconference, the method comprising:
determining a number of participants present at the near site;
determining a number and locations of talkers at the near site;
determining if the near site is muted;
determining if the far site is talking; and
traversing states of a state machine based on the determinations of number of participants present at the near site, number and locations of talkers at the near site, near site is muted and far site is talking and changes in those determinations,
wherein each state defines a particular view for framing,
wherein if the number of talkers at the near site changes, changing the view for framing to encompass all talkers as the number of talkers changes, and
wherein the number of talkers at the near site can equal or exceed three talkers.
2. The method of claim 1 , wherein traversing states of the state machine are further based on time periods from changes in determinations.
3. The method of claim 1 , wherein the states include an any talker state and a conversation mode state,
wherein in the any talker state, only a single participant is talking,
wherein in the any talker state the view for framing is the talker,
wherein in the conversation mode state a plurality of talkers are or have been talking, and
wherein in the conversation mode state the view for framing is all of the talkers.
4. The method of claim 3 , wherein in the conversation mode state, when the number of talkers changes, the view is reframed to be a view of all current talkers.
5. The method of claim 3 , wherein the states include an unambiguous talker state,
wherein the unambiguous talker state is entered when a given participant has been talking for a period of time to be considered a presenter, and
wherein in the unambiguous talker state the view for framing is the talker.
6. The method of claim 5 , wherein the states include a group framing state,
wherein in the group framing state either no participants are talking, the near site is muted or the far site is talking, and
wherein in the group framing state the view for framing is of all participants.
7. The method of claim 6 , wherein the states include an empty room state,
wherein in the empty room state there are no participants, and
wherein in the empty room state, the view for framing is the entire near site.
8. A videoconferencing device comprising:
a camera interface for receiving a videoconferencing video stream at a near site;
a microphone interface for receiving a videoconferencing audio stream at the near site;
a network interface for receiving videoconference streams from at least one far site and providing videoconference streams to the at least one far site;
a processor coupled to the camera interface, the microphone interface and the network interface for executing programs and operations to perform videoconferencing operations; and
memory coupled to the processor for storing programs executed by the processor, the memory storing programs executed by the processor to perform the operations of:
determining a number of participants present at the near site;
determining a number and locations of talkers at the near site;
determining if the near site is muted;
determining if the far site is talking; and
traversing states of a state machine based on the determinations of number of participants present at the near site, number and locations of talkers at the near site, near site is muted and far site is talking and changes in those determinations,
wherein each state defines a particular view for framing,
wherein if the number of talkers at the near site changes, changing the view for framing to encompass all talkers as the number of talkers changes, and
wherein the number of talkers at the near site can equal or exceed three talkers.
9. The videoconferencing device of claim 8 , wherein traversing states of the state machine are further based on time periods from changes in determinations.
10. The videoconferencing device of claim 8 , wherein the states include an any talker state and a conversation mode state,
wherein in the any talker state, only a single participant is talking,
wherein in the any talker state the view for framing is the talker,
wherein in the conversation mode state a plurality of talkers are or have been talking, and
wherein in the conversation mode state the view for framing is all of the talkers.
11. The videoconferencing device of claim 10 , wherein in the conversation mode state, when the number of talkers changes, the view is reframed to be a view of all current talkers.
12. The videoconferencing device of claim 10 , wherein the states include an unambiguous talker state,
wherein the unambiguous talker state is entered when a given participant has been talking for a period of time to be considered a presenter, and
wherein in the unambiguous talker state the view for framing is the talker.
13. The videoconferencing device of claim 12 , wherein the states include a group framing state,
wherein in the group framing state either no participants are talking, the near site is muted or the far site is talking, and
wherein in the group framing state the view for framing is of all participants.
14. The videoconferencing device of claim 13 , wherein the states include an empty room state,
wherein in the empty room state there are no participants, and
wherein in the empty room state, the view for framing is the entire near site.
15. A non-transitory processor readable memory containing programs that when executed cause a processor to perform the following method of framing a view of a near site provided to a far site in a videoconference, the method comprising:
determining a number of participants present at the near site;
determining a number and locations of talkers at the near site;
determining if the near site is muted;
determining if the far site is talking; and
traversing states of a state machine based on the determinations of number of participants present at the near site, number and locations of talkers at the near site, near site is muted and far site is talking and changes in those determinations,
wherein each state defines a particular view for framing,
wherein if the number of talkers at the near site changes, changing the view for framing to encompass all talkers as the number of talkers changes, and
wherein the number of talkers at the near site can equal or exceed three talkers.
16. The non-transitory processor readable memory of claim 15 , wherein traversing states of the state machine are further based on time periods from changes in determinations.
17. The non-transitory processor readable memory of claim 15 , wherein the states include an any talker state and a conversation mode state,
wherein in the any talker state, only a single participant is talking,
wherein in the any talker state the view for framing is the talker,
wherein in the conversation mode state a plurality of talkers are or have been talking, and
wherein in the conversation mode state the view for framing is all of the talkers.
18. The non-transitory processor readable memory of claim 17 , wherein in the conversation mode state, when the number of talkers changes, the view is reframed to be a view of all current talkers.
19. The non-transitory processor readable memory of claim 17 , wherein the states include an unambiguous talker state,
wherein the unambiguous talker state is entered when a given participant has been talking for a period of time to be considered a presenter, and
wherein in the unambiguous talker state the view for framing is the talker.
20. The non-transitory processor readable memory of claim 19 , wherein the states include a group framing state and an empty room state,
wherein in the group framing state either no participants are talking, the near site is muted or the far site is talking,
wherein in the group framing state the view for framing is of all participants,
wherein in the empty room state there are no participants, and
wherein in the empty room state, the view for framing is the entire near site.