Method for controlling interactive display system
A method for controlling a multi-user interactive display system including a soft-copy display including at least an information display region and a command control region, and a digital image capture system positioned to capture a time sequence of images of users located in a field-of-view of the soft-copy display. A time sequence of images is analyzed to detect a plurality of users, and at least one of the users is designated to be a controlling user. The captured images are displayed in the command control region, wherein the detected users are demarked using graphical elements. The captured time sequence of images is analyzed to detect a gesture made by the controlling user and content displayed in the information display region is updated accordingly.
1. A method for interactively controlling a multi-user interactive display system having a display including at least an information display region and a command control region, and a digital image capture system positioned to capture a time sequence images of users, comprising:
receiving a time sequence of images captured by the digital image capture system;
using a data processing system to analyze the captured time sequence of images to detect a plurality of users who are viewing the display;
designating at least one of the detected plurality of users to be a controlling user;
displaying the captured time sequence of images in the command control region, wherein the detected users are demarked using graphical elements, and wherein the graphical element associated with the at least one controlling user is differentiated from the graphical elements associated with the other detected users to provide an indication of which of the plurality of users is the controlling user; and
using the data processing system to analyze the captured time sequence of images to detect a gesture made by the controlling user and updating content displayed in the information display region in response to the detected gesture.
2. The method of claim 1 wherein the detected gesture is a pointing gesture and wherein the content displayed in the information display region is updated to include a fiducial indicating a pointing location where the controlling user is pointing.
3. The method of claim 2 wherein the pointing gesture is an arm movement gesture or a hand movement gesture or a finger movement gesture, a head pose gesture or an eye gaze gesture.
4. The method of claim 2 wherein the fiducial indicating the pointing location changes shape, color or size depending on the pointing location, a state of the multi-user interactive display system, a characteristic of the controlling user or a gesture made by the controlling user.
5. The method of claim 2 wherein the fiducial includes a tail indicating a history of pointing locations.
6. The method of claim 2 wherein the information display region includes one or more selectable active elements associated with predefined actions.
7. The method of claim 6 further including detecting a second gesture of the controlling user, wherein the second gesture is a selecting gesture, and wherein the predefined action associated with a particular selectable active element is initiated in response to the selecting gesture when the pointing location corresponds to the particular selectable active element.
8. The method of claim 1 wherein the detected gesture is a relinquish control gesture and wherein a different detected user is designated as the controlling user in response to the detected relinquish control gesture.
9. The method of claim 1 wherein the designation of the controlling user is determined responsive to a distance of the user from the display and a duration of time that the user has been viewing the display.
10. The method of claim 9 wherein the designation of the controlling user includes determining a priority value determined by computing a weighted summation of two or more terms, wherein one of the terms is responsive to the distance of the user from the display and a second term is responsive to the duration of time that the user has been viewing the display.
11. The method of claim 10 wherein the priority value is determined by:
P i =W D (1 −D i )+ W T T i
where P i is the priority value for an i th user, D i is a normalized distance of the i th user from the display, T i is a normalized time duration that the i th user has been viewing the display, and W D and W T are weighting constants.
12. The method of claim 10 wherein the weighted summation also includes a term that increases the priority value for users that are currently designated to be a controlling user.
13. The method of claim 10 wherein the weighted summation also includes a term that increases the priority value for users who perform a control request gesture.
14. The method of claim 1 wherein the detected users are sorted into a priority queue, such that the relative position in the priority queue indicates an order that the users will be considered to be selected as the controlling user.
15. The method of claim 14 wherein the graphical elements associated with the detected users provide an indication of the user's relative position in the priority queue.
16. The method of claim 15 wherein the indication of the user's relative position in the priority queue is provided by the color, shape, size or transparency of the graphical elements.
17. The method of claim 15 wherein the indication of the user's relative position in the priority queue is provided by numerical values included in the graphical elements.
18. The method of claim 14 further including detecting a control request gesture made by one of the detected users who is not the controlling user, and wherein the relative position of that user in the priority queue is adjusted in response to the detected control request gesture.
19. The method of claim 1 wherein the graphical elements associated with the detected users include geometrical shapes enclosing the faces of the detected users.
20. The method of claim 1 wherein the graphical elements associated with the detected users include avatars placed in proximity to the faces of the detected users.
21. The method of claim 20 wherein one or more attributes of the avatars are adjusted responsive to one or more attributes of the detected users.
22. The method of claim 21 wherein a facial expression attribute, a facial orientation attribute, a visual focus of attention attribute, an age attribute, a gender attribute, a race attribute, a hair color attribute, a hair style attribute, a facial hair attribute, an eye glasses attribute, a height attribute, a weight attribute, an identifying object attribute, a complexion attribute, a jewelry attribute, a clothing attribute or a makeup attribute associated with the avatar are adjusted to in accordance with corresponding attributes of the detected users.
23. The method of claim 1 wherein the content displayed in the information display region is updated responsive to a detected attribute of the controlling user.
24. The method of claim 23 wherein the detected attribute is a facial expression attribute, a facial orientation attribute, a visual focus of attention attribute, an age attribute, a gender attribute, a race attribute, a hair color attribute, a hair style attribute, a facial hair attribute, an eye glasses attribute, a height attribute, a weight attribute, an identifying object attribute, a complexion attribute, a jewelry attribute, a clothing attribute or a makeup attribute.
25. The method of claim 1 wherein more than one detected users is designated to be a controlling user.
26. The method of claim 25 wherein different graphical elements are associated with each of the controlling users.
27. The method of claim 26 wherein each of the controlling users controls a fiducial indicating a corresponding pointing location, and wherein an attribute of the graphical element associated with a particular controlling user matches a corresponding attribute of the fiducial associated with the particular controlling user.
28. A computer program product for interactively controlling a multi-user interactive display system having a display including at least an information display region and a command control region, and a digital image capture system positioned to capture a time sequence images of users, comprising a non-transitory tangible computer readable storage medium storing an executable software application for causing a data processing system to perform the steps of:
receiving a time sequence of images captured by the digital image capture system;
using a data processing system to analyze the captured time sequence of images to detect a plurality of users who are viewing the display;
designating at least one of the detected plurality of users to be a controlling user;
displaying the captured time sequence of images in the command control region, wherein the detected users are demarked using graphical elements, and wherein the graphical element associated with the at least one controlling user is differentiated from the graphical elements associated with the other detected users to provide an indication of which of the plurality of users is the controlling user; and
using a data processing system to analyze the captured time sequence of images to detect a gesture made by the controlling user and updating content displayed in the information display region in response to the detected gesture.