Systems and Methods for Updating Dynamic Three-Dimensional Displays with User Input
A dynamic three-dimensional image can be modified in response to poses or gestures, such as hand gestures, from a user. In one implementation, the gestures are made by a user who selects objects in the three-dimensional image. The gestures can include indications such as pointing at a displayed object, for example, or placing a hand into the volume of space occupied by the three-dimensional image to grab one or more of the displayed objects. In response to the gestures, the three-dimensional display is partially or completely redrawn, for example by an alteration or repositioning of the selected objects. In one implementation, a system simulates the dragging of a displayed three-dimensional object by a user who grabs and moves that object.
1 . A system comprising:
a display device for displaying a dynamic three-dimensional image;
a locating system configured to locate and recognize at least one pose made by a user; and
a processor coupled to the locating system and to the display device, and configured to communicate with the display device to revise the dynamic three-dimensional image in response to the pose.
2 . The system of claim 1 , wherein the processor is configured to modify a shape of at least one three-dimensional object in the three-dimensional image in response to the pose.
3 . The system of claim 1 , wherein the processor is configured to recognize a pointing pose, to identify an object in the dynamic three-dimensional image in response to the pointing pose, and to modify the dynamic three-dimensional image in response to the pointing pose.
4 . The system of claim 1 , wherein the processor is configured to recognize a grabbing pose, and to identify an object in the dynamic three-dimensional image as being grabbed.
5 . The system of claim 4 , wherein the processor is configured to reposition the object in response to a movement of the pose during the grabbing pose.
6 . The system of claim 1 , wherein the processor is configured to generate an output command signal for the control of physical objects in response to the pose.
7 . The system of claim 1 , wherein the processor is configured to generate an output command signal suitable for an adjunct processor, wherein:
the adjunct processor is configured to control an adjunct display device to revise an adjunct dynamic three-dimensional image in response to the output command signal.
8 . The system of claim 7 , wherein:
the adjunct processor is configured to revise the adjunct dynamic three-dimensional image in response to the detection by an adjunct locating system of a pose of a user of the adjunct display device; and
the processor is configured to receive, from the adjunct processor, an input command to revise the dynamic three-dimensional image in response to the pose of the user of the adjunct display device.
9 . The system of claim 1 , wherein the locating system is configured to locate and recognize poses made by a plurality of users, and the processor is configured to communicate with the display device to revise the dynamic three-dimensional image in response to the poses in a manner that facilitates communication among the users.
10 . The system of claim 1 , wherein the processor is configured to generate an output in response to a query from the user.
11 . The system of claim 10 , wherein the output comprises at least one of: an audible response to the query, a visual response to the query, or a haptic response to the query, and the query is at least one of: a spoken query, a pose query, a gestural query, or a keyboard-entered query.
12 . The system of claim 11 , wherein the processor is configured to generate an output in response to the query and to an accompanying pose indicating at least one object in the dynamic three-dimensional image.
13 . The system of claim 1 , wherein the processor is configured to generate an output in response to a command from the user.
14 . The system of claim 13 , wherein the output comprises modifying a shape of at least one three-dimensional object in the three-dimensional image in response to the command, and the command is at least one of: a spoken command, a pose command, a gestural command, or a keyboard-entered command.
15 . The system of claim 1 , wherein the processor is configured to update the dynamic three-dimensional image to display simultaneously a current situation and a desired situation.
16 . The system of claim 1 , wherein the pose is a component of a gesture.
17 . The system of claim 1 , further comprising at least one input device recognizable by the locating system and configured to enable a user to express the pose.
18 . The system of claim 17 , further comprising:
a first set of one or more tags mounted on the input device and recognizable by the locating system; and
a second set of one or more tags located with a fixed spatial relationship to the image and recognizable by the locating system.
19 . The system of claim 17 , wherein the input device comprises a glove wearable by a user, and the dynamic three-dimensional image comprises fully-shaded objects.
20 . A system comprising:
a display device for displaying a three-dimensional image in a display volume;
a locating system configured to locate and recognize at least one pose made by a user in an interaction volume that overlaps at least partially with the display volume; and
a processor coupled to the locating system and to the display device, and configured to register a pose in the interaction volume as being spatially related to the three-dimensional image in the display volume.
21 . The system of claim 20 , wherein the processor is configured to modify a shape of at least one three-dimensional object in the three-dimensional image in response to the pose.
22 . The system of claim 20 , wherein the processor is configured to recognize at least one of a pointing pose or a grabbing pose, to identify an object in the three-dimensional image in response to the identified pose, and to modify the three-dimensional image in response to the identified pose.
23 . The system of claim 22 , wherein the processor is configured to reposition the identified object in response to a movement of the identified pose.
24 . The system of claim 20 , wherein the processor is configured to generate an output command signal for the control of physical objects in response to the pose.
25 . The system of claim 20 , wherein the processor is configured to generate an output command signal suitable for an adjunct processor, wherein:
the adjunct processor is configured to control an adjunct display device to revise an adjunct three-dimensional image in response to the output command signal;
the adjunct processor is configured to revise the adjunct three-dimensional image in response to the detection by an adjunct locating system of a pose of a user of the adjunct display device; and
the processor is configured to receive, from the adjunct processor, an input command to revise the three-dimensional image in response to the pose of the user of the adjunct display device.
26 . The system of claim 20 , wherein the locating system is configured to locate and recognize poses made by a plurality of users, and the processor is configured to communicate with the display device to revise the three-dimensional image in response to the poses in a manner that facilitates communication among the users.
27 . The system of claim 20 , wherein the processor is configured to generate a response to a query from the user, and to generate an output in response to a command from the user.
28 . A method comprising:
displaying a dynamic three-dimensional image;
locating and recognizing at least one pose made by a user; and
revising the dynamic three-dimensional image in response to the pose.
29 . The method of claim 28 , wherein the revising comprises modifying a shape of at least one three-dimensional object in the three-dimensional image in response to the pose.
30 . The method of claim 28 , wherein the pose comprises a pointing pose, the method further comprising:
identifying an object in the dynamic three-dimensional image in response to the pointing pose, and
modifying the dynamic three-dimensional image in response to the pointing pose.
31 . The method of claim 28 , wherein the pose comprises a grabbing pose, the method further comprising:
identifying an object in the dynamic three-dimensional image as being grabbed.
32 . The method of claim 31 , further comprising:
repositioning the object in response to a movement of the pose during the grabbing pose.
33 . The method of claim 28 , wherein the dynamic three-dimensional image comprises fully-shaded dynamic objects, the method further comprising:
generating an output command signal for the control of physical objects in response to the pose.
34 . The method of claim 28 , further comprising:
generating an output command signal to control an adjunct display device to revise an adjunct dynamic three-dimensional image.
35 . The method of claim 28 , further comprising:
locating and recognizing poses made by a plurality of users; and
revising the dynamic three-dimensional image in response to the poses in a manner that facilitates communication among the users.
36 . The method of claim 28 , further comprising:
generating an output in response to a query from a user and to an accompanying pose that indicates at least one object in the dynamic three-dimensional image.
37 . The method of claim 28 , further comprising:
revising the dynamic three-dimensional image in response to in response to a command from a user and to an accompanying pose that indicates at least one object in the dynamic three-dimensional image.
38 . The method of claim 28 , further comprising:
updating the dynamic three-dimensional image to display simultaneously a current situation and a desired situation.
39 . A method comprising:
displaying a three-dimensional image in a display volume;
locating and recognizing at least one pose made by a user in an interaction volume that overlaps at least partially with the display volume; and
registering a pose in the interaction volume as being spatially related to the three-dimensional image in the display volume.
40 . The method of claim 39 , further comprising:
modifying a shape of at least one three-dimensional object in the three-dimensional image in response to the pose.
41 . The method of claim 39 , further comprising:
recognizing at least one of a pointing pose or a grabbing pose;
identifying an object in the three-dimensional image in response to the identified pose; and
modifying the three-dimensional image in response to the identified pose.
42 . The method of claim 41 , further comprising:
repositioning the identified object in response to a movement of the identified pose.
43 . The method of claim 39 , further comprising:
generating an output command signal for the control of physical objects in response to the pose.
44 . The method of claim 39 , further comprising:
generating an output command signal configured to control an adjunct display device to revise an adjunct three-dimensional image in response to the output command signal.
45 . The method of claim 39 , further comprising:
locating and recognizing poses made by a plurality of users; and
revising the three-dimensional image in response to the poses in a manner that facilitates communication among the users.
46 . The method of claim 39 , wherein the processor is configured to generate a response to a query from the user, and to generate an output in response to a command from the user.
47 . A system comprising:
means for displaying a dynamic three-dimensional image;
means for locating and recognizing at least one pose made by a user; and
means for revising the dynamic three-dimensional image in response to the pose.
48 . A system comprising:
means for displaying a three-dimensional image in a display volume;
means for locating and recognizing at least one pose made by a user in an interaction volume that overlaps at least partially with the display volume; and
means for registering a pose in the interaction volume as being spatially related to the three-dimensional image in the display volume.
49 . A computer program product comprising:
a computer readable medium; and
instructions encoded on the computer readable medium and executable by one or more processors to perform a method comprising:
displaying a dynamic three-dimensional image;
locating and recognizing at least one pose made by a user; and
revising the dynamic three-dimensional image in response to the pose.
50 . A computer program product comprising:
a computer readable medium; and
instructions encoded on the computer readable medium and executable by one or more processors to perform a method comprising:
displaying a three-dimensional image in a display volume;
locating and recognizing at least one pose made by a user in an interaction volume that overlaps at least partially with the display volume; and
registering a pose in the interaction volume as being spatially related to the three-dimensional image in the display volume.