Interactive video display system
View Patent ↗A device allows easy and unencumbered interaction between a person and a computer display system using the person's (or another object's) movement and position as input to the computer. In some configurations, the display can be projected around the user so that that the person's actions are displayed around them. The video camera and projector operate on different wavelengths so that they do not interfere with each other. Uses for such a device include, but are not limited to, interactive lighting effects for people at clubs or events, interactive advertising displays, etc. Computer-generated characters and virtual objects can be made to react to the movements of passers-by, generate interactive ambient lighting for social spaces such as restaurants, lobbies and parks, video game systems and create interactive information spaces and art installations. Patterned illumination and brightness and gradient processing can be used to improve the ability to detect an object against a background of video images.
1. A system comprising:
a camera system operable to provide images of an object;
a display system operable to render video images onto a surface; and
a processor coupled to receive said images from said camera system and coupled to drive said display system, said processor operable to determine a strength and a direction of interaction between the object and at least one of the video images.
2. A system in accordance with claim 1 , wherein said processor is further operable to determine an influence image that includes a region around an object image derived from the object, and wherein said strength and said direction of interaction are derived from said influence image.
3. A system in accordance with claim 2 , wherein said processor is farther operable to determine a gradient of the influence image and to derive said direction of interaction based on said gradient.
4. A system in accordance with claim 3 , wherein said processor is further operable to determine a brightness of the influence image and to derive said strength of interaction based on said brightness.
5. A system in accordance with claim 2 , wherein said processor is further operable to determine a brightness of the influence image and to derive said strength of interaction based on said brightness.
6. A system in accordance with claim 1 , wherein said interaction is the object pushing the at least one video image.
7. A system in accordance with claim 1 , wherein said interaction is the object touching the at least one video image.
8. A system in accordance with claim 1 , wherein said interaction is the object deforming the at least one video image.
9. A system in accordance with claim 1 , wherein said interaction is the object manipulating the at least one video image.
10. A system in accordance with claim 1 , wherein said processor is further operable to generate an adaptive model of a background to said object by analyzing changes over time in frames of the images from said camera system.
11. A system in accordance with claim 10 , wherein said processor is further operable to generate said video images based on said adaptive model and said images from said camera system.
12. The system as described in claim 10 wherein said processor is further operable to generate said adaptive model of said background by identifying pixels in said frames having substantially constant brightness over time.
13. The system as described in claim 12 wherein said processor is further operable to generate said adaptive model of said background by computing median values over time for said pixels.
14. The system as described in claim 10 wherein said processor is further operable to generate said adaptive model of said background by computing median values over time for pixels in said frames.
15. The system as described in claim 10 wherein said processor is further operable to generate said adaptive model of said background by incorporating into said adaptive model of said background changes in said background that occur over time.
16. The system as described in claim 10 wherein said processor is further operable to generate said adaptive model of said background by incorporating into said adaptive model of said background changes in said background that occur due to changes in lighting affecting said background.
17. The system as described in claim 10 wherein said processor is further operable to generate said adaptive model of said background by creating a model image from said images from said camera system that classifies pixels in said model image as either foreground or background.
18. The system as described in claim 10 wherein said processor is further operable to generate said adaptive model of said background by incorporating into said adaptive model of said background information relating to at least a portion of said object that has remained substantially stationary for a period of time.
19. The system as described in claim 10 wherein said processor is further operable to generate said adaptive model of said background by computing a weighted average of a current frame of said images from said camera system with said adaptive model of said background.
20. The system as described in claim 19 wherein said processor is further operable to tune said weighted average to change a rate at which said model of said background adapts to changes in said images from said camera system.
21. The system as described in claim 10 wherein said processor is further operable to generate said display item by distinguishing between foreground that corresponds to said object and said background.
22. The system as described in claim 21 wherein said distinguishing comprises comparing a current frame of said images from said camera system with said adaptive model of said background.
23. The system as described in claim 22 wherein said distinguishing further comprises determining if differences between corresponding pixels in said current frame of said images from said camera system and said adaptive model of said background are greater than a threshold to determine location of said object.
24. The system as described in claim 10 wherein said camera system comprises two cameras to provide a stereo image, and wherein said processor is further operable to compute depth data based on said stereo image and to use said depth data to generate said model of said background.
25. A system in accordance with claim 1 , wherein said camera system comprises a light source that is strobed.
26. A system in accordance with claim 25 , wherein said processor is further operable to process said images from said camera system based on said strobed light source.
27. A system in accordance with claim 1 , wherein said processor is further operable to detect a lighting phenomenon due to sources of light other than said camera system light source.
28. A system in accordance with claim 27 , wherein said lighting phenomenon comprises shadows.
29. A system in accordance with claim 10 , wherein at least a portion of said surface coincides with said background.
30. A system in accordance with claim 1 , wherein said processor is further operable to detect said object and positions thereof.
31. A system in accordance with claim 1 , wherein said video images comprise an advertisement.
32. A system in accordance with claim 1 , wherein said processor is further operable to generate a display item having an animated characteristic.
33. A system in accordance with claim 32 , wherein said processor is further operable to change said animated characteristic of said display item responsive to said interaction.