Stereo video for gaming
View Patent ↗A real-time stereo video signal of a captured scene with a physical foreground object and a physical background is received. In real-time, a foreground/background separation algorithm is used on the real-time stereo video signal to identify pixels from the stereo video signal that represent the physical foreground object. A video sequence may be produced by rendering a 3D virtual reality based on the identified pixels of the physical foreground object.
1. A system, comprising:
a processor; and
memory comprising instructions that, when executed by the processor, perform a method comprising:
receiving from a stereo camera a real-time stereo video signal comprising paired left and right images of a captured scene, the captured scene comprising a physical foreground object and a physical background;
scaling a kernel image associated with the physical foreground object using a first disparity map, the scaling based upon a distance of a portion of the physical foreground object relative to the stereo camera;
performing, in real-time, a foreground/background separation algorithm on the real-time stereo video signal, the foreground/background separation algorithm configured to compute differences between portions of the real-time stereo video signal and store the differences as a second disparity map; and
comparing the second disparity map to at least a portion of the scaled kernel image to identify pixels of the real-time stereo video signal that represent the physical foreground object.
2. The system of claim 1 , the physical foreground object comprising a first aspect and a second aspect.
3. The system of claim 2 , the first aspect comprising a human face.
4. The system of claim 1 , the method comprising providing the identified pixels that represent the physical foreground object to a two dimensional game.
5. The system of claim 1 , the method comprising providing the identified pixels that represent the physical foreground object to a three dimensional game.
6. A computer-readable storage device having computer-executable instructions that, when executed by a processor, are configured to perform a method, the method comprising:
receiving from a stereo camera a stereo video signal comprising paired left and right images of a captured scene, the captured scene comprising a physical foreground object and a physical background;
scaling a kernel image associated with the physical foreground object using a first disparity map, the scaling based upon a distance of a portion of the physical foreground object relative to the stereo camera;
performing a foreground/background separation algorithm on the stereo video signal, the foreground/background separation algorithm configured to compute differences between portions of the stereo video signal and store the differences as a second disparity map; and
comparing the second disparity map to at least a portion of the scaled kernel image to identify pixels of the stereo video signal that represent the physical foreground object.
7. The computer-readable storage device of claim 6 , the physical foreground object comprising a first aspect and a second aspect, the first aspect comprising a human face.
8. The computer-readable storage device of claim 6 , comprising providing the identified pixels that represent the physical foreground object to at least one of a two dimensional game or a three dimensional game.
9. A game console comprising the computer readable storage device of claim 6 .
10. A method, comprising:
receiving from a stereo camera a stereo video signal comprising paired left and right images of a captured scene, the captured scene comprising a physical foreground object and a physical background;
scaling a kernel image associated with the physical foreground object using a first disparity map, the scaling based upon a distance of a portion of the physical foreground object relative to the stereo camera;
performing a foreground/background separation algorithm on the stereo video signal, the foreground/background separation algorithm configured to compute differences between portions of the stereo video signal and store the differences as a second disparity map; and
comparing the second disparity map to at least a portion of the scaled kernel image to identify pixels of the stereo video signal that represent the physical foreground object.
11. A game console, comprising:
a processor; and
memory comprising instructions that, when executed by the processor, perform the method of claim 10 .
12. The method of claim 10 , the physical foreground object comprising a first aspect and a second aspect.
13. The method of claim 12 , the first aspect comprising a human face.
14. The method of claim 10 , comprising providing the identified pixels that represent the physical foreground object to a two dimensional game.
15. The method of claim 10 , comprising providing the identified pixels that represent the physical foreground object to a three dimensional game.
16. The method of claim 10 , the stereo video signal comprising a real-time stereo video signal.
17. The method of claim 10 , performing the foreground/background separation algorithm comprising performing the foreground/background separation algorithm in real-time.
18. The method of claim 10 , comprising using a third disparity map to generate the kernel image.
19. The method of claim 10 , the kernel image comprising an array of pixel values.
20. The method of claim 10 , the first disparity map different than the second disparity map.