Real-time video processing for pyramid holographic projections
Various techniques are described herein that provide for a real-time image or video processing system that is able to capture and stream or record/store video content of an object, and turn the captured content into a new video format that can be properly projected onto a pyramid holographic projector. In one specific embodiment, the techniques herein capture a video selfie of a user and stream it live or else store it for playback later as a saved message. Other embodiments, such as controlled avatars, animated characters, etc., may also be converted from a standard 2D format into a pyramid hologram format, either in real-time or else during post-processing, accordingly.
1. A method, comprising:
obtaining, by a process, one or more rectangular input images;
determining, by the process, a geometry of a pyramid holographic projector;
cropping, by the process, the one or more rectangular input images into one or more corresponding trapezoidal input images with a flat planar UV map based on the geometry of the pyramid holographic projector; and
producing, by the process, a pyramid holographic projection image source for projection on the pyramid holographic projector, the pyramid holographic projection image source stitching the one or more corresponding trapezoidal input images together on top of a transparent frame, wherein each of the one or more corresponding trapezoidal input images are positioned and rotated to form a shape corresponding to the geometry of the pyramid holographic projector, wherein the shape is centered in the transparent frame.
2. The method as in claim 1 , wherein obtaining comprises one or more of real-time image capture or stored image retrieval.
3. The method as in claim 1 , further comprising:
copying one or more of the trapezoidal input images to be stitched and projected on more than one side of the pyramid holographic projector.
4. The method as in claim 3 , wherein a same trapezoidal input image is stitched and projected on all sides of the pyramid holographic projector.
5. The method as in claim 1 , wherein obtaining comprises:
capturing a video selfie of a user.
6. The method as in claim 5 , further comprising:
separating an image of the user from a background environment.
7. The method as in claim 6 , wherein separating is based on depth-keying.
8. The method as in claim 6 , wherein separating is based on a separation technique selected from a group consisting of: chroma-keying; skeletal recognition; and the user being in front of a black background.
9. The method as in claim 1 , wherein the one or more rectangular input images are selected from a group consisting of: a single image; a plurality of separate images; a video stream of images; and a plurality of video streams of images.
10. The method as in claim 1 , wherein the one or more rectangular images are selected from a group consisting of: images of a user; images of an object; images of an avatar; and images of an animated character.
11. The method as in claim 1 , further comprising:
performing one or more image processing techniques on one or both of the one or more rectangular input images or one or more corresponding trapezoidal input images.
12. The method as in claim 1 , wherein one or more of determining, cropping, and producing are performed in real-time with the obtaining.
13. The method as in claim 1 , wherein one or more of determining, cropping, and producing are performed during post-processing after obtaining and storing the one or more rectangular input images.
14. The method as in claim 1 , further comprising:
streaming the produced pyramid holographic projection image source for real-time projection on the pyramid holographic projector.
15. The method as in claim 1 , further comprising:
storing the produced pyramid holographic projection image source for playback projection on the pyramid holographic projector.
16. The method as in claim 1 , wherein the geometry of a pyramid holographic projector is selected from a group consisting of: a three-sided pyramid; a four-sided pyramid; a three-sided pyramid with one side being a wall without a projected image; and a four-sided pyramid with one side being a wall without a projected image.
17. A tangible, non-transitory computer-readable media comprising instructions executable by a processor for executing a process comprising:
obtaining one or more rectangular input images;
determining a geometry of a pyramid holographic projector;
cropping the one or more rectangular input images into one or more corresponding trapezoidal input images with a flat planar UV map based on the geometry of the pyramid holographic projector; and
producing a pyramid holographic projection image source for projection on the pyramid holographic projector, the pyramid holographic projection image source stitching the one or more corresponding trapezoidal input images together on top of a transparent frame, wherein each of the one or more corresponding trapezoidal input images are positioned and rotated to form a shape corresponding to the geometry of the pyramid holographic projector, wherein the shape is centered in the transparent frame.
18. The computer-readable medium as in claim 17 , wherein obtaining comprises one or more of real-time image capture or stored image retrieval, and wherein the process further comprises one of either:
streaming the produced pyramid holographic projection image source for real-time projection on the pyramid holographic projector; or
storing the produced pyramid holographic projection image source for playback projection on the pyramid holographic projector.
19. The computer-readable medium as in claim 17 , wherein obtaining comprises:
capturing a video selfie of a user; and
separating an image of the user from a background environment.
20. The computer-readable medium as in claim 17 , wherein one or more of determining, cropping, and producing are performed either in real-time with the obtaining or during post-processing after obtaining and storing the one or more rectangular input images.