Method and system for reconstructing obstructed face portions for virtual reality environment
A method and a system for reconstructing obstructed face portions are provided herein. The method may include the following steps: obtaining off-line 3D data, being 3D data of a head of a person not wearing a face-obstructing object, being an object which obstructs a portion of the face of the person; obtaining in real time, real-time 3D data, being 3D data of said head, wherein said person wears said face-obstructing object; applying a 3D transformation to at least a portion of the off-line 3D data, based on the real-time 3D data, to yield reconstructed real time 3D data, being real-time 3D data related to the obstructed face portions; and merging the reconstructed real time 3D data into the real-time 3D data. The system may implement the aforementioned steps over a computer processor.
1. A method comprising:
obtaining, based on sensor data captured by a sensor on a face-obstructing object, data indicative of face portions of a person obstructed by the face-obstructing object;
reconstructing a 3D face image of the obstructed face portions based on the obtained data; and
merging the reconstructed 3D face image into a respective 3D image of the person.
2. The method according to claim 1 , further comprising presenting the merged image to a viewing user via a display.
3. The method of claim 1 , further comprising capturing an image of a scene containing the person wearing the face-obstructing object.
4. The method according to claim 1 , wherein the obtaining data indicative of face portions obstructed by the face-obstructing object comprises sensing the obstructed face portions from an inner side of the face-obstructing object.
5. The method according to claim 4 , wherein the sensing is of facial gestures, and wherein the reconstructing the 3D face image is carried out by modeling sensed gestures to change the face relative to a base image of the obstructed face portions captured previously.
6. The method according to claim 1 , wherein the obtained data comprises an image of the obstructed face portions.
7. The method according to claim 1 , further comprising retrieving a base image of the person from a social network.
8. The method according to claim 7 , further comprising modifying the base image of the person based on an environmental condition affecting the person.
9. The method according to claim 1 , wherein the obtaining data indicative of face portions obstructed by the face-obstructing object comprises obtaining a base image of the obstructed face portions, which has been previously captured.
10. The method according to claim 9 , wherein the reconstructing the 3D face image comprises monitoring at least one of: position, orientation, or profile; and applying the obtained data onto a three dimensional model of the face of the person to yield a modified face image.
11. The method according to claim 9 , wherein the reconstructing the 3D face image comprises monitoring vital signs of the at least one person wearing the face-obstructing object, and applying the obtained data onto a three dimensional model of the face of the at least one person, to yield a modified face image.
12. A system comprising:
a computer processor configured to:
obtain, from a sensor on a face-obstructing object, data indicative of face portions of a person obstructed by the face-obstructing object;
reconstruct a 3D face image of the obstructed face portions based on the obtained data; and
merge the reconstructed 3D face image into a respective 3D image of the person.
13. The system according to claim 12 , further comprising a display configured to present the merged image.
14. The system according to claim 12 , further comprising an image capturing device configured to capture an image of a scene containing the person wearing the face-obstructing object.
15. The system according to claim 12 , wherein the obtaining data indicative of face portions obstructed by the face-obstructing object is carried out by sensors configured to sense the obstructed face portions from an inner side of the face-obstructing object.
16. The system according to claim 15 , wherein the sensing is of facial gestures, and wherein the reconstructing the 3D face image is carried out by modeling sensed gestures to change the face relative to a base image of the obstructed face portions captured previously.
17. The system according to claim 12 , wherein the obtained data comprises an image of the obstructed face portions.
18. The system according to claim 12 , wherein the computer processor is further configured to retrieve a base image of the person from a social network.
19. The system according to claim 18 , wherein the computer processor is further configured to modify the base image of the person based on an environmental condition affecting the person.
20. A computer program product comprising a non-transitory computer-readable storage medium containing computer program code for:
obtaining, based on sensor data captured by a sensor on a face-obstructing object, data indicative of face portions of a person obstructed by the face-obstructing object;
reconstructing a 3D face image of the obstructed face portions based on the obtained data; and
merging the reconstructed 3D face image into a respective 3D image of the person.