ENHANCING VIDEOS OF PEOPLE INTERACTING WITH VIRTUAL OBJECTS IN AN EXTENDED REALITY ENVIRONMENT
A system and method for generating videos of individuals interacting with virtual objects. A wearable extended reality appliance generates an extended reality environment including at least one virtual object. First image data reflects a first perspective of an individual wearing the wearable extended reality appliance, including physical hand movements interacting with the at least one virtual object from the first perspective. Second image data reflects a second perspective facing the individual, including second physical hand movements interacting with a virtual object from the second perspective. The first image data and the second image data are analyzed to determine an interaction with the virtual object. The rendered representation of the virtual object from the second perspective is melded with the second image data to generate a video of the individual interacting with the virtual object from the second perspective.
1 - 60 . (canceled)
61 . A non-transitory computer readable medium containing instructions for causing at least one processor to perform operations for generating videos of individuals interacting with virtual objects, the operations comprising:
causing a wearable extended reality appliance to generate a presentation of an extended reality environment including at least one virtual object;
receiving first image data from at least a first image sensor, the first image data reflecting a first perspective of an individual wearing the wearable extended reality appliance;
receiving second image data from at least a second image sensor, the second image data reflecting a second perspective facing the individual;
identifying in the first image data first physical hand movements interacting with the at least one virtual object from the first perspective;
identifying in the second image data second physical hand movements interacting with the at least one virtual object from the second perspective;
analyzing at least one of the first image data or the second image data to determine an interaction with the at least one virtual object;
rendering for display a representation of the at least one virtual object from the second perspective; and
melding the rendered representation of the at least one virtual object from the second perspective with the second image data to generate a video of the individual interacting with the at least one virtual object from the second perspective.
62 . The non-transitory computer readable medium of claim 61 , wherein the first image sensor is part of the wearable extended reality appliance.
63 . The non-transitory computer readable medium of claim 61 , wherein the second image sensor is part of a computing device selectively connected to the wearable extended reality appliance.
64 . The non-transitory computer readable medium of claim 61 , wherein the operations further include analyzing at least one of the first image data or the second image data to determine an absence of interaction with a particular virtual object, and wherein the generated video excludes a representation of the particular virtual object.
65 . The non-transitory computer readable medium of claim 61 , wherein the identified first physical hand movements are associated with a gesture for causing a movement of the at least one virtual object, and wherein rendering the representation of the at least one virtual object from the second perspective reflects the movement of the at least one virtual object.
66 . The non-transitory computer readable medium of claim 61 , wherein the identified first physical hand movements are associated with a gesture for causing a modification to a visual appearance of a portion of a surface of the at least one virtual object, the portion of the surface is visible from the second perspective and is not visible from the first perspective; and wherein rendering the representation of the at least one virtual object from the second perspective reflects the modification to the visual appearance of the surface of the at least one virtual object.
67 . The non-transitory computer readable medium of claim 61 , wherein the operations further include determining a position of the at least one virtual object in the extended reality environment, and wherein rendering the representation of the at least one virtual object from the second perspective is based on the determined position.
68 . The non-transitory computer readable medium of claim 61 , wherein the at least one virtual object includes text presented by the wearable extended reality appliance on a side of the at least one virtual object that faces the individual, and wherein generating the video includes providing a representation of the text in the video.
69 . The non-transitory computer readable medium of claim 61 , wherein rendering the representation of the at least one virtual object includes determining an opacity for the representation of the at least one virtual object.
70 . The non-transitory computer readable medium of claim 69 , wherein the opacity is determined to be less than 75% when the at least one virtual object obscures at least a portion of the individual from the second perspective.
71 . The non-transitory computer readable medium of claim 61 , wherein the at least one virtual object contains private information, and wherein rendering the representation of the at least one virtual object includes obscuring the private information.
72 . The non-transitory computer readable medium of claim 61 , wherein the operations further include rendering for display a representation of the extended reality environment from the second perspective, and generating an additional video of the individual in the extended reality environment interacting with the at least one virtual object from the second perspective.
73 . The non-transitory computer readable medium of claim 61 , wherein the operations further include artificially deleting the wearable extended reality appliance from the second image data; and generating the video of the individual, without the wearable extended reality appliance, interacting with the at least one virtual object from the second perspective.
74 . The non-transitory computer readable medium of claim 61 , wherein the operations further include causing the wearable extended reality appliance to present a preview of the video of the individual interacting with the at least one virtual object from the second perspective while the individual is interacting with the at least one virtual object.
75 . The non-transitory computer readable medium of claim 61 , wherein the operations further include sharing the generated video with at least one other individual while the individual wearing the wearable extended reality appliance is interacting with the at least one virtual object.
76 . The non-transitory computer readable medium of claim 75 , wherein the operations further include receiving input that an additional video of the individual interacting with the at least one virtual object from the first perspective is preferred, melding a representation of the at least one virtual object from the first perspective with the first image data to generate the additional video of the individual interacting with the at least one virtual object from the first perspective; and switching from the video from the second perspective to the additional video from the first perspective.
77 . The non-transitory computer readable medium of claim 61 , wherein the at least one virtual object includes a first object in a position visible from the second perspective and a second object in a position hidden from the second perspective; and wherein the generated video includes a representation of the first object from the second perspective in a first visual format and a representation of the second object from the second perspective in a second visual format, the second visual format differs from the first visual format and indicative of the second object being in the position hidden from the second perspective.
78 . The non-transitory computer readable medium of claim 61 , wherein in the generated video the rendered representation of the at least one virtual object from the second perspective hides the physical hand and includes a virtual representation of the physical hand.
79 . A system for generating videos of individuals interacting with virtual objects, the system comprising:
at least one processor programmed to:
cause a wearable extended reality appliance to generate a presentation of an extended reality environment including at least one virtual object;
receive first image data from at least a first image sensor, the first image data reflecting a first perspective of an individual wearing the wearable extended reality appliance;
receive second image data from at least a second image sensor, the second image data reflecting a second perspective facing the individual;
identify in the first image data first physical hand movements interacting with the at least one virtual object from the first perspective;
identify in the second image data second physical hand movements interacting with the at least one virtual object from the second perspective;
analyze at least one of the first image data or the second image data to determine an interaction with the at least one virtual object;
render for display a representation of the at least one virtual object from the second perspective; and
meld the rendered representation of the at least one virtual object from the second perspective with the second image data to generate a video of the individual interacting with the at least one virtual object from the second perspective.
80 . A method for generating videos of individuals interacting with virtual objects, the method comprising:
causing a wearable extended reality appliance to generate a presentation of an extended reality environment including at least one virtual object;
receiving first image data from at least a first image sensor, the first image data reflecting a first perspective of an individual wearing the wearable extended reality appliance;
receiving second image data from at least a second image sensor, the second image data reflecting a second perspective facing the individual;
identifying in the first image data first physical hand movements interacting with the at least one virtual object from the first perspective;
identifying in the second image data second physical hand movements interacting with the at least one virtual object from the second perspective;
analyzing at least one of the first image data or the second image data to determine an interaction with the at least one virtual object;
rendering for display a representation of the at least one virtual object from the second perspective; and
melding the rendered representation of the at least one virtual object from the second perspective with the second image data to generate a video of the individual interacting with the at least one virtual object from the second perspective.
81 - 160 . (canceled)