Customized passthrough

1. A method comprising:

rendering, on a virtual reality (VR) device, VR content;

receiving user activity data of a user associated with the VR device;

predicting a next segment of VR content based on the rendered VR content and user activity data;

predicting, based on the user activity data, user movement with respect to a real-world environment;

activating a passthrough feed;

based on the next segment of VR content and the predicted user movement in the real-world environment, customizing the passthrough feed based on incorporating VR content features into the passthrough feed, wherein a degree of the customizing the passthrough feed with the incorporated VR content features corresponds to an estimated duration during which the passthrough feed is activated; and

displaying the customized passthrough feed simultaneously with the VR content on the VR device.

2. The method of claim 1 , further comprising:

customizing the passthrough feed based on an event or purpose which triggered activation of the passthrough feed.

3. The method of claim 1 , further comprising:

customizing the passthrough feed based on a size of a display window of the passthrough feed relative to a size of a display window of the VR content.

4. The method of claim 1 , further comprising:

generating a custom style filter for customizing the passthrough feed, wherein the custom style filter is based on low-level features extracted from at least an image frame of the rendered VR content.

5. The method of claim 4 , wherein a degree of blending by the custom style filter of the passthrough feed and the VR content corresponds to a level of color intensity difference between the passthrough feed and the VR content.

6. The method of claim 5 , wherein the custom style filter comprises a pixel-manipulation-based filter.

7. The method of claim 1 , further comprising:

creating a style transfer neural network for customizing the passthrough feed, wherein the style transfer neural network is based on low-level features extracted from at least an image frame of the rendered VR content.

8. The method of claim 7 , wherein a degree of blending by the style transfer neural network of the passthrough feed and the VR content corresponds to a level of color intensity difference between the passthrough feed and the VR content.

9. The method of claim 7 , further comprising:

estimating a number of layers of the style transfer neural network, the number of layers based on at least one of: a duration during which the passthrough feed is activated, a level of color intensity difference between the VR content and the passthrough feed, an event or purpose which triggered activation of the passthrough feed, or a size of a display window of the passthrough feed relative to a size of a display window of the VR content.

10. The method of claim 1 , wherein customizing the passthrough feed further comprises:

selecting an object of interest from the VR content; and

overlaying the object of interest on a portion of the passthrough feed.

11. The method of claim 1 , wherein user activity data may be detected by at least one of an inward facing camera, eye tracker, external camera, inertial measurement unit or a wearable device coupled to the VR device.

12. The method of claim 1 , wherein the user activity data may comprise at least one of eye movement, facial expression, bodily movement, or biometric data.

13. A system comprising control circuitry configured to:

render, on a virtual reality (VR) device, VR content;

receive user activity data of a user associated with the VR device;

predict a next segment of VR content based on the rendered VR content and user activity data;

predict, based on the user activity data, user movement with respect to a real-world environment;

activate a passthrough feed;

based on the next segment of VR content and the predicted user movement in the real-world environment, customize the passthrough feed, based on incorporating VR content features into the passthrough feed, wherein a degree of the customizing the passthrough feed with the incorporated VR content features corresponds to an estimated duration during which the passthrough feed is activated; and

display the customized passthrough feed simultaneously with the VR content on the VR device.

14. The system of claim 13 , wherein the control circuitry is further configured to:

customize the passthrough feed based on an event or purpose which triggered activation of the passthrough feed.

15. The system of claim 13 , wherein the control circuitry is further configured to:

customize the passthrough feed based on a size of a display window of the passthrough feed relative to a size of a display window of the VR content.

16. The system of claim 13 , wherein the control circuitry is further configured to:

generate a custom style filter for customizing the passthrough feed, wherein the custom style filter is based on low-level features extracted from at least an image frame of the rendered VR content.

17. The system of claim 16 , wherein the custom style filter comprises a pixel-manipulation-based filter.

18. The system of claim 13 , wherein the control circuitry is further configured to:

create a style transfer neural network for customizing the passthrough feed, wherein the style transfer neural network is based on low-level features extracted from at least an image frame of the rendered VR content.