Systems and methods for video data depth determination and video modification
System, process, and device configurations are provided for identifying depth of gaming elements. A method can include detecting at least one game object in video data, wherein the at least one game object is a display element and determining a depth parameter for the at least one game object, wherein the depth parameter is a distance value of the at least one game object relative to at least one background display element for the video data. The video data may be output using the depth parameter for the at least one game object. Embodiments can create title specific data from analysis of rendered video data to determine modifications for adding depth and modifying rendering of titles. Depth determinations can include features to allow for different rendering styles and presentation in a three-dimensional format or by a headset. Processes and configurations are also provided for generating contextual information about game objects.
1 . A method comprising:
determining, using one or more machine learning models, a rendering style of video data;
detecting, by a device, at least one object in the video data;
determining, by the device, a set of one or more graphical features that indicate depth in the video data based on the determined rendering style of the video data;
determining, by the device, a depth parameter for the at least one object from the rendered video data based on the set of one or more graphical features that indicate depth in the video data;
modifying, by the device and based on the determined rendering style of the video data, the video data using the determined depth parameter for the at least one object; and
outputting, by the device, the modified video data.
2 . The method of claim 1 , wherein detecting the at least one object of the video data includes decoding at least a part of the video data using at least one trained object identification parameter.
3 . The method of claim 1 , wherein the depth parameter identifies order of the at least one object relative to a graphical background of the video data based on pixel overlap of the object to the graphical background.
4 . The method of claim 1 , wherein the depth parameter is determined based on overlap of at least one element with the at least one object.
5 . The method of claim 1 , wherein the style used in rendering the video data includes at least one of a flat, top down, front view, front view with floor, angled down view, or first person view rendering style.
6 . The method of claim 1 , wherein the depth parameter includes a value for modifying presentation of the at least one object relative to a graphical background of the video data.
7 . The method of claim 1 , wherein determining the depth parameter for the at least one object from the rendered video data comprises:
associating a plane, selected from a set of multiple planes each indicating a distance reference, with the at least one object, wherein modifying the video data using the determined depth parameter for the at least one object comprises:
updating a display configuration from a two-dimensional display configuration to a three-dimensional display configuration including updating position of the at least one object based on the associated plane selected from the set of multiple planes.
8 . The method of claim 1 , wherein modifying the video data using the determined depth parameter comprises:
updating presentation of the object using the depth parameter to distance the object from a graphical background using the depth parameter for the at least one object.
9 . The method of claim 1 , wherein determining the depth parameter comprises:
providing data representing the rendered video data to the one or more machine learning models, wherein the one or more machine learning models are trained to generate at least one of object identification data or object order data for input video data based on training operations for a media title; and
obtaining output from the one or more machine learning models processing the data representing the rendered video data as at least part of the depth parameter.
10 . A device comprising:
an interface;
a memory storing executable instructions; and
a controller coupled to the interface and the memory, wherein the executable instructions are operable, when executed by the controller, to cause the controller to perform operations comprising:
determining, using one or more machine learning models, a rendering style of video data;
detecting, at least one object in the video data;
determining a set of one or more graphical features that indicate depth in the video data based on the determined rendering style of the video data;
determining a depth parameter for the at least one object from the rendered video data based on the set of one or more graphical features that indicate depth in the video data;
modifying, by the device and based on the determined rendering style of the video data, the video data using the determined depth parameter for the at least one object; and
outputting output the modified video data.
11 . The device of claim 10 , wherein detecting the at least one object of the video data includes decoding at least a part of the video data using at least one trained object identification parameter.
12 . The device of claim 10 , wherein the object is detected in rendered two-dimensional video data, the video data including a graphical background.
13 . The device of claim 10 , wherein the depth parameter identifies order of the at least one object relative to a graphical background of the video data based on pixel overlap of the object to the graphical background.
14 . The device of claim 10 , wherein the depth parameter is determined based on overlap of at least one element with the at least one object.
15 . The device of claim 10 , wherein the style used in rendering the video data includes at least one of a flat, top down, front view, front view with floor, angled down view, or first person view rendering style.
16 . The device of claim 10 , wherein the depth parameter includes a value for modifying presentation of the at least one object relative to a graphical background of the video data.
17 . The device of claim 10 , wherein determining the depth parameter for the at least one object from the rendered video data comprises:
associating a plane, selected from a set of multiple planes each indicating a distance reference, with the at least one object, wherein modifying the video data using the determined depth parameter for the at least one object comprises:
updating a display configuration from a two-dimensional display configuration to a three-dimensional display configuration including updating position of the at least one object based on the associated plane selected from the set of multiple planes.
18 . The device of claim 10 , wherein modifying the video data using the determined depth parameter comprises:
updating presentation of the object using the depth parameter to distance the object from a graphical background using the depth parameter for the at least one object.
19 . The device of claim 10 , wherein determining the depth parameter comprises:
providing data representing the rendered video data to the one or more machine learning models, wherein the one or more machine learning models are trained to generate at least one of object identification data or object order data for input video data based on training operations for a media title; and
obtaining output from the one or more machine learning models processing the data representing the rendered video data as at least part of the depth parameter.
20 . One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
determining, using one or more machine learning models, a rendering style of video data;
detecting, by a device, at least one object in the video data;
determining, by the device, a set of one or more graphical features that indicate depth in the video data based on the determined rendering style of the video data;
determining, by the device, a depth parameter for the at least one object from the rendered video data based on the set of one or more graphical features that indicate depth in the video data;
modifying, by the device and based on the determined rendering style of the video data, the video data using the determined depth parameter for the at least one object; and
outputting, by the device, the modified video data.