System, method, and computer program for automatically generating branch scenes for a branching video
This disclosure pertains to a system, method, and computer program for automatically creating branching scenes for videos within a video production workspace. The workspace offers a call-to-action feature that enables a user to initiate the automated creation of branching scenes using a natural language request. In response to receiving a request for branching scenes, the system identifies the video's current state within the workspace, discerning scenes, assets, and timelines within the video. System-defined attributes of the assets in the current state, as well as user-defined attributes for the branch, guide the branching process. These attributes are inputted into a generative AI model, which generates a branching narrative for the video and outputs structured data files describing the branching scenes. The branching scenes are rendered in the workspace for user review and refinement. Users can further refine scenes by inputting further instructions for the AI model.
1 . A method, performed by a computer system, for automatically creating branching scenes for branching videos, the method comprising:
providing a multimedia video production workspace for creating scenes for a video, wherein the multimedia video production workspace includes a video timeline that illustrates an order and time in which scenes in the video appear;
providing a call-to-action in the video production workspace to enable a user to initiate automated creation of scenes for a branching narrative in a video being created within the video production workspace;
in response to receiving user input to create scenes for a branching narrative in a video being created in the video production workspace, performing the following:
identifying a current state of the video in the video production workspace, including identifying all the assets in the current state of the video;
obtaining metadata related to assets in the current state of the video;
inputting said metadata into a generative AI model configured to generate a branching narrative for the video and output data describing a plurality of branching scenes for the branching narrative; and
rendering the branching scenes in the video production workspace in accordance with the data outputted by the generative AI model, wherein the branching scenes are incorporated into the video timeline such that child branching scenes follow a parent branch scene.
2 . The method of claim 1 , further comprising receiving natural language input from the user with user-defined attributes for the branching narrative, and inputting said user input into the generative AI model in addition to the metadata related to the current state of the video to obtain a branching narrative and corresponding branching scenes that comply with the user's natural language input.
3 . The method of claim 1 , wherein the output data from the generative AI model describes at least two child branching scenes.
4 . The method of claim 3 , wherein the output data further describes a parent branch scene.
5 . The method of claim 1 , further comprising enabling a user to iteratively transform the branching scenes via a natural language interface.
6 . The method of claim 1 , wherein metadata for an asset includes metadata tags for the asset, one or more scenes associated with the asset, an asset position, an asset size, and timeline data associated with the asset.
7 . The method of claim 6 , wherein identifying attributes of the assets in the current state of the video comprising using a computer vision model to classify a visual asset in the video with one or more attributes.
8 . A video production system for automatically creating branching scenes for branching videos comprising:
a video workspace module (VWM) that provides a multimedia video production workspace for creating scenes for a video, wherein the VWM enables a user to input a branch generation request, and wherein the multimedia video production workspace includes a video timeline that illustrates an order and time in which scenes in the video appear;
a branch creation module (BCM) configured to receive both the branch generation request and metadata from the VWM related to video scenes in the video production workspace and configured to generate a branching narrative for the video, including a parent branch and two or more child branches; and
a generative AI model integrated within the BCM, wherein the BCM utilizes the generative AI model to produce data describing a plurality of branching scenes for the branching narrative.
9 . The system of claim 8 , wherein the BCM utilizes the generative AI model to produce a question and multiple choice answers related to the branching narrative, with each answer leading to a subsequent scene described in the data outputted by the generative AI model.
10 . The system of claim 9 , wherein the output of the BCM is a structured data file that is readable by the VWM to render the branching video scenes in the video production workspace.
11 . A non-transitory computer-readable medium comprising a computer program, that, when executed by a computer system, enables the computer system to perform the following method for automatically creating branching scenes for branching videos, the method comprising:
providing a multimedia video production workspace for creating scenes for a video, wherein the multimedia video production workspace includes a video timeline that illustrates an order and time in which scenes in the video appear;
providing a call-to-action in the video production workspace to enable a user to initiate automated creation of scenes for a branching narrative in a video being created within the video production workspace;
in response to receiving user input to create scenes for a branching narrative in a video being created in the video production workspace, performing the following:
identifying a current state of the video in the video production workspace, including identifying all the assets in the current state of the video;
obtaining metadata related to assets in the current state of the video;
inputting said metadata into a generative AI model configured to generate a branching narrative for the video and output data describing a plurality of branching scenes for the branching narrative; and
rendering the branching scenes in the video production workspace in accordance with the data outputted by the generative AI model, wherein the branching scenes are incorporated into the video timeline such that child branching scenes follow a parent branch scene.
12 . The non-transitory computer-readable medium of claim 11 , further comprising receiving natural language input from the user with user-defined attributes for the branching narrative, and inputting said user input into the generative AI model in addition to the metadata related to the current state of the video to obtain a branching narrative and corresponding branching scenes that comply with the user's natural language input.
13 . The non-transitory computer-readable medium of claim 11 , wherein the output data from the generative AI model describes at least two child branching scenes.
14 . The non-transitory computer-readable medium of claim 13 , wherein the output data further describes a parent branch scene.
15 . The non-transitory computer-readable medium of claim 11 , further comprising enabling a user to iteratively transform the branching scenes via a natural language interface.
16 . The non-transitory computer-readable medium of claim 11 , wherein metadata for an asset includes metadata tags for the asset, one or more scenes associated with the asset, an asset position, an asset size, and timeline data associated with the asset.
17 . The non-transitory computer-readable medium of claim 16 , wherein identifying attributes of the assets in the current state of the video comprising using a computer vision model to classify a visual asset in the video with one or more attributes.