Techniques for model-based element creation in effect creation tools
Described are examples for creating elements in an effect creation tool, including receiving, via a user interface provided for the effect creation tool, a natural language string requesting creation of an element in the effect creation tool, providing, to a model, an input including at least a portion of the natural language string, receiving, from the model and based on the input, an output string, in an expected syntax, corresponding to creating the element, mapping the output string to one or more commands of a format for creating the element in the effect creation tool, and providing the one or more commands to the effect creation tool to cause creation of the element in the effect creation tool.
1 . A computer-implemented method for creating elements in an effect creation tool, comprising:
receiving, via a user interface provided for the effect creation tool, a natural language string requesting creation of an element in the effect creation tool;
providing, to a model, an input including at least a portion of the natural language string;
receiving, from the model and based on the input, an output string, in an expected syntax, corresponding to creating the element;
mapping the output string to one or more commands of a format supported by the effect creation tool for creating, by the effect creation tool, the element as a selectable element that is selectable, via the effect creation tool, for applying to a stored or real-time video content; and
providing the one or more commands to the effect creation tool to cause creation of the element in the effect creation tool.
2 . The computer-implemented method of claim 1 , further comprising providing, to the model, an indication of the expected syntax of the output string.
3 . The computer-implemented method of claim 2 , wherein the indication of the expected syntax includes multiple parameter names, and for each of the multiple parameter names, multiple possible parameter values, wherein the output string includes at least one of the multiple parameter names, and for each of the at least one of the multiple parameter names, at least one of the multiple possible parameter values.
4 . The computer-implemented method of claim 3 , wherein the multiple parameter names correspond to elements that can be created in the effect creation tool.
5 . The computer-implemented method of claim 1 , wherein providing the one or more commands to the effect creation tool includes calling an application programming interface (API) function provided by the effect creation tool to cause creation of the element.
6 . The computer-implemented method of claim 1 , wherein the element includes one or more of a texture, material, visual effects, or gameplay element in the effect creation tool.
7 . An apparatus for creating elements in an effect creation tool, the apparatus comprising one or more processors and one or more non-transitory memories with instructions thereon, wherein the instructions upon execution by the one or more processors, cause the one or more processors to:
receive, via a user interface provided for the effect creation tool, a natural language string requesting creation of an element in the effect creation tool;
provide, to a model, an input including at least a portion of the natural language string;
receive, from the model and based on the input, an output string, in an expected syntax, corresponding to creating the element;
map the output string to one or more commands of a format supported by the effect creation tool for creating, by the effect creation tool, the element as a selectable element that is selectable, via the effect creation tool, for applying to a stored or real-time video content; and
provide the one or more commands to the effect creation tool to cause creation of the element in the effect creation tool.
8 . The apparatus of claim 7 , wherein the instructions upon execution by the one or more processors, cause the one or more processors to provide, to the model, an indication of the expected syntax of the output string.
9 . The apparatus of claim 8 , wherein the indication of the expected syntax includes multiple parameter names, and for each of the multiple parameter names, multiple possible parameter values, wherein the output string includes at least one of the multiple parameter names, and for each of the at least one of the multiple parameter names, at least one of the multiple possible parameter values.
10 . The apparatus of claim 9 , wherein the multiple parameter names correspond to elements that can be created in the effect creation tool.
11 . The apparatus of claim 7 , wherein the instructions upon execution by the one or more processors, cause the one or more processors to provide the one or more commands to the effect creation tool at least in part by calling an application programming interface (API) function provided by the effect creation tool to cause creation of the element.
12 . The apparatus of claim 7 , wherein the element includes one or more of a texture, material, visual effects, or gameplay element in the effect creation tool.
13 . One or more non-transitory computer-readable storage media storing instructions that when executed by one or more processors cause the one or more processors to execute a method for creating elements in an effect creation tool, wherein the method comprises:
receiving, via a user interface provided for the effect creation tool, a natural language string requesting creation of an element in the effect creation tool;
providing, to a model, an input including at least a portion of the natural language string;
receiving, from the model and based on the input, an output string, in an expected syntax, corresponding to creating the element;
mapping the output string to one or more commands of a format supported by the effect creation tool for creating, by the effect creation tool, the element as a selectable element that is selectable, via the effect creation tool, for applying to a stored or real-time video content; and
providing the one or more commands to the effect creation tool to cause creation of the element in the effect creation tool.
14 . The one or more non-transitory computer-readable storage media of claim 13 , the method further comprising providing, to the model, an indication of the expected syntax of the output string.
15 . The one or more non-transitory computer-readable storage media of claim 14 , wherein the indication of the expected syntax includes multiple parameter names, and for each of the multiple parameter names, multiple possible parameter values, wherein the output string includes at least one of the multiple parameter names, and for each of the at least one of the multiple parameter names, at least one of the multiple possible parameter values.
16 . The one or more non-transitory computer-readable storage media of claim 15 , wherein the multiple parameter names correspond to elements that can be created in the effect creation tool.
17 . The one or more non-transitory computer-readable storage media of claim 13 , wherein providing the one or more commands to the effect creation tool includes calling an application programming interface (API) function provided by the effect creation tool to cause creation of the element.