Techniques for model-based image operation in effect creation tools
Described are examples for image operations in an effect creation tool, including receiving, via a user interface provided for the effect creation tool, a natural language string requesting an operation related to obtaining or modifying an image, providing, to a model, an input including at least a portion of the natural language string, receiving, from the model, an image result output based on the input, and importing the image result as an asset in the effect creation tool.
1 . A computer-implemented method for image operations in an effect creation tool, comprising:
receiving, via a user interface provided for the effect creation tool, a natural language string requesting an operation related to obtaining or modifying an image;
providing, to multiple models, an input including at least a portion of the natural language string;
receiving, from the multiple models, multiple image results output based on the input;
displaying an indication of the multiple models, and, based on selection of one of the multiple models, a portion of the multiple image results corresponding to the one of the multiple models; and
importing an image result of the multiple image results as an asset in the effect creation tool.
2 . The computer-implemented method of claim 1 , further comprising generating the input to include the natural language string and a context parameter associated with the operation.
3 . The computer-implemented method of claim 1 , wherein the operation includes obtaining the image, wherein the image result includes a display image, and wherein importing the image result includes creating the asset for the display image in the effect creation tool.
4 . The computer-implemented method of claim 1 , wherein the operation includes obtaining the image as a variation of an original image, wherein the image result includes a display image, and wherein importing the image result includes creating the asset for the display image in the effect creation tool.
5 . The computer-implemented method of claim 1 , further comprising prompting, via the user interface, for a location on the image of a modification to the image, wherein the operation includes the modification to the image, and wherein importing the image result includes applying the image result as the modification to the image.
6 . The computer-implemented method of claim 1 , wherein the operation includes applying the image to an object as a mask, wherein the natural language string indicates the image to be applied as the mask, and wherein importing the image result includes applying the image result as the mask over the image.
7 . The computer-implemented method of claim 1 , wherein importing the image result as the asset in the effect creation tool is based on selection, via the user interface, of the image result from the multiple image results.
8 . An apparatus for image operations in an effect creation tool, the apparatus comprising one or more processors and one or more non-transitory memories with instructions thereon, wherein the instructions upon execution by the one or more processors, cause the one or more processors to:
receive, via a user interface provided for the effect creation tool, a natural language string requesting an operation related to obtaining or modifying an image;
provide, to multiple models, an input including at least a portion of the natural language string;
receive, from the multiple models, multiple image results output based on the input;
display an indication of the multiple models, and, based on selection of one of the multiple models, a portion of the multiple image results corresponding to the one of the multiple models; and
import an image result of the multiple image results as an asset in the effect creation tool.
9 . The apparatus of claim 8 , wherein the instructions upon execution by the one or more processors, cause the one or more processors to generate the input to include the natural language string and a context parameter associated with the operation.
10 . The apparatus of claim 8 , wherein the operation includes obtaining the image, wherein the image result includes a display image, and wherein the instructions upon execution by the one or more processors, cause the one or more processors to import the image result including creating the asset for the display image in the effect creation tool.
11 . The apparatus of claim 8 , wherein the operation includes obtaining the image as a variation of an original image, wherein the image result includes a display image, and wherein the instructions upon execution by the one or more processors, cause the one or more processors to import the image result including creating the asset for the display image in the effect creation tool.
12 . The apparatus of claim 8 , wherein the instructions upon execution by the one or more processors, cause the one or more processors to prompt, via the user interface, for a location on the image of a modification to the image, wherein the operation includes the modification to the image, and wherein the instructions upon execution by the one or more processors, cause the one or more processors to import the image result including applying the image result as the modification to the image.
13 . The apparatus of claim 8 , wherein the operation includes applying the image to an object as a mask, wherein the natural language string indicates the image to be applied as the mask, and wherein the instructions upon execution by the one or more processors, cause the one or more processors to import the image result including applying the image result as the mask over the image.
14 . The apparatus of claim 8 , wherein the instructions upon execution by the one or more processors, cause the one or more processors to import the image result as the asset in the effect creation tool is based on selection, via the user interface, of the image result from the multiple image results.
15 . One or more non-transitory computer-readable storage media storing instructions that when executed by one or more processors cause the one or more processors to execute a method for image operations in an effect creation tool, wherein the method comprises:
receiving, via a user interface provided for the effect creation tool, a natural language string requesting an operation related to obtaining or modifying an image;
providing, to multiple models, an input including at least a portion of the natural language string;
receiving, from the multiple models, multiple image results output based on the input;
displaying an indication of the multiple models, and, based on selection of one of the multiple models, a portion of the multiple image results corresponding to the one of the multiple models; and
importing an image result of the multiple image results as an asset in the effect creation tool.
16 . The one or more non-transitory computer-readable storage media of claim 15 , the method further comprising generating the input to include the natural language string and a context parameter associated with the operation.
17 . The one or more non-transitory computer-readable storage media of claim 15 , wherein the operation includes obtaining the image, wherein the image result includes a display image, and wherein importing the image result includes creating the asset for the display image in the effect creation tool.
18 . The one or more non-transitory computer-readable storage media of claim 15 , wherein the operation includes obtaining the image as a variation of an original image, wherein the image result includes a display image, and wherein importing the image result includes creating the asset for the display image in the effect creation tool.
19 . The one or more non-transitory computer-readable storage media of claim 15 , further comprising prompting, via the user interface, for a location on the image of a modification to the image, wherein the operation includes the modification to the image, and wherein importing the image result includes applying the image result as the modification to the image.
20 . The one or more non-transitory computer-readable storage media of claim 15 , wherein the operation includes applying the image to an object as a mask, wherein the natural language string indicates the image to be applied as the mask, and wherein importing the image result includes applying the image result as the mask over the image.