Fine-tuning images generated by artificial intelligence based on aesthetic and accuracy metrics and systems and methods for the same

1 . At least one non-transitory, computer-readable storage medium storing instructions, which, when executed by at least one data processor of a system, cause the system to:

receive, from a first user device, a first image generation prompt comprising a first text string describing a target image;

receive, from a second user device, a second image generation prompt comprising a second text string describing the target image;

determine a first subscription status associated with a first user of the first user device;

using the determined first subscription status, compute a first weight for the first image generation prompt;

determine a second subscription status associated with the second user device;

using the determined second subscription status, compute a second weight for the second image generation prompt;

in response to the first weight for the first image generation prompt being greater than or equal to the second weight for the second image generation prompt, generate a second image approximating the target image by applying an image generation model to the first image generation prompt in lieu of applying the image generation model to the second image generation prompt;

in response to the second weight for the second image generation prompt being greater than the first weight for the first image generation prompt, generate a second image approximating the target image by applying an image generation model to the second image generation prompt in lieu of applying the image generation model to the first image generation prompt;

compute, for the second image, an accuracy metric,

wherein the accuracy metric indicates a degree of correspondence between the target image and the second image;

compute, for the second image, an aesthetic metric,

wherein the aesthetic metric indicates a degree of aesthetic value for the second image; and

when: (1) the computed accuracy metric is less than or equal to a threshold accuracy metric, or (2) the computed aesthetic metric is less than or equal to a threshold aesthetic metric:

generate a seed artifact indicative of a next action to be performed in response to receiving the first image generation prompt;

based on the generated seed artifact, generate a third image approximating the target image by applying the image generation model to the generated second image; and

transmit the generated third image to the first user device to enable display of the generated third image at a user interface of the first user device.

2 . The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for computing, for the second image, the accuracy metric cause the system to:

determine a token within the first image generation prompt,

wherein the token comprises a set of alphanumeric characters representing an object in the target image;

compute a probability that the second image includes a depiction of the object associated with the token by applying an image recognition model to the second image; and

compute the accuracy metric based on the probability.

3 . The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for computing, for the second image, the accuracy metric cause the system to:

generate a label for the second image by applying an image labeling model to the second image,

wherein the label comprises an alphanumeric string indicating a description of the second image; and

compute the accuracy metric based on comparing the label for the second image with the first image generation prompt.

4 . The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for computing, for the second image, the aesthetic metric cause the system to generate the aesthetic metric based on applying an aesthetic evaluation model to the second image,

wherein the aesthetic evaluation model is trained to output aesthetic metrics based on at least one of: depth of field, lighting, position, focus, contrast, color, and brightness.

5 . The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for computing, for the second image, the aesthetic metric cause the system to:

extract descriptive tokens from the first image generation prompt, wherein the descriptive tokens indicate aesthetic qualities of the target image;

generate indicators of one or more aesthetic properties for the second image by applying an aesthetic recognition model to the second image,

wherein the aesthetic recognition model is trained to output text describing aesthetic properties for input images; and

compute the aesthetic metric based on comparing one or more of the descriptive tokens with the indicators of the one or more aesthetic properties.

6 . The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the instructions cause the system to:

compute a quality metric based on the computed accuracy metric and the computed aesthetic metric; and

generate the seed artifact based on determining that the quality metric is less than a threshold quality metric.

7 . The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for generating the seed artifact cause the system to:

generate a control token based on the computed accuracy metric or the computed aesthetic metric,

wherein the control token indicates a property of the target image missing in the second image;

update the first image generation prompt to include the control token; and

store the updated first image generation prompt as the generated seed artifact.

8 . The at least one non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for generating the seed artifact cause the system to generate the seed artifact based on the second image,

wherein the generated seed artifact comprises a vector representation of the second image.

9 . A system comprising:

at least one hardware processor; and

at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to:

receive, from a first user device, a first image generation prompt comprising a first text string describing a target image;

receive, from a second user device, a second image generation prompt comprising a second text string describing the target image;

determine a first subscription status associated with a first user of the first user device;

using the determined first subscription status, compute a first weight for the first image generation prompt;

determine a second subscription status associated with a second user of the second user device;

using the determined second subscription status, compute a second weight for the second image generation prompt;

in response to the first weight for the first image generation prompt being greater than or equal to the second weight for the second image generation prompt, generate a second image approximating the target image by applying an image generation model to the first image generation prompt in lieu of applying the image generation model to the second image generation prompt;

in response to the second weight for the second image generation prompt being greater than the first weight for the first image generation prompt, generate a second image approximating the target image by applying an image generation model to the second image generation prompt in lieu of applying the image generation model to the first image generation prompt;

compute, for the second image, an accuracy metric,

wherein the accuracy metric indicates a degree of correspondence between the target image and the second image;

compute, for the second image, an aesthetic metric,

wherein the aesthetic metric indicates a degree of aesthetic value for the second image;

compute a quality metric for the second image based on the accuracy metric and the aesthetic metric; and

when the quality metric is less than a threshold metric:

generate a seed artifact indicative of a next action to be performed in response to receiving the first image generation prompt;

based on the generated seed artifact, generate a third image approximating the target image by applying the image generation model to the generated second image; and

transmit the generated third image to the first user device to enable display of the generated third image at a user interface of the first user device.

10 . The system of claim 9 , wherein the instructions for computing, for the second image, the accuracy metric cause the system to:

determine a token within the first image generation prompt,

wherein the token comprises a set of alphanumeric characters representing an object in the target image;

compute a probability that the second image includes a depiction of the object associated with the token by applying an image recognition model to the second image; and

compute the accuracy metric based on the probability.

11 . The system of claim 9 , wherein the instructions for computing, for the second image, the accuracy metric cause the system to:

generate a label for the second image by applying an image labeling model to the second image,

wherein the label comprises an alphanumeric string indicating a description of the second image; and

compute the accuracy metric based on comparing the label for the second image with the first image generation prompt.

12 . The system of claim 9 , wherein the instructions for computing, for the second image, the aesthetic metric cause the system to generate the aesthetic metric based on applying an aesthetic evaluation model to the second image,

wherein the aesthetic evaluation model is trained to output aesthetic metrics based on at least one of: depth of field, lighting, position, focus, contrast, color, and brightness.

13 . The system of claim 9 , wherein the instructions for computing, for the second image, the aesthetic metric cause the system to:

extract descriptive tokens from the first image generation prompt, wherein the descriptive tokens indicate aesthetic qualities of the target image;

generate indicators of one or more aesthetic properties for the second image by applying an aesthetic recognition model to the second image,

wherein the aesthetic recognition model is trained to output text describing aesthetic properties for input images; and

compute the aesthetic metric based on comparing one or more of the descriptive tokens with the indicators of the one or more aesthetic properties.

14 . The system of claim 9 , wherein the instructions for generating the seed artifact cause the system to:

generate a control token based on the computed accuracy metric or the computed aesthetic metric,

wherein the control token indicates a property of the target image missing in the second image;

update the first image generation prompt to include the control token; and

store the updated first image generation prompt as the generated seed artifact.

15 . The system of claim 9 , wherein the instructions for generating the seed artifact cause the system to generate the seed artifact based on the second image,

wherein the generated seed artifact comprises a vector representation of the second image.

16 . A method comprising:

receiving, from a first user device, a first image generation prompt comprising a first text string describing a target image;

receiving, from a second user device, a second image generation prompt comprising a second text string describing the target image;

determining a first subscription status associated with a first user of the first user device;

using the determined first subscription status, computing a first weight for the first image generation prompt;

determining a second subscription status associated with a second user of the second user device;

using the determined second subscription status, computing a second weight for the second image generation prompt;

in response to the first weight for the first image generation prompt being greater than or equal to the second weight for the second image generation prompt, generating a second image approximating the target image by applying an image generation model to the first image generation prompt in lieu of applying the image generation model to the second image generation prompt;

in response to the second weight for the second image generation prompt being greater than the first weight for the first image generation prompt, generating a second image approximating the target image by applying an image generation model to the second image generation prompt in lieu of applying the image generation model to the first image generation prompt;

computing, for the second image, an accuracy metric,

wherein the accuracy metric indicates a degree of correspondence between the target image and the second image;

computing, for the second image, an aesthetic metric,

wherein the aesthetic metric indicates a degree of aesthetic value for the second image; and

when: (1) the computed accuracy metric is less than or equal to a threshold accuracy metric, or (2) the computed aesthetic metric is less than or equal to a threshold aesthetic metric:

generating a seed artifact indicative of a next action to be performed in response to receiving the first image generation prompt;

based on the generated seed artifact, generating a third image approximating the target image by applying the image generation model to the generated second image; and

transmitting the generated third image to the first user device to enable display of the generated third image at a user interface of the first user device.

17 . The method of claim 16 , wherein computing, for the second image, the accuracy metric comprises:

determining a token within the first image generation prompt,

wherein the token comprises a set of alphanumeric characters representing an object in the target image;

computing a probability that the second image includes a depiction of the object associated with the token by applying an image recognition model to the second image; and

computing the accuracy metric based on the probability.

18 . The method of claim 16 , wherein computing, for the second image, the accuracy metric comprises:

generating a label for the second image by applying an image labeling model to the second image,

wherein the label comprises an alphanumeric string indicating a description of the second image; and

computing the accuracy metric based on comparing the label for the second image with the first image generation prompt.

19 . The method of claim 16 , wherein computing, for the second image, the aesthetic metric comprises generating the aesthetic metric based on applying an aesthetic evaluation model to the second image,

wherein the aesthetic evaluation model is trained to output aesthetic metrics based on at least one of: depth of field, lighting, position, focus, contrast, color, and brightness.

20 . The method of claim 16 , wherein computing, for the second image, the aesthetic metric comprises:

extracting descriptive tokens from the first image generation prompt,

wherein the descriptive tokens indicate aesthetic qualities of the target image;

generating indicators of one or more aesthetic properties for the second image by applying an aesthetic recognition model to the second image,

wherein the aesthetic recognition model is trained to output text describing aesthetic properties for input images; and

computing the aesthetic metric based on comparing one or more of the descriptive tokens with the indicators of the one or more aesthetic properties.