Context-based prompt generation

1 . A method comprising:

receiving, by a processing device, a selection of a UI element of first digital content;

capturing, by the processing device, contextual information associated with the selection;

in response to receiving the selection, generating, using a first machine-learning model, a prompt based on the selection and the contextual information, the selection triggering the generating of the prompt before receiving a subsequent user input;

generating, using generative artificial intelligence (AI) implemented using one or more second machine-learning models based on the prompt, second digital content; and

presenting, by the processing device, the second digital content for display in a user interface.

2 . The method of claim 1 , wherein:

the selection of the UI element comprises highlighting, selecting, or marking of text, images, charts, forms, or other UI elements of the first digital content; and

the second digital content comprises text, images, or audio content aligned with a context of the selection.

3 . The method of claim 1 , wherein capturing the contextual information comprises analyzing a document object model (DOM) structure associated with the selection and the first digital content.

4 . The method of claim 3 , wherein capturing the contextual information further comprises collecting metadata associated with the selection.

5 . The method of claim 1 , wherein generating the prompt comprises:

extracting one or more features from the selection or the contextual information;

applying one or more natural language processing techniques to analyze the one or more features;

determining a context and an intent of the selection based on the one or more features;

formulating a structured query based on the context and the intent; and

outputting the prompt based on the structured query, wherein the prompt includes parameters for guiding the generative AI in producing the second digital content.

6 . The method of claim 5 , wherein the first machine-learning model is trained using a dataset comprising examples of digital content that demonstrated user-approved performance metrics.

7 . The method of claim 5 , wherein generating the prompt further comprises:

capturing a screenshot of the selection of the UI element, the screenshot being included in the prompt to the generative AI.

8 . The method of claim 7 , wherein the selection of the UI element, the screenshot, the contextual information associated with the selection, and the second digital content are transmitted to a chat interface, a computer system remote from the processing device, a remote third party, or a memory associated with the processing device as a storage file.

9 . The method of claim 1 , further comprising:

receiving the subsequent user input related to the second digital content;

generating an updated prompt based on the subsequent user input and the prompt; providing the updated prompt to the generative AI; and

presenting updated second digital content for display in the user interface.

10 . The method of claim 1 , wherein receiving the selection of the UI element comprises:

using, a weighted scoring algorithm, to evaluate potential boundaries of the UI element, the weighted scoring algorithm considering at least two of semantic HTML5 tags, content patterns, spatial proximity of other UI elements, or styling patterns of the other UI elements.

11 . The method of claim 1 , wherein the second digital content includes at least one of:

explanatory information related to the UI element;

suggested actions or next steps based on the UI element;

relevant data or statistics associated with the UI element;

visual representations or charts derived from information in the UI element; or

interactive elements allowing further exploration of content related to the UI element.

12 . A system comprising:

memory; and

a hardware processing device coupled to the memory, the hardware processing device configured to perform operations comprising:

detecting activation of a selection mode;

receiving a user interface (UI) selection of a UI element while in the selection mode;

capturing contextual information associated with the UI selection;

in response to receiving the UI selection, generating, using a machine-learning model, a prompt based on the UI selection and the contextual information, the UI selection triggering the generating of the prompt before receiving a subsequent user input;

providing the prompt to a generative machine-learning model; and

outputting generative digital content produced by the generative machine-learning model in response to the prompt.

13 . The system of claim 12 , wherein the hardware processing device performs additional operations comprising:

providing, while in the selection mode, visual feedback indicating the UI element is selectable; and

in response to providing the visual feedback that the UI element is selectable and before receiving the UI selection, generating a preliminary prompt using the machine-learning model and generating preliminary generative digital content by the generative machine-learning model based on the preliminary prompt.

14 . The system of claim 12 , wherein:

the UI element and the generative machine-learning model are provided as part of a single application; or

the UI element is displayed as part of a first application and the selection mode and the generative machine-learning model are provided as part of a second application.

15 . The system of claim 12 , wherein capturing the contextual information comprises:

analyzing a document object model (DOM) structure associated with the UI selection;

extracting hierarchical relationships between UI elements from the DOM structure;

identifying attributes and properties of the selected UI element and its parent elements; and

determining a position and a layout of the selected UI element relative to other elements on a page.

16 . The system of claim 15 , wherein capturing the contextual information further comprises collecting metadata associated with the UI selection and user information, the user information including at least one of a role or a position of a user, permissions associated with the user, or user history.

17 . One or more non-transitory computer-readable storage media storing instructions that, when executed by a processor, cause the processor to:

receive a user interface (UI) selection of a UI element;

capture contextual information associated with the UI selection;

in response to receiving the UI selection, generate, using a machine-learning model, a prompt based on the UI selection and the captured contextual information, the UI selection triggering generation of the prompt before receiving a subsequent user input;

provide the generated prompt to a generative machine-learning model; and

output generative digital content produced by the generative machine-learning model in response to the generated prompt.

18 . The one or more non-transitory computer-readable storage media of claim 17 , wherein the contextual information includes at least one of:

user interaction history with the UI element;

temporal data associated with the UI selection;

device characteristics of a computing device used for the UI selection; or

environmental data collected at a time of the UI selection.

19 . The one or more non-transitory computer-readable storage media of claim 17 storing additional instructions that, when executed by the processor, cause the processor to:

analyze user feedback on the generative digital content;

adjust the machine-learning model based on the user feedback; and

store the adjusted machine-learning model for use in subsequent prompt generation.

20 . The one or more non-transitory computer-readable storage media of claim 17 , wherein the instructions, when executed by the processor, cause the processor, in generating the generative digital content, to:

identify a domain associated with the UI selection and the contextual information;

select a domain-specific generative AI model based on the identified domain; and

utilize the selected domain-specific generative AI model to produce the generative digital content.