Implementation of machine learning state machines
Implementations are disclosed for automated design and implementation of machine learning (ML) state machines that include at least some aspect of machine learning. In various implementations, unstructured input may be received from a user. The unstructured input may convey operational aspect(s) of a machine learning (ML) state machine desired by the user. The unstructured input may be semantically processed to determine an intent of the user. The intent may include the operational aspect(s) of the ML state machine desired by the user. Based on the intent of the user, a plurality of modular logical routines may be selected from an existing library of modular logical routines. At least one logical routine of the selected plurality of logical routines may include logical operations that process data using one or more machine learning models. The selected plurality of logical routines may be assembled into the desired state ML state machine.
1 . A method implemented using one or more processors, the method comprising:
receiving unstructured input from a user, wherein the unstructured input conveys one or more operational aspects of a machine learning (ML) state machine desired by the user;
generating a semantic embedding to determine an intent of the user, wherein the intent includes the one or more operational aspects of the ML state machine desired by the user, wherein the semantic embedding represents the one or more operational aspects of the ML state machine desired by the user in a first embedding space;
determining a similarity between the semantic embedding and a reference embedding, wherein the reference embedding represents an output of a modular logical routine in a second embedding space;
based on the similarity between the semantic embedding and the reference embedding, selecting, from an existing library of modular logical routines, a plurality of modular logical routines to be assembled into the desired ML state machine including the modular logical routine of the reference embedding, wherein at least one logical routine of the selected plurality of logical routines includes logical operations that process data using one or more machine learning models;
assembling the selected plurality of logical routines into the desired state ML state machine;
rendering, on a working canvas of a graphical user interface (GUI) provided as part of an integrated development environment, an editable visual representation of the desired ML state machine; and
training the one or more machine learning models based on user feedback received via the GUI, wherein the user feedback causes the desired ML state machine to train the one or more machine learning models in response to manipulation of graphical elements representing at least one of an edge or a node in relation to each other.
2 . The method of claim 1 , wherein the semantic processing includes generating the semantic embedding based on the unstructured input.
3 . The method of claim 2 , wherein the selecting is based on a comparison between the semantic embedding generated based on the unstructured input and a plurality of reference embeddings generated based on the existing library of logical routines.
4 . The method of claim 1 , wherein the unstructured input includes one or more free-form natural language inputs provided by the user in response to one or more prompts.
5 . The method of claim 1 , wherein the unstructured input includes one or more digital images.
6 . The method of claim 5 , wherein the one or more digital images are annotated to identify one or more instances of an object class, and the at least one logical routine is selected based on one or more of the machine learning models having been trained to detect instances of objects in the object class.
7 . The method of claim 1 , wherein the visual representation of the desired ML state machine includes a graph with nodes representing the selected plurality of logical routines and edges representing logical couplings between the selected plurality of logical routines.
8 . The method of claim 1 , wherein the ML state machine includes an agricultural state machine, and the one or more machine learning models include a phenotyping agricultural machine learning model.
9 . A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions, cause the one or more processors to:
receive unstructured input from a user, wherein the unstructured input conveys one or more operational aspects of a machine learning (ML) state machine desired by the user;
generate a semantic embedding to determine an intent of the user, wherein the intent includes the one or more operational aspects of the ML state machine desired by the user, wherein the semantic embedding represents the one or more operational aspects of the ML state machine desired by the user in a first embedding space;
determine a similarity between the semantic embedding and a reference embedding, wherein the reference embedding represents an output of a modular logical routine in a second embedding space;
based on the similarity between the semantic embedding and the reference embedding, select, from an existing library of modular logical routines, a plurality of modular logical routines to be assembled into the desired ML state machine including the modular logical routine of the reference embedding, wherein at least one logical routine of the selected plurality of logical routines includes logical operations that process data using one or more machine learning models;
assemble the selected plurality of logical routines into the desired state ML state machine;
render, on a working canvas of a graphical user interface (GUI) provided as part of an integrated development environment, an editable visual representation of the desired ML state machine; and
train the one or more machine learning models based on user feedback received via the GUI, wherein the user feedback causes the desired ML state machine to train the one or more machine learning models in response to manipulation of graphical elements representing at least one of an edge or a node in relation to each other.
10 . The system of claim 9 , including instructions to generate the semantic embedding based on the unstructured input.
11 . The system of claim 10 , wherein the selecting is based on a comparison between the semantic embedding generated based on the unstructured input and a plurality of reference embeddings generated based on the existing library of logical routines.
12 . The system of claim 9 , wherein the unstructured input includes one or more free-form natural language inputs provided by the user in response to one or more prompts.
13 . The system of claim 9 , wherein the unstructured input includes one or more digital images.
14 . The system of claim 13 , wherein the one or more digital images are annotated to identify one or more instances of an object class, and the at least one logical routine is selected based on one or more of the machine learning models having been trained to detect instances of objects in the object class.
15 . The system of claim 9 , wherein the visual representation of the desired ML state machine includes a graph with nodes representing the selected plurality of logical routines and edges representing logical couplings between the selected plurality of logical routines.
16 . The system of claim 9 , wherein the ML state machine includes an agricultural state machine, and the one or more machine learning models include a phenotyping agricultural machine learning model.
17 . A non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a processor, cause the processor to:
receive unstructured input from a user, wherein the unstructured input conveys one or more operational aspects of a machine learning (ML) state machine desired by the user;
generate a semantic embedding to determine an intent of the user, wherein the intent includes the one or more operational aspects of the ML state machine desired by the user, wherein the semantic embedding represents the one or more operational aspects of the ML state machine desired by the user in a first embedding space;
determine a similarity between the semantic embedding and a reference embedding, wherein the reference embedding represents an output of a modular logical routine in a second embedding space;
based on the similarity between the semantic embedding and the reference embedding, select, from an existing library of modular logical routines, a plurality of modular logical routines to be assembled into the desired ML state machine including the modular logical routine of the reference embedding, wherein at least one logical routine of the selected plurality of logical routines includes logical operations that process data using one or more machine learning models;
assemble the selected plurality of logical routines into the desired state ML state machine;
render, on a working canvas of a graphical user interface (GUI) provided as part of an integrated development environment, an editable visual representation of the desired ML state machine; and
train the one or more machine learning models based on user feedback received via the GUI, wherein the user feedback causes the desired ML state machine to train the one or more machine learning models in response to manipulation of graphical elements representing at least one of an edge or a node in relation to each other.
18 . The non-transitory computer-readable medium of claim 17 , including instructions to generate the semantic embedding based on the unstructured input.