Constraint based inference and machine learning system
A method including receiving a pre-determined constraint on user actions. A constraint vector is generated based on the pre-determined constraint. The constraint vector is input into a machine learning model. A first output is generated from the machine learning model by executing the machine learning model using the constraint vector as a first input to the machine learning model. The constraint vector is converted into a legal action mask. A probability vector is generated by executing a masked softmax operator. The masked softmax operator takes, as a second input, the first output. The masked softmax operator takes, as a third input, the legal action mask. The masked softmax operator generates, as a second output, the probabilities vector. Action outputs are generated by applying a sampling system to the probability vector. The action outputs include a subset of the user actions, and wherein the subset includes only allowed user actions.
1. A method comprising:
receiving a pre-determined constraint on a plurality of user actions;
generating a constraint vector based on the pre-determined constraint;
inputting the constraint vector into a machine learning model;
generating a first output from the machine learning model by executing the machine learning model using the constraint vector as a first input to the machine learning model;
converting the constraint vector into a legal action mask;
generating a probability vector by executing a masked softmax operator, wherein:
the masked softmax operator takes, as a second input, the first output,
the masked softmax operator takes, as a third input, the legal action mask, and
the masked softmax operator generates, as a second output, the probabilities vector; and
generating a plurality of action outputs by applying a sampling system to the probability vector, wherein the plurality of action outputs comprise a subset of the plurality user actions, and wherein the subset includes only allowed user actions.
2. The method of claim 1 , wherein generating the constraint vector comprises:
converting the pre-determined constraint into a first machine learning vector;
receiving, from a user interface, a user-designated constraint to the plurality of user actions, the user-designated constraint received from a user;
converting the user-designated constraint into a second machine learning vector; and
combining the first machine learning vector and the second machine learning vector into the constraint vector.
3. The method of claim 2 , wherein combining comprises concatenating the first machine learning vector and the second machine learning vector.
4. The method of claim 1 , further comprising:
displaying, on a display device, both the subset of the plurality of user actions and the allowed user actions; and
highlighting the subset of the plurality of user actions.
5. The method of claim 1 , further comprising:
disabling a control of a machine, wherein use of the control is not one of the allowed user actions.
6. The method of claim 2 , wherein the pre-determined constraint comprises at least one of sensor data received contemporaneously with receiving the user designated constraint, sensor data received prior to receiving the user designated constraint, a constraint received over a communication device, and a pre-mission constraint related to a mission assigned to the user.
7. The method of claim 1 , wherein:
the legal action mask comprises a third vector having features comprising zeros or ones,
a corresponding constrained action is set to zero, and
a corresponding allowed action is set to one.
8. The method of claim 1 , further comprising:
transmitting the subset of the plurality of user actions to a computer operated by a second user; and
displaying constraints to a second plurality of user actions related to the second user, based on the subset of the plurality of user actions.
9. The method of claim 1 , wherein:
the pre-determined constraint comprises mission constraints that constrain the plurality of user actions based on forbidden actions specified by a second user.
10. The method of claim 1 , wherein:
the pre-determined constraint comprises system constraints that constrain the plurality of user actions based on sensed physical parameters of a vehicle operated at least in part by the user.
11. The method of claim 10 , wherein:
the sensed physical parameters are received from at least one sensor operably connected to the vehicle.
12. The method of claim 1 , wherein:
adding an updated constraint, received after receiving the pre-determined constraint, to the pre-determined constraint.
13. The method of claim 1 , wherein:
the pre-determined constraint comprises mission constraints that constrain the plurality of user actions based on forbidden actions specified by a second user;
the pre-determined constraint further comprises system constraints that constrain the plurality of user actions based on sensed physical parameters of a vehicle operated at least in part by the user;
the sensed physical parameters are received from at least one sensor operably connected to the vehicle; and
the pre-determined constraint further comprises an updated constraint received after initially receiving the pre-determined constraint.
14. A method comprising:
receiving a pre-determined constraint on a plurality of user actions;
converting the pre-determined constraint into a first machine learning vector;
receiving, from a user interface, a user-designated constraint to the plurality of user actions, the user-designated constraint received from a user;
converting the user-designated constraint into a second machine learning vector;
generating a constraint vector by concatenating the first machine learning vector and the second machine learning vector;
inputting the constraint vector into a first machine learning model;
generating a first output from the first machine learning model by executing the first machine learning model using the constraint vector as a first input to first the machine learning model;
converting the constraint vector into a legal action mask;
generating a first probability vector by executing a masked softmax operator, wherein:
the masked softmax operator takes, as a second input, the first output,
the masked softmax operator takes, as a third input, the legal action mask, and
the masked softmax operator generates, as a second output, the first probabilities vector;
generating a plurality of action outputs by applying a sampling system to the first probabilities vector, wherein the plurality of action outputs comprise a subset of the plurality of user actions, and wherein the subset includes only allowed user actions;
transmitting the constraint vector to a second vehicle operated by a second user;
calculating a second probabilities vector using a second machine learning model, wherein the second machine learning model uses at least the constraint vector as input; and
generating a plurality of second action outputs by applying a second sampling system to the second probabilities vector, wherein the plurality of second action outputs comprise a second subset of a plurality of second user actions of a second user operating the second vehicle, and wherein the second subset includes only second allowed user actions of the second user.
15. The method of claim 14 , further comprising:
disabling at least a portion of a sub-system of the second vehicle based on the second subset.
16. The method of claim 14 , wherein:
calculating the second probabilities vector further comprises using a second pre-determined constraint as input to the second machine learning model; and
the second pre-determined constraint comprises mission constraints that constrain the plurality of second user actions based on forbidden actions specified by a third user.
17. The method of claim 14 , wherein:
calculating the second probabilities vector further comprises using a second pre-determined constraint as input to the second machine learning model;
the second pre-determined constraint comprises system constraints that constrain the plurality of second user actions based on second sensed physical parameters of the second vehicle; and
the second sensed physical parameters are received from second sensors operably connected to the second vehicle.
18. The method of claim 14 , wherein:
calculating the second probabilities vector further comprises using a second pre-determined constraint as input to the second machine learning model; and
the second pre-determined constraint comprises a second updated constraint received by the second vehicle after receiving the second pre-determined constraint.
19. The method of claim 14 , wherein the pre-determined constraint comprises at least one of sensor data received contemporaneously with receiving a user designated constraint, sensor data received prior to receiving the user designated constraint, a constraint received over a communication device, and a pre-mission constraint related to a mission assigned to the user.
20. The method of claim 14 , wherein:
the legal action mask comprises a third vector having features comprising zeros or ones,
a corresponding constrained action is set to zero, and
a corresponding allowed action is set to one.