Method of monitoring a state of a machine learning classifier

1 . A method of determining a state of a machine learning classifier configured to determine an operational mode of an aircraft system, the machine learning classifier having an input layer comprising input nodes, an output layer comprising output nodes, at least one hidden layer between the input layer and the output layer, the at least one hidden layer comprising hidden layer nodes, and the machine learning classifier configured to provide an output for determining the operational mode, the method comprising:

providing a model of the aircraft system, wherein the model comprises representations of: a) components configured to operate as part of the aircraft system, and b) connections between the components of the aircraft system, wherein each component in the aircraft system is associated with an operational mode status, and the connections between the components defining a plurality of series component paths within the representations of the aircraft system, each respective series component path of the plurality of series component paths having an order of priority for the respective components in the respective series component path and a set of operational mode status combinations;

applying, to each input node of the machine learning classifier, an operational mode status of a respective component and a node priority of the order of priority of the respective series component path;

varying, for an operational mode status combination, an input node state of at least one of the input nodes that has a relatively low node priority;

generating a normal state indication for the determined state of the machine learning classifier based on determining output node states do not vary in response to a corresponding varying of the input node state; and

generating an altered state indication for the determined state of the machine learning classifier based on determining a hidden layer node state and an output node state vary in response to a corresponding varying of the input node state.

2 . The method according to claim 1 , wherein the method comprises indicating a normal state of the machine learning classifier where hidden layer node states of hidden layer nodes linked to the input node do not vary in response to the varying of the input node state.

3 . The method according to claim 1 , wherein the method comprises indicating a normal state of the machine learning classifier where hidden layer node states of hidden layer nodes linked to the input node vary in response to the varying of the input node state, and output node states do not vary in response to the varying of the input node state.

4 . The method according to claim 1 , wherein each path comprises a respective path priority, and the operational mode determined by the machine learning classifier is based at least in part on the path priorities.

5 . The method according to claim 1 , wherein the method is performed when the aircraft system is non-operational.

6 . The method according to claim 1 , wherein the method is performed during operation of the aircraft system.

7 . The method according to claim 1 , wherein at least one of an input node state, a hidden layer node state, and an output node state, are bounded by maximum and minimum values.

8 . The method according to claim 1 , wherein the machine learning classifier comprises a neural network.

9 . The method according to claim 8 , wherein variation of output node states and/or hidden layer node states are determined based on at least one of a current output node state, a current hidden layer node state, a weight of a connection between an input layer node and a linked hidden layer node, a weight of a connection between a hidden layer node and a linked subsequent hidden layer node, and a weight of a connection between a hidden layer node and a linked output node.

10 . The method according to claim 8 , wherein output node states are determined not to vary in response to varying of the input node state where at least one of:

a weight of a connection between the input node and a hidden layer node to which the output node is linked is zero;

a weight of a connection between the output node and a hidden layer node to which the output node is linked is zero;

a value of a hidden layer node state of a hidden layer node to which the output node is linked is at a maximum, and a weight of a connection between the hidden layer node and the input node is positive;

a value of an output node state of the output node is at a maximum, and a weight of a connection between the output node and a hidden layer node to which the output node is linked is positive;

a value of a hidden layer node state of a hidden layer node to which the output node is linked is at a minimum, and a weight of a connection between the hidden layer node and the input node is negative; and

a value of an output node state of the output node is at a minimum, and a weight of a connection between the output node and a hidden layer node to which the output node is linked is negative.

11 . The method according to claim 8 , wherein hidden layer node states and/or output layer node states are determined to vary in response to varying of the input node state where at least one of:

a value of a hidden layer node state of a hidden layer node is at a maximum, and a weight of a connection between the hidden layer node and the input node is negative;

a value of a hidden layer node state of a hidden layer node is at a minimum, and a weight of a connection between the hidden layer node and the input node is positive;

a value of an output node state of an output node is at a maximum, and a weight of a connection between the output node and a hidden layer node to which the output node is linked is negative; and

a value of an output node state of an output node is at a minimum, and a weight of a connection between the output node and a hidden layer node to which the output node is linked is positive.

12 . The method according to claim 1 , wherein an input node state of an input node having a relatively high node priority is constant whilst varying the input node state of the input node having the relatively low node priority.

13 . The method according to claim 12 , wherein varying the input node state of the input node having the relatively low node priority takes place where the input node state of the input node having the relatively high node priority is indicative of an altered operational mode status of the component associated with the input node.

14 . The method according to claim 1 , wherein operational mode status combinations are grouped into sub-sets based on any of respective input node states and respective outputs of the machine learning classifier.

15 . The method according to claim 1 , wherein the aircraft system comprises an aircraft braking system.

16 . The method of claim 1 , further comprising:

based on generation of the altered state indication for the machine learning classifier, performing at least one of:

a) operating a secondary system instead of the machine learning classifier, wherein the secondary system comprises at least one of a secondary machine learning classifier, rule-based logic, or configuration tables;

b) generating an alert to crew of the aircraft indicating the altered state of the machine learning classifier;

c) transferring a decision regarding which components of the aircraft system to use;

d) performing further training of the machine learning classifier; and/or

e) adjusting weights of connections between nodes of the machine learning classifier.

17 . The method of claim 1 , further comprising:

based on generating the normal state indication for the machine learning classifier, enabling use of the machine learning classifier; and

determining, based on the machine learning classifier, the operational mode of the aircraft system.

18 . An aircraft system comprising:

a machine learning classifier configured to determine an operational mode of the aircraft system, the machine learning classifier having an input layer comprising input nodes, an output layer comprising output nodes, at least one hidden layer between the input layer, and the output layer, wherein the at least one hidden layer comprising hidden layer nodes, and the machine learning classifier is configured to provide an output for determining the operational mode; and

a controller configured to run the machine learning classifier that causes one or more hardware processors to perform operations comprising:

loading data for a model of the aircraft system, wherein the model comprises representations of: a) components configured to operate as part of the aircraft system, and b) connections between the components of the aircraft system, wherein each component in the aircraft system is associated with an operational mode status, and the connections between the components defining a plurality of series component paths within the representations of the aircraft system, each respective series component path of the plurality of series component paths having an order of priority for the respective components in the respective series component path and a set of operational mode status combinations;

applying, to each input node of the machine learning classifier, an operational mode status of a respective component and a node priority of the order of priority of the respective series component path;

varying, for an operational mode status combination, an input node state of at least one of the input nodes having a relatively low node priority; and

determining a state of the machine learning classifier based on determining whether a hidden layer node state and/or an output node state vary in response to the varying of the input node state;

wherein the state of the machine learning classifier is determined to be a normal state as a result of determination that output node states do not vary in response to the varying of the input node state,

wherein the state of the machine learning classifier is determined to be an altered state as a result of determination that a hidden layer node state and an output node state vary in response to the varying of the input node state.

19 . An aircraft comprising the aircraft system according to claim 18 .

20 . A non-transitory machine readable storage medium storing machine-readable instructions for use with one or more processors of an aircraft system that comprises a machine learning classifier, the machine learning classifier configured to determine an operational mode of an aircraft system, the machine learning classifier having an input layer comprising input nodes, an output layer comprising output nodes, at least one hidden layer between the input layer and the output layer, the at least one hidden layer comprising hidden layer nodes, and the machine learning classifier configured to provide an output for determining the operational mode, the machine-readable instructions comprising instructions that cause the one or more processors to perform operations comprising:

loading a model of the aircraft system, wherein the model comprises representations of: a) components configured to operate as part of the aircraft system, and b) connections between the components of the aircraft system, wherein each component in the aircraft system is associated with an operational mode status, and the connections between the components defining a plurality of series component paths within the representations of the aircraft system, each respective series component path of the plurality of series component paths having an order of priority for the respective components in the respective series component path and a set of operational mode status combinations;

applying, to each input node of the machine learning classifier, an operational mode status of a respective component and a node priority of the order of priority of the respective series component path;

varying, for an operational mode status combination, an input node state of at least one of the input nodes having a relatively low node priority; and

determining a state of the machine learning classifier based on determining whether a hidden layer node state and/or an output node state vary in response to the varying of the input node state;

wherein the state of the machine learning classifier is a normal state based on determination that output node states do not vary in response to the varying of the input node state,

wherein the state of the machine learning classifier is an altered state based on determination that a hidden layer node state and an output node state vary in response to the varying of the input node state.