Computing systems and methods for multi-modal authentication and learning

1 . A method comprising:

receiving multi-modal input from a user, wherein the multi-modal input includes facial image data, voice data, and textual input data;

feeding the multi-modal input to a plurality of machine learning (ML) models to generate a plurality of preliminary results;

applying a first weight to a first preliminary result of the plurality of preliminary results, and applying a second weight to a second preliminary result of the plurality of preliminary results, wherein the first preliminary result corresponds to the facial image data, wherein the second preliminary result corresponds to the voice data, and wherein a third preliminary result corresponds to the textual input data, and wherein the first weight is based upon a first pre-defined set of factors and wherein the second weight is based upon a second pre-defined set of factors;

determining whether a weighted sum of the preliminary results, including the first preliminary result times the first weight and the second preliminary result times the second weight, corresponds to an authentication condition; and

either allowing or disallowing the user access to a computing resource based upon whether the weighted sum of the preliminary results corresponds to the authentication condition.

2 . The method of claim 1 , wherein the first pre-defined set of factors includes a first set of environmental factors and a first set of contextual factors, and wherein the second pre-defined set of factors includes a second set of environmental factors and a second set of contextual factors.

3 . The method of claim 2 , wherein the first weight includes a product of a parameterization of the first set of environmental factors and a parameterization of the first set of contextual factors times a first sensitivity parameter, and wherein the second weight includes a product of a parameterization of the second set of environmental factors and a parameterization of the second set of contextual factors times a second sensitivity parameter.

4 . The method of claim 1 , wherein the first pre-defined set of factors includes at least one item selected from the list consisting of:

background noise; and

microphone quality.

5 . The method of claim 1 , wherein the first pre-defined set of factors includes at least one item selected from a list consisting of:

lighting conditions; and

facial pose variability.

6 . The method of claim 1 , wherein the first pre-defined set of factors includes at least one item selected from a list consisting of:

language of received text; and

typing speed for received text.

7 . The method of claim 1 , wherein the first pre-defined set of factors includes at least one item selected from a list consisting of:

time of day; and

day of the week.

8 . The method of claim 1 , wherein the first pre-defined set of factors includes at least one item selected from a list consisting of:

location of the user; and

network environment of the user.

9 . The method of claim 1 , wherein the first pre-defined set of factors includes at least one item selected from a list consisting of:

interaction patterns of the user; and

task context of the user.

10 . A method comprising:

receiving multi-modal input from a user;

feeding the multi-modal input to a plurality of machine learning (ML) models to generate a plurality of preliminary results;

applying a first weight to a first preliminary result of the plurality of preliminary results, and applying a second weight to a second preliminary result of the plurality of preliminary results, wherein the first weight is based upon a first pre-defined set of factors and wherein the second weight is based upon a second pre-defined set of factors, and wherein the first pre-defined set of factors includes a first set of environmental factors and a first set of contextual factors, and wherein the second pre-defined set of factors includes a second set of environmental factors and a second set of contextual factors;

determining whether a weighted sum of the preliminary results, including the first preliminary result times the first weight and the second preliminary result times the second weight, corresponds to an authentication condition; and

either allowing or disallowing the user access to a computing resource based upon whether the weighted sum of the preliminary results corresponds to the authentication condition.

11 . The method of claim 10 , wherein the first weight includes a product of a parameterization of the first set of environmental factors and a parameterization of the first set of contextual factors times a first sensitivity parameter, and wherein the second weight includes a product of a parameterization of the second set of environmental factors and a parameterization of the second set of contextual factors times a second sensitivity parameter.

12 . An IHS (Information Handling System) comprising:

one or more processors;

one or more memory devices coupled to the one or more processors, the one or more memory devices storing computer-readable instructions that, upon execution by the one or more processors, cause the IHS to:

receive multi-modal input from a user during authorized use of the IHS by the user, wherein the multi-modal input includes facial image data, voice data, and textual input data;

feed the multi-modal input to a plurality of machine learning (ML) models to generate a plurality of preliminary results;

apply a first weight to a first preliminary result of the plurality of preliminary results, and applying a second weight to a second preliminary result of the plurality of preliminary results, wherein the first preliminary result corresponds to the facial image data, wherein the second preliminary result corresponds to the voice data, and wherein a third preliminary result corresponds to the textual input data, and wherein the first weight is based upon a first pre-defined set of factors and wherein the second weight is based upon a second pre-defined set of factors;

determine whether a weighted sum of the preliminary results, including the first preliminary result times the first weight and the second preliminary result times the second weight, corresponds to an authentication condition; and

authorize the user and allow access to the IHS in response to the weighted sum of the preliminary results corresponding to the authentication condition.

13 . The IHS of claim 12 , further comprising computer-readable instructions to cause the IHS to:

encode the multi-modal input to generate encoded data;

decode the encoded data to generate decoded data;

determine whether an error between the decoded data and the multi-modal input indicates an anomaly; and

disallow further access to the IHS by the user based upon determining that the error indicates the anomaly.

14 . The IHS of claim 13 , wherein the computer-readable instructions cause the IHS to:

prompt the user for an alternative authentication; and

allow further access to the IHS by the user based upon the user providing the alternative authentication.

15 . The IHS of claim 12 , further comprising computer-readable instructions to cause the IHS to:

determine a validation status of a first mode of the multi-modal user input, including causing parallel validation functions to be performed at a plurality of nodes and applying a consensus function to results of the validation functions; and

either use the first mode of the multi-modal user input for incremental learning by first ML model of the plurality of ML models or discard the first mode of the multi-modal user input based upon a result of the consensus function.

16 . The IHS of claim 15 , wherein the computer-readable instructions to cause the IHS to cause the parallel validation functions to be performed includes computer-readable instructions to cause the IHS to:

cause the parallel validation functions to be performed by a plurality of software components running on the IHS, wherein each of the software components is configured to manage a respective one of the ML models.