IP Library › Granted Patent US 11,574,245
Granted Patent B2
US 11,574,245 · App. 16/640,549 · Granted Feb 7, 2023

Method for securing a machine learning based decision system

Inventors: Frédéric Dao (Gemenos, FR); Thomas Dandelot (Gemenos, FR); Frédéric Paillart (Gemenos, FR); Frédéric Faure (Gemenos, FR); Fabrice Delhoste (Gemenos, FR)
Assignee: THALES DIS FRANCE SAS
G06N20/00G06F21/44
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,574,245
App. No.
16/640,549
Granted
Feb 7, 2023
Kind
B2
Abstract

A system configured to perform decision tasks carried out by a machine learning engine operates with a machine learning model, and includes a training component for improving the machine learning model, a device for carrying out decisions based on a set of input data, and an interaction interface for switching the machine learning model between training component and a device that includes a model attestation checker. The device performs acquiring input data, and ascertaining at least one machine learning model over the interaction interface. The model attestation checker performs checking if said machine learning model is trusted by a model attestation, and considering, for decision making, only those machine learning models that are trusted. The machine learning engine performs carrying out the decision task for input data by using a trusted machine learning model, and providing a result attestation for the decision output.

Claims (58)

1. A method for securing a system configured to perform decision tasks carried out by a machine learning engine, equipped for operating with at least one machine learning model,

the system comprising at least one training component for improving the at least one machine learning model, a device for carrying out decisions based on said machine learning model and a set of input data, and an interaction interface for exchanging said at least one machine learning model between training component and device, wherein the device comprises a model attestation checker, wherein the method comprises the steps of:

for the device:

acquiring input data,

ascertaining at least one machine learning model over the interaction interface, for the model attestation checker:

checking if said at least one machine learning model is trusted by a model attestation,

considering by the machine learning engine, for decision making, only those machine learning models that are trusted,

for the machine learning engine:

carrying out the decision task for acquired input data by using said at least one trusted machine learning model,

providing a result attestation for the decision output.

2. The method according to claim 1 ,

wherein the training component further comprises a training attestation checker, the method further comprising the steps for the training attestation checker of:

acquiring training data,

approving training data in case acquired training data comprise a data attestation, for the training component:

improving said at least one machine learning model only with training data approved by the training attestation checker,

providing a model attestation to the at least one machine learning model.

3. The method according to claim 2 ,

the method further comprising the steps for the model attestation checker of:

acquiring at least one set of input data,

checking if said set of input data is trusted by digital signature, and for the device:

considering for said decision making only input data that are trusted.

4. The method according to claim 1 ,

wherein the training data comprise a data attestation when training data are supplied by a supervised content provider.

5. The method according to claim 1 ,

wherein the machine learning engine is capable of using at least one of input data that are not signed and a machine learning model that is not attested,

wherein in case at least one of the input data or the machine learning model is not trusted, omitting providing the decision output with a result attestation.

6. A device for performing a decision task, comprising a machine learning engine and at least one interaction interface for retrieving at least one machine learning model from a training component,

wherein the decision task is carried out with the machine learning engine, configured to use at least one machine learning model,

the device further comprises a model attestation checker for controlling the at least one interaction interface of the machine learning engine,

wherein the model attestation checker is configured to check if said machine learning model retrieved over the interaction interface is trusted by a model attestation, and

the machine learning engine is configured to consider, for decision making, only said trusted machine learning model, and to provide a result attestation for a decision output.

7. The device according to claim 6 ,

wherein the device is further configured to retrieve at least one set of input data, and the model attestation checker is configured to check if said set of input data is trusted by a digital signature,

and the machine learning engine is configured to consider for said decision making measure only a trusted input data.

8. The device according to claim 6 ,

wherein the model attestation checker is executed in a tamper proof environment being part of the device.

9. The device according to claim 6 ,

further comprising a result attestation component which is configured to provide a result attestation to the decision output in case the at least one machine learning model employed by the machine learning engine for performing the decision task is trusted by a model attestation.

10. The device according to claim 6 ,

further being communicatively coupled to at least one training component comprising a training attestation checker and a training data acquisition interface,

wherein the training data acquisition interface is configured to acquire training data,

the training attestation checker is configured to approve training data in case acquired training data comprise a data attestation,

and the training component is configured to improve the machine learning model only with training data approved by the training attestation checker,

and to provide a model attestation to the improved machine learning model.

11. The device according to claim 10 , wherein the training component is part of the device.

12. The device according to claim 10 ,

wherein the training data are further approved by the training attestation checker when the training data are supplied by a supervised content provider.

13. The device according to claim 10 ,

wherein the training data are further approved by the training attestation checker in case said training data encompass attested results from a machine learning engine using an attested machine learning model.

14. The device according to claim 6 ,

wherein the machine learning engine is capable of using at least one of input data that are not signed and a machine learning model that is not attested,

wherein in case at least one of the input data or the machine learning model is not trusted, the device is configured to omit providing a result attestation to the decision output.

15. The method of claim 3 , wherein the securing of said system sequentially implicates two levels of attestation, consisting of:

training data attestation, provided through the training attestation checker by way of certificate based attestation; and

model attestation, provided through the model attestation checker by way of certificate based attestation,

wherein when said training data attestation is available, then the training attestation checker approves signed training data for use in the machine learning model; and

thereafter,

wherein when said model attestation confirms a reliability of the machine learning model, it provides assurances that the machine learning model, and training data used therein, was not compromised since creation and is not produced by an unreliable source.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2022
From: THALES DIS FRANCE SA
To: THALES DIS FRANCE SAS
Reel/Frame 058903/0188 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 15, 2021
From: DAO, FREDERIC; DANDELOT, THOMAS; PAILLART, FREDERIC; FAURE, FREDERIC; DELHOSTE, FABRICE
To: THALES DIS FRANCE SA
Reel/Frame 055257/0287 →
Priority Claims (1)
EP 17306085 · Aug 21, 2017 · regional
Continuity (1)
Related Publication 20200219009A1 · Jul 9, 2020