IP Library Granted Patent US 11,361,058
Granted Patent B2
US 11,361,058 · App. 16/233,891 · Granted Jun 14, 2022

Method used in a mobile equipment with a trusted execution environment for authenticating a user based on his face

Inventors: Francois Moulin (Volleges, CH); Alain Fogel (Eglharting, DE)
Assignee: KeyLemon S.A.
G06F21/32G06F21/53G06N3/02G06V30/194G06V40/172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,361,058
App. No.
16/233,891
Granted
Jun 14, 2022
Kind
B2
Abstract

A method used in a mobile equipment for authenticating or identifying a user based on his face includes acquiring a sequence of biometric face sample corresponding to successive image frames of the user in a Trusted Execution Environment of the mobile equipment, and, in the Trusted Execution Environment, modifying or replacing some of the biometric face samples in the sequence, so as to generate a modified sequence with test biometric face samples. The method includes transmitting the modified sequence to a Rich Execution Environment of the mobile equipment, and classifying at least one portion of the biometric face samples using a classifier in the Rich Execution Environment, so as to generate for each classified biometric face sample a set of intermediary outputs and a classification result. The method also includes, for each classified biometric face sample, sending at least some of the intermediary outputs or classification result, and/or a digest of the intermediary outputs, to the Trusted Execution Environment. The method further includes, for at least one test biometric face sample, verifying the at least some of the intermediary outputs and/or result and/or digest in the Trusted Execution Environment.

Claims (34)

1. A method used in a mobile equipment for authenticating or identifying a user based on his face, the method comprising:

acquiring a sequence of biometric face sample corresponding to successive image frames of the user in a Trusted Execution Environment of the mobile equipment;

in the Trusted Execution Environment, modifying or replacing some of the biometric face samples in said sequence, so as to generate a modified sequence with test biometric face samples;

transmitting the modified sequence to a Rich Execution Environment of the mobile equipment;

classifying at least one portion of the biometric face samples using a classifier in the Rich Execution Environment, so as to generate for each classified biometric face sample a set of intermediary outputs and a classification result;

for each classified biometric face sample, sending at least some of the intermediary outputs or classification result, and/or a digest of the intermediary outputs, to the Trusted Execution Environment; and

for at least one test biometric face sample, verifying the at least some of the intermediary outputs and/or result and/or digest in said Trusted Execution Environment.

2. The method of claim 1 , wherein intermediary outputs and/or digests are only verified for biometric face samples or portions of biometric face samples when said classifiers delivers a positive result.

3. The method of claim 1 , wherein said Rich Execution Environment has more processing cores or graphic units or hardware neural networks than said Trusted Execution Environment.

4. The method of claim 1 , wherein said classifier comprises a neural network and wherein said intermediary outputs comprise some of the intermediary outputs of said neural network.

5. The method of claim 4 , wherein only one or more first layer of the neural network are computed in the Rich Execution Environment, while at least one last layer is only computed in the Trusted Execution Environment.

6. The method of claim 4 , wherein the at least some of the intermediary outputs are verified in the Trusted Execution Environment with a neural network equivalent to the one used in the Rich Execution Environment for computing the intermediary outputs.

7. The method of claim 1 , wherein only biometric face samples corresponding to regions of images in which a face has been authenticated are verified in said Rich Execution Environment.

8. The method of claim 1 , said digest being a hash of the intermediary outputs and/or result.

9. The method of claim 1 , wherein the verification is based on expected values for the intermediary outputs and/or result and/or digest that were pre-computed in the Trusted Execution Environment.

10. The method of claim 1 , wherein the verification is based on intermediary outputs and/or result and/or digest in said Trusted Execution Environment computed in real time in said Trusted Execution Environment.

11. The method of claim 1 , wherein the modified or replaced biometric face samples in said sequence are randomly selected in said Trusted Execution Environment.

12. The method of claim 1 , wherein said modified biometric face samples are modified by adding noise or a watermark.

13. The method of claim 1 , wherein the replaced biometric face samples are replaced by test biometric face samples selected from a set of test biometric face samples.

14. The method of claim 1 , wherein the replaced biometric face samples are replaced by test biometric face samples, the method comprising a step of selecting and storing new test biometric face samples from said acquired sequences.

15. The method of claim 1 , comprising a face detection step for detecting portions of images including a face, said face detection being performed in the Rich Execution Environment and verified in the Trusted Execution Environment.

16. The method of claim 15 , wherein only biometric face samples corresponding to regions of images in which a face has been detected are verified in said Rich Execution Environment.

17. The method of claim 1 , comprising pre-processing said image frame in order to obtain said biometric face sample.

18. The method of claim 1 , comprising unlocking said mobile equipment as soon as said user has been authenticated.

19. The method of claim 18 , comprising:

a) acquiring a sequence of image frames with at least one image sensor in said Trusted Execution Environment, said image frames comprising at least one 3D image;

b) performing the method of claim 1 on a biometric sample corresponding to one image frame or to one region of an image frame where a face has been detected;

c) repeating b) until the user has been authenticated from one of said frames or regions.

20. The method of claim 18 , wherein said test biometric face samples and the corresponding intermediary outputs or results or digests are processed before the acquisition of the sequence.

21. The method of claim 1 , wherein the user is continuously authenticated when he is facing his mobile equipment.

22. The method of claim 21 , wherein some of the biometric face samples in said sequence are randomly replaced by test biometric face samples.

23. A mobile equipment comprising:

a processor including a Trusted Execution Environment and a Rich Execution Environment, and

an electronic memory storing a program for causing said processor to perform the method of claim 1 .

Assignments (2)
MERGER Recorded Sep 11, 2023
From: KEYLEMON S.A.
To: AMS INTERNATIONAL AG
Reel/Frame 064856/0461 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2019
From: MOULIN, FRANCOIS; FOGEL, ALAIN
To: KEYLEMON S.A.
Reel/Frame 048529/0019 →
Continuity (1)
Related Publication 20190205518A1 · Jul 4, 2019