IP Library › Granted Patent US 11,295,117
Granted Patent B2
US 11,295,117 · App. 16/521,519 · Granted Apr 5, 2022

Facial modelling and matching systems and methods

Inventors: Sultan A. Alrasheed (Jeddah, SA); Yasir Osama Atabani (Khartoum, SD)
Assignee: Sultan A. Alrasheed
G06K9/00288G06K9/00308G06K9/6263G06K2009/00322
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,295,117
App. No.
16/521,519
Granted
Apr 5, 2022
Kind
B2
Abstract

A matching apparatus for characterising the human face in order to facilitate the search for people with similar faces. The apparatus uses 3D modelling of a variety of image sources including video to characterise a subject's face using a set of parameters. These parameters are then used to identify other people or image sources which have a set of parameters which are similar to the subject's. Feedback from the users is used to improve future matching.

Claims (48)

1. A matching apparatus comprising a processor and memory, the processor and memory configured to:

i) receive multiple subject image sources of a subject person including at least one video;

ii) process the multiple subject image sources including images from the at least one video using 3D facial modelling to determine a neutral pose and a set of facial characteristics of the subject person in the neutral pose;

iii) determine an overall similarity score based on a degree of similarity of each of the determined set of facial characteristics, wherein a contribution to the overall similarity score for each facial characteristic is weighted by a weighting value associated with a relative importance of an associated facial characteristic in determining the overall similarity score;

iv) perform a similarity search for image sources of faces having undergone an analysis as in i) to iii) which are similar to the subject person's face based on the overall similarity score;

v) in response to the similarity search, identify one or more matching image sources of one or more candidate persons that are similar in appearance to the subject person, wherein the one or more candidate persons are persons other than the subject person;

vi) provide search results based on the identified one or more matching image sources; and

vii) adjust the weighting values based on user input, wherein the user input comprises one or more of:

the subject person requesting a digital connection with at least one of the identified matches;

the digital connection being established between the subject person and at least one of the identified matches;

a number of digital interactions between the subject person and one at least one of the identified matches after a digital relationship has been established; and

a duration of digital interaction between the subject person and one at least one of the identified matches after the digital relationship has been established.

2. The matching apparatus of claim 1 , wherein the apparatus is configured to enable the subject person to electronically contact individuals corresponding to the at least one of the identified matches.

3. The matching apparatus of claim 1 , wherein establishing the digital connection comprises an identified candidate person accepting a request for digital connection from the subject person.

4. The matching apparatus of claim 1 , wherein the digital connection is configured to enable the subject person and the identified matches to communicate directly using the matching apparatus.

5. The matching apparatus of claim 4 , wherein the digital connection is configured to facilitate direction communication via one or more of: text messages, video messages and audio calls.

6. The matching apparatus of claim 1 , wherein the digital connection is configured to allow the subject person and the identified matches to see content generated by each other.

7. The matching apparatus of claim 6 , wherein the content comprises one or more of: text and images.

8. The matching apparatus of claim 1 , wherein a said digital interaction comprises one or more of the following: one person interacting with a digital post of a second person.

9. The matching apparatus of claim 1 , wherein processing the subject image sources from the at least one video includes analysis of one or more of expression changes, movement, distance from a camera, face direction, and lighting.

10. The matching apparatus of claim 1 configured to collate image data from two or more sources or channels.

11. The matching apparatus of claim 10 , wherein the two or more sources or channels include a photograph provided by the subject person and a video harvested from a social media channel of the subject person.

12. The matching apparatus of claim 1 , wherein processing to determine the neutral pose and the set of facial characteristics of the subject person in the neutral pose includes analyzing the subject person's face and adjusting for differences between different image sources to determine features not to be used for matching.

13. The matching apparatus as in claim 1 , wherein processing to determine the neutral pose derives a neutral pose model of the subject person's resting face directed towards a camera.

14. The matching apparatus of claim 13 , wherein the neutral pose model includes determining a neutral peak as a point in time/condition at which the subject person is determined to be most neutral or most distinct in looks from media files.

15. A method of comparing facial appearance, the method comprising:

i) receiving multiple subject image sources of a subject person including at least one video;

ii) processing the multiple subject image sources including images from the at least one video using 3D facial modelling to determine a neutral pose and a set of facial characteristics of the subject person in the neutral pose;

iii) determining an overall similarity score based on a degree of similarity of each of the determined set of facial characteristics, wherein a contribution to the overall similarity score for each facial characteristic is weighted by a weighting value associated a relative importance of the associated facial characteristic in determining the overall similarity score;

iv) performing a similarity search for facial image sources of faces having undergone an analysis as in i) to ii) which are similar to the subject person's face based on the determined set of facial characteristics;

v) in response to the similarity search, identifying one or more matching image sources of one or more candidate persons that are similar in appearance to subject person, wherein the one or more candidate persons are persons other than the subject person;

vi) providing search results based on the identified one or more matching image sources; and

vii) adjusting the weighting values based on user input, wherein the user input comprises one or more of:

the subject person requesting a digital connection with at least one of the identified matches;

the digital connection being established between the subject person and at least one of the identified matches;

a number of digital interactions between the subject person and one at least one of the identified matches after a digital relationship has been established; and

a duration of digital interaction between the subject person and one at least one of the identified matches after the digital relationship has been established.

16. A non-transitory medium storing computer program code configured, when run on a computer to enable the computer to:

i) receive multiple image sources of a subject person including at least one video;

ii) process the image sources using 3D facial modelling including images from the at least one video to determine a neutral pose and a set of facial characteristics of the subject person in the neutral pose;

iii) perform a similarity search for facial image sources of faces which are similar to the subject person's face based on the determined set of facial characteristics;

iv) in response to the similarity search, identifying one or more matching image sources of one or more candidate persons having undergone an analysis as in i) to iii) that are similar in appearance to the subject person, wherein the one or more candidate persons are persons other than the subject person;

v) provide search results based on the identified one or more matching image sources; and

vi) adjust weighting values based on user input, wherein the user input comprises one or more of:

the subject person requesting a digital connection with at least one of the identified matches;

the digital connection being established between the subject person and at least one of the identified matches;

a number of digital interactions between the subject person and one at least one of the identified matches after a digital relationship has been established; and

a duration of digital interaction between the subject person and one at least one of the identified matches after the digital relationship has been established.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 19, 2020
From: ATABANI, YASIR OSAMA
To: ALRASHEED, SULTAN A.
Reel/Frame 051864/0771 →
Continuity (2)
Provisional Application 62702431 · Jul 24, 2018
Related Publication 20200034604A1 · Jan 30, 2020