IP Library › Granted Patent US 10,713,532
Granted Patent B2
US 10,713,532 · App. 15/925,028 · Granted Jul 14, 2020

Image recognition method and apparatus

Inventors: Shouhong Ding (Shenzhen, CN); Jilin Li (Shenzhen, CN); Chengjie Wang (Shenzhen, CN); Feiyue Huang (Shenzhen, CN); Yongjian Wu (Shenzhen, CN); Guofu Tan (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G06K9/6256G06K9/00248G06K9/00268G06K9/00281G06K9/00288G06K9/6247G06K9/6278G06K2009/4666
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,713,532
App. No.
15/925,028
Granted
Jul 14, 2020
Kind
B2
Abstract

The present disclosure discloses an image recognition method and apparatus, and belongs to the field of computer technologies. The method includes: extracting a local binary pattern (LBP) feature vector of a target image; calculating a high-dimensional feature vector of the target image according to the LBP feature vector; obtaining a training matrix, the training matrix being a matrix obtained by training images in an image library by using a joint Bayesian algorithm; and recognizing the target image according to the high-dimensional feature vector of the target image and the training matrix. The image recognition method and apparatus according to the present disclosure may combine LBP algorithm with a joint Bayesian algorithm to perform recognition, thereby improving the accuracy of image recognition.

Claims (208)

1. An image recognition method, comprising:

extracting a local binary pattern (LBP) feature vector of a target image, the target image being a face image;

calculating a high-dimensional feature vector of the target image according to the LBP feature vector;

obtaining a training matrix, the training matrix being a matrix obtained by training images in an image library by using a joint Bayesian algorithm, the image library comprising m*n face images, m being a number of persons corresponding to images in the image library, and n being a number of face images of each person; and

performing image recognition on the target image according to the high-dimensional feature vector of the target image and the training matrix,

wherein obtaining the training matrix comprises:

obtaining a high-dimensional feature vector of each person corresponding to the face images in the image library;

initializing a first covariance matrix of n face images of each person in the image library and a second covariance matrix of images of different persons in the image library, the first covariance matrix being an m-dimensional square matrix and the second covariance matrix being an m*n square matrix;

calculating a mean value of Gaussian distribution of each person in the image library;

calculating a joint distribution covariance matrix of two persons in the image library according to the first covariance matrix, the second covariance matrix, and the high-dimensional feature vectors of the two persons;

updating the first covariance matrix according to the mean value of Gaussian distribution, and updating the second covariance matrix according to the joint distribution covariance matrix; and

obtaining the training matrix based on the updated first covariance matrix and the updated second covariance matrix.

2. The method according to claim 1 , wherein performing image recognition on the target image according to the high-dimensional feature vector of the target image and the training matrix comprises:

obtaining a high-dimensional feature vector of a reference image;

calculating a similarity between the target image and the reference image according to the high-dimensional feature vector of the target image, the high-dimensional feature vector of the reference image, and the training matrix:

r ( x 1 ,x 2 )= x 1 T Ax 1 +x 2 T Ax 2 −2 x 1 T Gx 2 ;

wherein x 1 is the high-dimensional feature vector of the target image, x 2 is the high-dimensional feature vector of the reference image, x 1 T a transposed vector of x 1 , x 2 T is a transposed vector of x 2 , and A and G are training matrixes.

3. The method according to claim 2 , wherein the image library comprises the target image and the reference image.

4. The method according to claim 1 , wherein the extracting an LBP feature vector of a target image comprises:

separately scaling the target image according to k preset scaling factors to obtain k scaled images, k being a positive integer;

determining, for each of the target image and the k scaled images, a key point in the image, and extracting an LBP feature vector at the key point; and

determining the LBP feature vector of the target image according to the extracted LBP feature vector of the key point.

5. The method according to claim 1 , wherein the calculating a high-dimensional feature vector of the target image according to the LBP feature vector comprises:

performing principal component analysis (PCA) dimension reduction on x r T x r , and reserving features of first N dimensions, to obtain a reduced-dimension matrix P, N being a positive integer, x r being the LBP feature vector of the target image, and x r T being a transposed vector of x r ;

for x r , calculating X p =Px r ;

performing latent Dirichlet allocation (LDA) dimension reduction on x p to obtain a reduced-dimension matrix L; and

calculating the high-dimensional feature vector of the target image x=Lx p .

6. The method according to claim 1 , wherein obtaining the training matrix, further comprises:

initializing the first covariance matrix S μ of n face images of each person in the image library and the second covariance matrix S ε of images of different persons in the image library;

calculating F and G: F=S μ −1 , G=−(mS μ +S ε ) −1 S μ S ε −1 ;

calculating the mean value μ i of Gaussian distribution of an i th person in the image library: μ i =Σ i=1 m S μ (F+mG)x i , and the joint distribution covariance matrix ε ij of the i th person and a j th person according to F and G: ε ij =x j +Σ i=1 m S ε Gx i , x i being a high-dimensional feature vector of the i th person, and x j being a high-dimensional feature vector of the j th person;

updating S μ and S ε according to the following formula:

S

μ

=

cov

⁡

(

μ

)

=

m

-

1

⁢

∑

i

⁢

⁢

μ

i

⁢

μ

i

T

,

⁢

S

ɛ

=

cov

⁡

(

ɛ

)

=

m

-

1

⁢

∑

i

⁢

⁢

∑

j

⁢

ɛ

ij

⁢

ɛ

ij

T

wherein μ i T is a transposed vector of μ i , and ε ij T is a transposed vector of ε ij ; and

calculating the training matrixes A=(S μ +S ε ) −1 −(F+G) and G=−(mS μ +S ε ) −1 S μ S ε −1 according to the updated S μ and S ε , F, and G.

7. The method according to claim 6 , further comprising:

determining whether the updated S μ and S ε are convergent; and

if the updated S μ and S ε are not convergent, performing the step of calculating F and G again; or

if the updated S μ and S ε are convergent, performing a step of calculating the training matrix according to F and G corresponding to the convergent S μ and S ε , and the convergent S μ and S ε .

8. The method according to claim 1 , wherein the obtaining a high-dimensional feature vector of each person corresponding to the images in the image library comprises:

extracting, for each face image in the image library, an LBP feature vector of the face image;

calculating a mean value of all features in the LBP feature vector of the face image;

subtracting the mean value from each feature in the LBP feature vector of the face image to obtain a feature vector x r ={x ri , 0<i<m*n}, wherein x ri represents an LBP feature vector of the face image after mean value processing;

performing PCA dimension reduction on x r T x r , and reserving features of the first N dimensions, to obtain a reduced-dimension matrix P, N being a positive integer, and x r T being a transposed vector of x r ;

for x r , calculating x p =Px r ;

performing, for face images of each person, LDA dimension reduction on x p to obtain a reduced-dimension matrix L; and

calculating the high-dimensional feature vector of each person: x=Lx p .

9. An image recognition apparatus, comprising:

at least one processor; and

a memory, the memory storing program instructions, and the image recognition apparatus being configured to perform the following operations when the program instructions are performed by the processor:

extracting a local binary pattern (LBP) feature vector of a target image, the target image being a face image;

calculating a high-dimensional feature vector of the target image according to the LBP feature vector;

obtaining a training matrix, the training matrix being a matrix obtained by training images in an image library by using a joint Bayesian algorithm, the image library comprising m*n face images, m being a number of persons corresponding to images in the image library, and n being a number of face images of each person; and

performing image recognition on the target image according to the high-dimensional feature vector of the target image and the training matrix,

wherein obtaining the training matrix comprises:

obtaining a high-dimensional feature vector of each person corresponding to the face images in the image library;

initializing a first covariance matrix of n face images of each person in the image library and a second covariance matrix of images of different persons in the image library, the first covariance matrix being an m-dimensional square matrix and the second covariance matrix being an m*n square matrix;

calculating a mean value of Gaussian distribution of each person in the image library;

calculating a joint distribution covariance matrix of two persons in the image library according to the first covariance matrix, the second covariance matrix, and the high-dimensional feature vectors of the two persons;

updating the first covariance matrix according to the mean value of Gaussian distribution, and updating the second covariance matrix according to the joint distribution covariance matrix; and

obtaining the training matrix based on the updated first covariance matrix and the updated second covariance matrix.

10. The apparatus according to claim 9 , wherein the performing image recognition on the target image according to the high-dimensional feature vector of the target image and the training matrix comprises:

obtaining a high-dimensional feature vector of a reference image;

calculating a similarity between the target image and the reference image according to the high-dimensional feature vector of the target image, the high-dimensional feature vector of the reference image, and the training matrix:

r ( x 1 ,x 2 )= x 1 T Ax 1 +x 2 T Ax 2 −2 x 1 T Gx 2 ;

wherein x 1 is the high-dimensional feature vector of the target image, x 2 is the high-dimensional feature vector of the reference image, x 1 T a transposed vector of x 1 , x 2 T is a transposed vector of x 2 , and A and G are training matrixes.

11. The apparatus according to claim 10 , wherein the image library comprises the target image and the reference image.

12. The apparatus according to claim 9 , wherein the extracting an LBP feature vector of a target image comprises:

separately scaling the target image according to k preset scaling factors to obtain k scaled images, k being a positive integer;

determining, for each of the target image and the k scaled images, a key point in the image, and extracting an LBP feature vector at the key point; and

determining the LBP feature vector of the target image according to the LBP feature vector of the extracted key point.

13. The apparatus according to claim 9 , wherein the calculating a high-dimensional feature vector of the target image according to the LBP feature vector comprises:

performing principal component analysis (PCA) dimension reduction on x r T x r , and reserving features of first N dimensions, to obtain a reduced-dimension matrix P, N being a positive integer, x r being the LBP feature vector of the target image, and x r T being a transposed vector of x r ;

for x r , calculating X p =Px r ;

performing latent Dirichlet allocation (LDA) dimension reduction on x p to obtain a reduced-dimension matrix L; and

calculating the high-dimensional feature vector of the target image x=Lx p .

14. The apparatus according to claim 9 , wherein obtaining the training matrix further comprises:

initializing the first covariance matrix S μ of n face images of each person in the image library and the second covariance matrix S ε of images of different persons in the image library;

calculating F and G: F=S μ −1 , G=−(mS μ +S ε ) −1 S μ S ε −1 ;

calculating the mean value μ i of Gaussian distribution of an i th person in the image library: μ i =Σ i=1 m S μ (F+mG)x i , and the joint distribution covariance matrix ε ij of the i th person and a j th person according to F and G: ε ij =x j +Σ i=1 m S ε Gx i , x i being a high-dimensional feature vector of the i th person, and x j being a high-dimensional feature vector of the j th person;

updating S μ and S ε according to the following formula:

S

μ

=

cov

⁡

(

μ

)

=

m

-

1

⁢

∑

i

⁢

⁢

μ

i

⁢

μ

i

T

,

⁢

S

ɛ

=

cov

⁡

(

ɛ

)

=

m

-

1

⁢

∑

i

⁢

⁢

∑

j

⁢

ɛ

ij

⁢

ɛ

ij

T

wherein μ i T is a transposed vector of μ i , and ε ij T is a transposed vector of ε ij ; and

calculating the training matrixes A=(S μ +S ε ) −1 −(F+G) and G=−(mS μ +S ε ) −1 S μ S ε −1 according to the updated S μ and S ε , F, and G.

15. The apparatus according to claim 14 , wherein the operations further comprise:

determining whether the updated S μ and S ε are convergent; and

if the updated S μ and the updated S ε are not convergent, performing the step of calculating F and G again; or

if the updated S μ and the updated S ε are convergent, performing a step of calculating a training matrix according to F and G corresponding to the convergent S μ and S ε , and the convergent S μ and S ε .

16. The apparatus according to claim 9 , wherein the obtaining a high-dimensional feature vector of each person corresponding to images in the image library comprises:

extracting, for each face image in the image library, an LBP feature vector of the face image;

calculating a mean value of all features in the LBP feature vector of the face image;

subtracting the mean value from each feature in the LBP feature vector of the face image to obtain a feature vector x r ={x ri 0<i<m*n}, wherein x ri represents an LBP feature vector of the face image after mean value processing;

performing PCA dimension reduction on x r T x r , and reserving features of the first N dimensions, to obtain a reduced-dimension matrix P, N being a positive integer, and x r T being a transposed vector of x r ;

for x r , calculating x p =Px r ;

performing, for face images of each person, LDA dimension reduction on x p to obtain a reduced-dimension matrix L; and

calculating the high-dimensional feature vector of each person: x=Lx p .

17. A non-transitory computer-readable storage medium, the storage medium storing program instructions, and when being executed by a processor of a computing device, the program instructions causing the processor to perform:

extracting a local binary pattern (LBP) feature vector of a target image, the target image being a face image;

calculating a high-dimensional feature vector of the target image according to the LBP feature vector;

obtaining a training matrix, the training matrix being a matrix obtained by training images in an image library by using a joint Bayesian algorithm, the image library comprising m*n face images, m being a number of persons corresponding to images in the image library, and n being a number of face images of each person; and

performing image recognition on the target image according to the high-dimensional feature vector of the target image and the training matrix,

wherein obtaining the training matrix comprises:

obtaining a high-dimensional feature vector of each person corresponding to the face images in the image library;

initializing a first covariance matrix of n face images of each person in the image library and a second covariance matrix of images of different persons in the image library, the first covariance matrix being an m-dimensional square matrix and the second covariance matrix being an m*n square matrix;

calculating a mean value of Gaussian distribution of each person in the image library;

calculating a joint distribution covariance matrix of two persons in the image library according to the first covariance matrix, the second covariance matrix, and the high-dimensional feature vectors of the two persons;

updating the first covariance matrix according to the mean value of Gaussian distribution, and updating the second covariance matrix according to the joint distribution covariance matrix; and

obtaining the training matrix based on the updated first covariance matrix and the updated second covariance matrix.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2018
From: DING, SHOUHONG; LI, JILIN; WANG, CHENGJIE; HUANG, FEIYUE; WU, YONGJIAN; TAN, GUOFU
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 045275/0803 →
Priority Claims (1)
CN 2015 1 0843973 · Nov 26, 2015 · national
Continuity (2)
Continuation PCTCN2016084418 · Jun 2, 2016
Related Publication 20180204094A1 · Jul 19, 2018
Cited By (1)
US 12,380,611