Systems and methods for determining personal characteristics
View Patent ↗Systems and methods are disclosed for determining personal characteristics from images by generating a baseline gender model and an age estimation model using one or more convolutional neural networks (CNNs); capturing correspondences of faces by face tracking, and applying incremental learning to the CNNs and enforcing correspondence constraint such that CNN outputs are consistent and stable for one person.
1. A computer implemented method to classify camera images, comprising:
a. generating a baseline gender model and an age estimation model using one or more convolutional neural networks (CNNs);
b. capturing correspondences of faces by face tracking;
c. applying a stochastic update rule for each face image determined as
Ω
-
γ
t
[
λ
(
Ω
-
Ω
0
)
+
(
F
(
X
t
,
Ω
)
-
y
t
~
)
∂
∂
Ω
F
(
X
,
θ
)
]
,
where
y
t
~
=
1
T
(
X
t
)
∑
Z
∈
T
(
X
t
)
F
(
Z
,
Ω
)
,
γ t is a step size, term λ(Ω−Ω 0 ) represents weight decay, and a baseline neural network function is y=F(X,Ω 0 ), and
d. applying incremental learning to the CNNs and enforcing correspondence constraint such that CNN outputs are consistent and stable for one person.
2. The method of claim 1 , comprising collecting correspondences of faces by face tracking.
3. The method of claim 1 , comprising providing incremental learning in the neural network by enforcing a correspondence constraint.
4. The method of claim 1 , comprising implementing incremental training with an online stochastic gradient descent process.
5. The method of claim 1 , comprising applying a correspondence given by visual tracking to update a pre-trained model.
6. The method of claim 1 , comprising:
a. performing face detection and tracking;
b. aligning the detected faces;
c. normalizing the faces to a plurality of patches; and
d. sending the normalized faces to the CNNs to estimate gender and age.
7. The method of claim 1 , comprising applying CNNs to face detection and face alignment.
8. The method of claim 1 , comprising performing multi-hypothesis visual tracking to obtain correspondence of faces in successive video frames.
9. The method of claim 1 , comprising updating the baseline models using data collected online to avoid model drift.
10. The method of claim 1 , comprising applying weakly supervised incremental training to face correspondences.
11. A system to classify camera images, comprising
a. means for generating a baseline gender model and an age estimation model using one or more convolutional neural networks (CNNs);
b. means for capturing correspondences of faces by face tracking;
c. means for applying a stochastic update rule for each face image determined as
Ω
-
γ
t
[
λ
(
Ω
-
Ω
0
)
+
(
F
(
X
t
,
Ω
)
-
y
t
~
)
∂
∂
Ω
F
(
X
,
θ
)
]
,
where
y
t
~
=
1
T
(
X
t
)
∑
Z
∈
T
(
X
t
)
F
(
Z
,
Ω
)
,
γ t is the step size, term λ(Ω−Ω 0 ) represents weight decay, and a baseline neural network function is represented by y=F(X,Ω 0 ), and
d. means for applying incremental learning to the CNNs and enforcing correspondence constraint such that CNN outputs are consistent and stable for one person.
12. The system of claim 11 , comprising means for collecting correspondences of faces by face tracking.
13. The system of claim 11 , comprising means for providing incremental learning in the neural network by enforcing a correspondence constraint.
14. The system of claim 11 , comprising means for implementing incremental training with an online stochastic gradient descent process.
15. The system of claim 11 , comprising means for applying a correspondence given by visual tracking to update a pre-trained model.
16. The system of claim 11 , comprising:
a. means for performing face detection and tracking;
b. means for aligning the detected faces;
c. means for normalizing the faces to a plurality of patches; and
d. means for sending the normalized faces to the CNNs to estimate gender and age.
17. The system of claim 11 , comprising means for applying CNNs to face detection and face alignment.
18. The system of claim 11 , comprising means for performing multi-hypothesis visual tracking to obtain correspondence of faces in successive video frames.
19. The system of claim 11 , comprising means for updating the baseline models using data collected online to avoid model drift.
20. The system of claim 11 , comprising means for applying weakly supervised incremental training to face correspondences.