IP Library › Granted Patent US 7,352,881
Granted Patent B2
US 7,352,881 · App. 10/663,689 · Granted Apr 1, 2008

Method for tracking facial features in a video sequence

Assignee: Canon Kabushiki Kaisha
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,352,881
App. No.
10/663,689
Granted
Apr 1, 2008
Kind
B2
Abstract

A method ( 100 ) of tracking facial features in a video sequence is disclosed. The method receives ( 110 ) as input facial features for tracking in a first frame of the video sequence. The video sequence is also segmented ( 120 ) using spatiotemporal segmentation to provide a sequence of associated two-dimensional segments, with a first segment in the sequence of associated two-dimensional segments including the facial features for tracking. Candidate facial features are then identified ( 140 ) in at least a second two-dimensional segment in the sequence of associated segments. Method ( 100 ) then verifies ( 155 ) which of the candidate facial features correspond with the facial features for tracking. Method ( 100 ) also recovers lost facial features by using known geometric relations between facial features.

Claims (35)

1. A method of tracking facial features in a video sequence, said method comprising the steps of:

(a) receiving facial features for tracking in a first frame of said video sequence;

(b) spatiotemporally segmenting said video sequence to provide a sequence of associated two-dimensional segments, a first two-dimensional segment in said sequence of associated two-dimensional segments including said facial features for tracking;

(c) identifying candidate facial features in a second two-dimensional segment in a second frame of said video sequence, said second two-dimensional segment being one of said sequence of associated two-dimensional segments; and

(d) verifying which of said candidate facial features correspond with said facial features for tracking.

2. A method as claimed in claim 1 comprising the further step of:

(e) recovering lost facial features by using known geometric relations between facial features.

3. A method as claimed in claim 1 wherein step (c) comprises the sub-steps of:

(ci) forming a sub-image including said second two-dimensional segment in said sequence of associated two-dimensional segments;

(cii) normalising the size of said sub-image; and

(ciii) identifying candidate facial features in said normalised sub-image.

4. A method as claimed in 1 wherein step (d) measures the correspondence between said candidate facial features and said facial features for tracking.

5. A method as claimed in claim 4 wherein step (d) comprises determining whether said candidate facial features are within a region of said facial features for tracking in a previous frame.

6. A method as claimed in claim 5 wherein step (d) further comprises determining whether said candidate facial features within each of said regions are similar in shape to said facial features for tracking in said previous frame.

7. An apparatus for tracking facial features in a video sequence, said apparatus comprising:

means for receiving facial features for tracking in a first frame of said video sequence;

means for spatiotemporally segmenting said video sequence to provide a sequence of associated two-dimensional segments, a first two-dimensional segment in said sequence of associated two-dimensional segments including said facial features for tracking;

means for identifying candidate facial features in a second two-dimensional segment in a second frame of said video sequence, said second two-dimensional segment being one of said sequence of associated two-dimensional segments; and

means for verifying which of said candidate facial features correspond with said facial features for tracking.

8. An apparatus as claimed in claim 7 further comprising: means for recovering lost facial features by using known geometric relations between facial features.

9. An apparatus as claimed in claim 7 wherein said means for identifying comprises:

means for forming a sub-image including said second two-dimensional segment in said sequence of associated two-dimensional segments;

means for normalising the size of said sub-image; and

means for identifying candidate facial features in said normalised sub-image.

10. A program stored on a memory medium for tracking facial features in a video sequence, said program comprising:

code for receiving facial features for tracking in a first frame of said video sequence;

code for spatiotemporally segmenting said video sequence to provide a sequence of associated two-dimensional segments, a first two-dimensional segment in said sequence of associated two-dimensional segments including said facial features for tracking;

code for identifying candidate facial features in at least a second two-dimensional segment in a second frame of said video sequence, said second two-dimensional segment being one of said sequences of associated two-dimensional segments; and

code for verifying which of said candidate facial features correspond with said facial features for tracking.

11. A program as claimed in claim 10 further comprising:

code for recovering lost facial features by using known geometric relations between facial features.

12. A program as claimed in claim 10 wherein said code for identifying comprises:

code for forming a sub-image including said second two-dimensional segment in said sequence of associated two-dimensional segments;

code for normalising the size of said sub-image; and

code for identifying candidate facial features in said normalised sub-image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2004
From: LESTIDEAU, FABRICE
To: CANON KABUSHIKI KAISHA
Reel/Frame 014858/0682 →
Priority Claims (1)
AU 2002951473 · Sep 18, 2002 · national
Continuity (1)
Related Publication 20040109584A1 · Jun 10, 2004