METHOD OF VISUAL VOICE RECOGNITION WITH SELECTION OF GROUPS OF MOST RELEVANT POINTS OF INTEREST
The method comprises steps of: a) forming a starting set of microstructures of n points of interest, each defined by a tuple of order n, with n≧1; b) determining, for each tuple, associated structured visual characteristics, based on local gradient and/or movement descriptors of the points of interest; and c) iteratively searching for and selecting the most discriminant tuples. Step c) operates by: c1) applying to the set of tuples an algorithm of the Multi-Kernel Learning MKL type; c2) extracting a sub-set of tuples producing the highest relevancy scores; c3) aggregating to these tuples an additional tuple to obtain a new set of tuples of higher order; c4) determining structured visual characteristics associated to each aggregated tuple; c5) selecting a new sub-set of most discriminant tuples; and c6) reiterating steps c1) to c5) up to a maximal order N.
1 . A method for automatic language recognition by analysis of the visual voice activity of a video sequence comprising a succession of images of the mouth region of a speaker, by following-up the local deformations of a set of predetermined points of interest selected on this mouth region of the speaker,
the method being characterized in that it comprises the following steps:
a) forming a starting set of microstructures of n points of interest ( 10 ), each defined by a tuple of order n, with 1≦n≦N;
b) determining ( 30 ), for each tuple of step a), associated structured visual characteristics, based on local gradient and/or movement descriptors of the points of interest of the tuple;
c) iteratively searching for and selecting ( 32 - 36 ) the most discriminant tuples by:
c1) applying to the set of tuples an algorithm adapted to consider combinations of tuples with their associated structured characteristics and determining, for each tuple of the combination, a corresponding relevancy score;
c2) extracting, from the set of tuples considered at step c1), a sub-set of tuples producing the highest relevancy scores;
c3) aggregating additional tuples of order 1 to the tuples of the sub-set extracted at step c2), to obtain a new set of tuples of higher order;
c4) determining structured visual characteristics associated to each aggregated tuple formed at step c3);
c5) selecting, in said new set of higher order, a new sub-set of most discriminant tuples; and
c6) reiterating steps c1) to c5) up to a maximal order N; and
d) executing a visual language recognition algorithm ( 38 ) based on the tuples selected at step c).
2 . The method of claim 1 , wherein:
the algorithm of step c1) is an algorithm of the Multi-Kernel Learning MKL type;
the combinations of step c1) are linear combinations of tuples, with, for each tuple, an optimum weighting, calculated by the MKL algorithm, of its contribution in the combination; and
the sub-set of tuples extracted at step c2) is that of the tuples having the highest weights.
3 . The method of claim 1 , wherein:
steps c3) to c5) implement an algorithm adapted to:
evaluate the velocity, over a succession of images, of the points of interest of the considered tuples, and
calculate a distance between the additional tuples of step c3) and the tuples of the sub-set extracted at step c2); and
the sub-set of most discriminant tuples extracted at step c5) is that of the tuples satisfying a Variance Maximization Criterion VMC.
4 . The method of claim 1 , wherein:
steps c3) to c5) implement an algorithm of the Multi-Kernel Learning MKL type adapted to:
form linear combinations of tuples, and
calculate for each tuple an optimal weighting of its contribution in the combination; and
the sub-set of most discriminant tuples extracted at step c5) is that of the tuples having the highest weights.