Character mouth shape control method
View Patent ↗A section corresponding to a given duration is sampled from sound data that indicates the voice of a player collected by a microphone, and a vocal tract cross-sectional area function of the sampled section is calculated. The vertical dimension of the mouth is calculated from a throat-side average cross-sectional area of the vocal tract cross-sectional area function, and the area of the mouth is calculated from a mouth-side average cross-sectional area. The transverse dimension of the mouth is calculated from the area of the mouth and the vertical dimension of the mouth.
1. A method of controlling a mouth shape of an animated character on a display that is implemented by a processor, the method comprising:
estimating by the processor the shape of a vocal tract of a speaker during speaking by analyzing sound data of the speaker's voice and estimating the shape based on (i) an open area of the vocal tract, (ii) a cross-sectional area within a given mouth-side range of the vocal tract, and (iii) a cross-sectional area within a given throat-side range of the vocal tract that is closer to a throat side of the vocal tract than the given mouth-side range; and
controlling using the processor, the mouth shape of the animated character using the estimated shape of the vocal tract, including controlling a vertical dimension of the mouth of the character based on the cross-sectional area within the given throat-side range and controlling a size of the mouth of the animated character based on the cross-sectional area within the given mouth-side range;
displaying the animated character, using the display;
controlling the output voice of the animated character based on the sound data; and
controlling the mouth shape of the character in synchronization with the output voice.
2. The method as defined in claim 1 , wherein
the estimating of the shape of the vocal tract includes analyzing the sound data to calculate the shape of a vocal tract model, the vocal tract model having a plurality of cross-sectional areas including the open area, the cross-sectional area within the given mouth-side range, and the cross-sectional area within the given throat-side range.
3. The method as defined in claim 1 , further comprising:
controlling output of voice based on the sound data; and
controlling the mouth shape of the character in synchronization with the output voice.
4. The method as defined in claim 1 , wherein
the controlling of the mouth shape of the character includes decreasing the vertical dimension as the cross-sectional area within the given throat-side range increases.
5. The method as defined in claim 1 , further comprising:
detecting whether or not the sound data indicates sound that corresponds to a fricative, wherein
the controlling of the mouth shape of the character includes decreasing a size of the mouth of the character when sound that corresponds to a fricative has been detected as compared with the size of the mouth of the character when sound that corresponds to a fricative has not been detected.
6. The method as defined in claim 1 , further comprising:
detecting whether or not the sound data indicates sound that corresponds to a fricative; and
determining a fricative type when sound that corresponds to a fricative has been detected, wherein
the controlling of the mouth shape of the character includes controlling the mouth shape based on the determined fricative type.
7. The method as defined in claim 1 , further comprising:
detecting whether or not the sound data indicates sound that corresponds to a nasal, wherein
the controlling of the mouth shape of the character includes decreasing a size of the mouth of the character when sound that corresponds to a nasal has been detected as compared with the size of the mouth of the character when sound that corresponds to a nasal has not been detected.
8. The method as defined in claim 1 , further comprising:
detecting whether or not the sound data indicates sound that corresponds to a nasal; and
determining a nasal type when sound that corresponds to a nasal has been detected, wherein
the controlling of the mouth shape of the character includes controlling the mouth shape based on the determined nasal type.
9. The method as defined in claim 1 , further comprising:
detecting whether or not the sound data indicates a silent state, wherein
the controlling of the mouth shape of the character includes gradually decreasing a size of the mouth of the character with time as a duration of the silent state increases.
10. The method as defined in claim 1 , wherein
the controlling of the mouth shape of the character includes suppressing a sudden change of the mouth shape so that the mouth shape gradually changes.
11. A mouth shape control device comprising:
a vocal tract shape calculation section that estimates the shape of a vocal tract of a speaker during speaking by analyzing sound data of the speaker's voice and estimating the shape based on (i) an open area of the vocal tract, (ii) a cross-sectional area within a given mouth-side range of the vocal tract, and (iii) a cross-sectional area within a given throat-side range of the vocal tract that is closer to a throat side of the vocal tract than the given mouth-side range; and
a mouth shape control section that controls the mouth shape of a character using the shape of the vocal tract estimated by the vocal tract shape calculation section, including i) controlling a vertical dimension of the mouth of the character based on the cross-sectional area within the given throat-side range; ii) controlling a size of the mouth of the character based on the cross-sectional area within the given mouth-side range; iii) controlling the output voice of the character based on the sound data; and iv) controlling the mouth shape of the character in synchronization with the output voice.