Voice recognition method and apparatus using video recognition

1. A method of recognizing a voice command by an electronic device, the method comprising:

analyzing first video data and first audio data to determine a start of speech, the first video data being a mouth shape;

when it is determined that results of both analyzing the first video data and analyzing the first audio data indicate the speech starts, converting a mode to a voice recognition mode;

when it is determined that the result of analyzing the first audio data indicates that the speech starts,

attenuating a threshold of video data for determining whether the speech starts and re-determining whether the speech starts;

when it is determined that the result of analyzing the first video data indicates that the speech starts, attenuating a threshold of audio data for determining whether the speech starts and re-determining whether the speech starts;

simultaneously displaying at least one user-selectable icon operatively connected to an executable function of the electronic device and a voice command corresponding to each of the at least one user-selectable icon in the voice recognition mode; and

determining whether the speech is terminated based on second video data and second audio data.

2. The method of claim 1 , further comprising removing display of the voice command after determining whether the speech is terminated.

3. The method of claim 2 , wherein determining whether the speech starts comprises identifying a lip of the user from the first video data and determining whether the speech starts based on whether the lip is in an open position.

4. The method of claim 1 , wherein determining whether the speech is terminated comprises determining that the speech is terminated when a voice of the user is not included in the second audio data for at least a preset time.

5. The method of claim 1 , wherein determining whether the speech is terminated comprises determining that the speech is terminated when a face of the user is not included in the second video data.

6. An electronic device which recognizes a voice command, the electronic device comprising:

an audio processor which collects and records a voice input;

a camera unit which collects and records a video input;

a display unit which displays a screen; and

a controller which:

analyzes first video data and first audio data to determine the start of speech, the first video data being a mouth shape;

when it is determined that results of both analyzing the first video data and analyzing the first audio data indicate that the speech starts, converts a mode to a voice recognition mode;

when it is determined that the result of analyzing the first audio data indicates that the speech starts, attenuates a threshold of video analysis for determining whether the speech starts and re-determines whether the speech starts;

when it is determined that the result of analyzing the first video data indicates that the speech starts, attenuates a threshold of audio analysis for determining whether the speech starts and re-determines whether the speech starts;

when it is determined that the speech starts, controls the display unit to simultaneously display at least one user-selectable icon operatively connected to an executable function of the electronic device and a voice command corresponding to each of the at least one user-selectable icon; and

determines whether the speech is terminated based on second video data and second audio data.

7. The electronic device of claim 6 ,

wherein the controller controls the display unit to remove display of the voice command when the speech is terminated.

8. The electronic device of claim 7 , wherein the controller identifies a lip of a user from the first video data, and determines whether the speech starts based on whether the lip is in an open position.

9. The electronic device of claim 6 , wherein the controller determines that the speech is terminated when a voice of the user is not included in the second audio data for at least a preset time.

10. The electronic device of claim 6 , wherein the controller determines that the speech is terminated when a face of the user is not included in the second video data.

11. A non-transitory processor-readable medium for storing a platform for recognizing a voice command, the platform comprising:

a user interface framework which displays a screen, wherein the user interface framework simultaneously displays at least one user-selectable icon operatively connected to an executable function of the electronic device and a voice command corresponding to each of the at least one user-selectable icon;

a multi-media framework which collects and records voice input and video input to generate first video data and first audio data before conversion into a voice recognition mode, and generates second video data and second audio data after conversion into the voice recognition mode; and

a voice framework which:

analyzes the first video data and the first audio data to determine the start of speech, the first video data being a mouth shape;

when it is determined that results of both analyzing the first video data and analyzing the first audio data indicate that speech starts, converts a mode to a voice recognition mode;

when it is determined that the result of analyzing the first audio data indicates that the speech starts, attenuates a threshold of video analysis for determining whether the speech starts and re-determines whether the speech starts;

when it is determined that the result of analyzing the first video data indicates that the speech starts, attenuates a threshold of audio analysis for determining whether the speech starts and re-determines whether the speech starts;

when it is determined that the speech starts, requests voice command configuration to the user interface framework, and

determines whether the speech is terminated based on the second video data and the second audio data.

12. The medium of claim 11 ,

wherein the user interface framework removes display of the voice command when the voice framework determines that the speech is terminated.

13. The medium of claim 11 , wherein the voice framework identifies a lip of a user from the first video data to determine whether the speech starts based on whether the lip is in an open position, and determines that the speech is terminated when a voice of the user is not included in the second audio data for at least a preset time.

14. The medium of claim 13 , wherein the voice framework determines that the speech is terminated when a face of the user is not included in the second video data.