Method for controlling and requesting information from displaying multimedia
The invention provides a robust method to control interactive media using gestures. A method to retrieve metadata information from a multimedia outlet device, wherein the method follows the steps: (1) extracting image hot spot areas in a current captured image using face detection, (2) detecting a human gesture directive in at least one image hot spot area using gesture recognition, (3) determining if the gesture directive matches a pre-assigned command to a rich interaction module, (4) sending a signal to a rich interaction module corresponding to the pre-assigned command detected, (5) extracting a media image hot spot area from electrical signals sent from the multimedia, (6) matching any detected human gestures in at least one image hot spot area using gesture recognition with a specific pixel on a device screen, and (7) retrieving information from metadata assigned to an area of pixels on the screen.
1. A method to retrieve metadata from a multimedia device comprising:
determining a motion blob in a captured image from a multimedia device using face detection, wherein the motion blob includes at least a first area outside of a face area;
detecting motion in at least one motion blob using gesture recognition of command gestures viewed and captured by a camera; and
retrieving metadata assigned to pixels of the at least one motion blob,
wherein the gesture recognition includes quantifying and qualifying motions over time in the at least one motion blob, and wherein the detecting comprises detecting the motion in the captured image.
2. The method of determining a motion blob in claim 1 , further comprising taking the captured image using the camera.
3. The method of determining a motion blob in claim 1 , further comprising retrieving image signals from multimedia device.
4. A media control apparatus comprising:
a camera having an image sensor;
an input image module that receives picture images from the image sensor;
a memory connected to the input image module;
a face detection module connected to the input image module, wherein the face detection module is configured to determine a motion blob including at least a first area outside of a face area;
a gesture recognition module to capture command gestures in at least one motion blob viewed through the camera connected to the input image module;
a media control interface that receives commands from the input image module and outputs electrical signals to a rich interaction module; and
a rich interaction module attached to the multimedia outlet device, capable of receiving and sending electrical image signals from and to a multimedia outlet device, said rich interaction module retrieving metadata assigned to pixels of the at least one motion blob;
wherein the gesture recognition module is adapted to quantify and qualify motions over time in the at least one motion blob, and wherein the gesture recognition module detects the motion in the current image.
5. The media control apparatus of claim 4 , wherein the image sensor is integral with the camera.
6. The media control apparatus of claim 4 , wherein the input image module is integral with the camera.
7. The media control apparatus of claim 4 , wherein the input image module is a microprocessor.
8. The media control apparatus of claim 4 , wherein the memory, the face detection module and the gesture recognition module are integral with the input image.
9. The media control apparatus of claim 4 , wherein the media control interface is integral with the input image module.
10. The media control apparatus of claim 4 , wherein:
the camera, image sensor, input image module, memory, face detection module, gesture recognition module, and media control interface are integrally constructed as one component; and
the media control apparatus is an external component connected to the multimedia outlet device.
11. The media control apparatus of claim 4 , wherein:
the camera, image sensor, input image module, memory, face detection module, gesture recognition module, and media control interface are integrated in to one component; and
the media control apparatus is an internal component of the multimedia outlet device.
12. The method according to claim 1 , wherein said detecting the motion in the current captured image comprises finding, in the motion history image, largest x, y coordinates and smallest x, y coordinates that are denoted each as lx, ly, sx and sy, and then comparing each of (ly−sy) and (lx−sx) with a corresponding threshold.
13. The media control apparatus according to claim 4 , wherein the gesture recognition module detects the motion in the current image by finding largest x, y and smallest x, y coordinates that are denoted each as lx, ly, sx and sy, and then comparing each of (ly−sy) and (lx−sx) with a corresponding threshold.