Image matching-based pointing techniques
Techniques are disclosed that can be implemented as a system configured for image matching-based pointing and/or selection from information presented in an image displayed by a projection device. A pointing device configured as described herein may be pointed at a target portion of an image displayed by a projection device and allowed to capture an image inclusive of that target portion. In turn, the captured image data may be compared with data pertaining to the original displayed image. The resultant pointing location data can be used to generate one or more pointing indicators within the original image, and the projection device may display the resultant annotated image on screen, in some cases in real time and/or with precision resolution of a single pixel. A subsequent user selection can be made based on the known location of the pointing indicator. The techniques provide for more native human-machine interaction than existing approaches.
1. A method for providing user interaction with a machine that present images via a display, the method comprising:
receiving captured image data, the captured image data comprising at least part of a displayed image and pointing location data;
comparing the captured image data with original image data from which the displayed image is generated, the original image data not including the pointing location data;
identifying the pointing location data based on such comparison;
identifying a location within the displayed image of a pointer indicator based on the identified pointing location data; and
receiving an input based on selection of a feature within the displayed image, the feature being at the identified location and associated with a function to be carried out in response to the selection.
2. The method of claim 1 , wherein the captured image data includes the entirety of the displayed image.
3. The method of claim 1 , wherein the captured image data is provided by an image capture device hosted by a pointing device, the image capture device comprising a still camera or a video camera.
4. The method of claim 1 , wherein the captured image data is provided continuously.
5. The method of claim 1 , wherein comparing the captured image data with the original image data comprises at least one of:
performing edge detection between the captured image data and the original image data;
performing pixel comparison between the captured image data and the original image data; and
matching a center of the captured image with the target location within the displayed image.
6. The method of claim 1 , further comprising updating the original image data to include the pointer indicator at the identified location, wherein at least one of comparing the captured image data with the original image data and updating the original image data to include the pointer indicator is performed in real time.
7. The method of claim 1 , wherein the machine is at least one of a television, set-top box, a projector, a fixed computing system, a mobile computing system, and a media player.
8. One or more non-transitory computer program products encoded with instructions that, when executed by one or more processors, cause a process to be carried out, the process comprising:
receiving captured image data, the captured image data comprising at least part of a displayed image and pointing location data;
comparing the captured image data with original image data from which the displayed image is generated, the original image data not including the pointing location data;
identifying the pointing location data based on such comparison;
identifying a location within the displayed image of a pointer indicator based on the identified pointing location data;
receiving an input based on selection of a feature within the displayed image, the feature being at the identified location and associated with a function to be carried out in response to the selection.
9. The one or more non-transitory computer program products of claim 8 , wherein the captured image data includes the entirety of the displayed image.
10. The one or more non-transitory computer program products of claim 8 , wherein the captured image data is provided continuously.
11. The one or more non-transitory computer program products of claim 8 , the process further comprising updating the original image data to include the pointer indicator at the identified location.
12. A system comprising the one or more non-transitory computer program products of claim 8 , wherein the system further comprises at least one of a television, set-top box, a projector, a fixed computing system, a mobile computing system, a game console, and a media player.
13. The system of claim 12 , further comprising a remote control configured with an image capture device to provide the captured image data, wherein the user input based on selection of a feature within the displayed image is responsive to a user selection via the remote control.
14. A user interface system, comprising:
a processor;
a pointing calculation module executable by the processor and configured to:
receive captured image data, the captured image data comprising at least part of a displayed image and pointing location data;
compare the captured image data with original image data from which the displayed image is generated, the original image data not including the pointing location data;
identify the pointing location data based on such comparison; and
identify a location within the displayed image of a pointer indicator based on the identified pointing location data;
a marking module executable by the processor and configured to update the original image data to include the pointer indicator at the identified location; and
a remote control configured with an image capture device to provide the captured image data, wherein an input selecting a feature within the displayed image is responsive to a selection via the remote control.
15. The system of claim 14 , further comprising at least one of a television, set-top box, a projector, a fixed computing system, a mobile computing system, a game console, and a media player.
16. The system of claim 15 , wherein the image capture device is configured to provide the captured image data, wherein the input provided to the system is based at least in part on the remote control pointing at a target feature within the displayed image.
17. The system of claim 16 , wherein the remote control is for controlling at least one of a television and a set-top box.
18. The system of claim 16 , wherein the remote control is a game controller for controlling at least one of a game console and a set-top box.
19. The system of claim 16 , wherein the remote control is for controlling a projector.
20. The system of claim 16 , wherein the remote control is for controlling a computing system.