System and method for tracking an input device using a display screen in captured frames of image data
View Patent ↗A system and method for tracking an input device uses positional information of a display screen in frames of image data to determine the relative position of the input device with respect to the display screen.
1. A method for tracking an input device, said method comprising:
electronically capturing at least a portion of a display screen in frames of image data without capturing any beacons using an image sensor array of said input device as said input device is moved relative to the display screen;
extracting different positional information of said display screen in said frames of image data without any beacons captured by said input device as said input device is moved relative to the display screen using only said portion of the display screen that is electronically captured in said frames of image data, including identifying at least a portion of an outline of said display screen to extract said different positional information of said display screen; and
determining relative positions of said input device with respect to said display screen using said different positional information of said display screen in said frames of image data to track movements of said input device relative to said display screen.
2. The method of claim 1 wherein said extracting includes locating a plurality of corners of said display screen in said frames of image data, and wherein said determining includes determining said relative positions of said input device using locations of said corners of said display screen in said frames of image data.
3. The method of claim 2 wherein said extracting further includes calculating an estimated location of a missing corner of said display screen in one or more of said frames of image data.
4. The method of claim 1 wherein said extracting includes thresholding pixel values of said frames of image data to find said display screen in said image frames.
5. The method of claim 1 wherein said extracting includes searching for a quadrilateral area having a dominant color in said frames of image data to find said display screen in said frames of image data.
6. The method of claim 1 wherein said extracting includes searching for a quadrilateral area having an image pattern similar to a reference image pattern in said frames of image data to find said display screen in said frames of image data, said reference image pattern being associated with an image being displayed on said display screen.
7. The method of claim 1 further comprising performing scene-based navigation on some of said frames of image data to determine said relative positions of said input device.
8. The method of claim 7 wherein said performing includes converting some of said frames of image data to lower resolution frames of image data to reduce the amount of computations needed for said scene-based navigation.
9. A system for optical navigation, said system comprising:
an input device including an image sensor array configured to capture frames of image data as said input device is moved relative to a display screen, said frames of image data including at least a portion of said display screen without any beacons; and
a navigation engine operatively connected to said image sensor to process said frames of image data, said navigation engine being configured to extract different positional information of said display screen in said frames of image data without any beacons captured by said input device as said input device is moved relative to the display screen using only said portion of the display screen that is captured in said frames of image data, the navigation engine being configured to identify at least a portion of an outline of said display screen to extract said different positional information of said display screen, said navigation engine being further configured to determine relative positions of said input device with respect to said display screen using said different positional information of said display screen in said frames of image data to track movements of said input device relative to said display screen.
10. The system of claim 9 wherein said navigation engine is configured to locate a plurality of corners of said display screen in said frames of image data to extract said different positional information of said display screen in said frames of image data and to determine said relative positions of said input device using locations of said corners of said display screen in said frames of image data.
11. The system of claim 10 wherein said navigation engine is configured to calculate an estimated location of a missing corner of said display screen in one or more of said image frames to extract said different positional information of said display screen in said frames of image data.
12. The system of claim 9 wherein said navigation engine is configured to threshold pixel values of said frames of image data to find said display screen in said frames of image data.
13. The system of claim 9 wherein said navigation engine is configured to search for a quadrilateral area having a dominant color in said frames of image data to find said display screen in said frames of image data.
14. The system of claim 9 wherein said navigation engine is configured to search for a quadrilateral area having an image pattern similar to a reference image pattern in said frame of image data to find said display screen in said frames of image data, said reference image pattern being associated with an image being displayed on said display screen.
15. The system of claim 9 wherein said navigation engine is configured to perform scene-based navigation on some of said frames of image data to determine said relative position of said input device.
16. A method for tracking an input device, said method comprising:
electronically capturing at least a portion of a display screen as an imaged display screen in frames of image data without capturing any beacons using an image sensor array of said input device as said input device is moved relative to the display screen;
extracting different positional information of said imaged display screen in said frames of image data without any beacons captured by said input device as said input device is moved relative to the display screen, including searching for a quadrilateral frame having a uniform color in said frames of image data to find said display screen in said frames of image data, said searching including identifying at least a portion of an outline of said display screen to extract said different positional information of said display screen, said quadrilateral frame having an outer quadrilateral outline and an inner quadrilateral outline, said inner quadrilateral outline representing said display screen; and
determining relative positions of said input device with respect to said display screen using said different positional information of said imaged display screen in said frames of image data to track movements of said input device relative to said display screen.