System and method for interacting with objects via a camera enhanced mobile device
View Patent ↗Embodiments of the present invention enable an image based controller to control and manipulate objects with simple point-and-capture operations via images captured by a camera enhanced mobile device. Powered by this technology, a user is able to complete many complicated control tasks via guided control of objects without utilizing laser pointers, IR transmitters, mini-projectors, or bar code tagging and/or customized wall paper are not needed for the environment control. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.
1. A system to support image based object controlling, comprising:
a camera enhanced mobile device operable to:
point to and capture one or more images of an object of interest; and
send the captured one or more images of the object to a processing center via a network;
a database of object features containing features of a plurality of objects;
said processing center operable to:
extract a plurality of features of the object from the one or more images captured by the camera enhanced mobile device;
identify the object the user points to based on the database of object features; and
manipulate and control the identified object via an interface unit; and
said interface unit operable to interact with both the processing center and the object.
2. The system according to claim 1 , wherein:
the object is one of a visible networked device and a visible un-networked physical object.
3. The system according to claim 2 , wherein:
the visible networked device is one of: a video projector, a side or rear flat panel, a front or back room loudspeaker, a television, and an air conditioner.
4. The system according to claim 2 , wherein:
the visible un-networked physical object is one of a paper document, a white board, a traditional instruments not connected to any network, and an audio signal recorded by a microphone attached to a mobile device.
5. The system according to claim 1 , wherein:
the camera enhanced mobile device is one of a cell phone and a PDA.
6. The system according to claim 1 , wherein:
the network is one of Internet and a wireless communication network.
7. The system according to claim 1 , wherein:
each of the plurality of features of the object is one of: a Scale Invariant Feature Transform (SIFT) feature, a Gabor feature, a Discrete Cosine Transform (DCT) coefficient, an Optical Character Recognition (OCR) results, and data captured by other sensors on the camera enhanced mobile device.
8. The system according to claim 1 , wherein:
the processing center is one of a stand-alone computing unit and an embedded unit in a mobile device or a service center.
9. The system according to claim 1 , wherein:
the interface unit has backward compatibility if the object is a traditional device.
10. The system according to claim 1 , wherein:
The processing center is operable to manipulate the object by sending a command to the object if the object is on the same network or transforming the command into Infra-Red (IR) signals to control the object.
11. A method to support image based object controlling, comprising:
establishing a database containing features of a plurality of objects;
capturing one or more images of an object via a camera enhanced mobile device and sending the one or more images to a processing center via a network;
extracting a plurality of features of the object from the one or more images captured by the camera enhanced mobile device;
identifying the object the user points to based on the database of object features; and
manipulating and controlling the identified object via commands.
12. The method according to claim 11 , further comprising:
defining a plurality of operation rules between the one or more images of the object and its manipulation actions.
13. The method according to claim 11 , further comprising:
interacting with the object via an interface unit if the object is a traditional device.
14. The method according to claim 11 , further comprising:
manipulating the object by sending a command to the object if the object is on the same network or transforming the command into Infra-Red (IR) signals to control the object.
15. The method according to claim 11 , further comprising:
identifying the object in the one or more images via Scale Invariant Feature Transform (SIFT).
16. A non-transitory computer readable storage medium having instructions stored thereon that when executed cause a system to:
establish a database containing features of a plurality of objects;
capture one or more images of an object via a camera enhanced mobile device and send the one or more images to a processing center via a network;
extract a plurality of features of the object from the one or more images captured by the camera enhanced mobile device;
identify the object the user points to based on the database of object features; and
manipulate and control the identified object via commands.
17. A system to support image based object controlling, comprising:
means for establishing a database containing features of a plurality of objects;
means for capturing one or more images of an object via a camera enhanced mobile device and sending the one or more images to a processing center via a network;
means for extracting a plurality of features of the object from the one or more images captured by the camera enhanced mobile device;
means for identifying the object the user points to based on the database of object features; and
means for manipulating and controlling the identified object via commands.