Device, system and method for recognizing inputted data using memory having blackboard data structure
One arrangement concerns a portable device (e.g., a smartphone) that executes plural recognition agents, such as agents that perform fingerprint-based object recognition, fingerprint-based audio recognition, barcode reading, watermark decoding, etc. Each of the agents reads from and writes to a blackboard data structure, to which camera and microphone sensors also post their data. Queues of stored sensor data are thus available for the agents to process. In some arrangements, the agents also post—to the blackboard—estimates of the resource costs required to perform certain functions, and estimates of the quality of results that may be achieved by such functions. This allows the system to make informed decisions about how to deploy the device's limited resources (battery, processing cycles, network bandwidth, etc.). A great variety of other features and arrangements are also detailed.
1. A portable device including a processor, a memory, and plural sensors including at least a camera and a microphone, the aforesaid elements cooperating to define plural recognition agents, said agents being configured to read data from and write data to a blackboard data structure of the memory, the blackboard data structure also being configured to receive data from the sensors and compile queues of stored image data and audio data that said agents process.
2. The portable device of claim 1 in which plural of the recognition agents are each configured to estimate a resource cost required to perform a function, and estimate a quality of result that may be achieved by such function, and to store such estimates in the blackboard structure, the device further including an event controller configured to assess such stored estimates from plural recognition agents and decide which recognition agent(s) to activate, based thereon.
3. The portable device of claim 1 which further includes an event controller and an inference engine, the inference engine being configured to monitor the blackboard data structure and infer contextual data, the event controller being configured to use said contextual data in deciding which recognition agent(s) to activate.
4. The portable device of claim 1 in which the device is configured to process audio and image streams in separate threads, and is configured so that each recognition agent runs in its own thread.
5. The portable device of claim 1 in which the recognition agents comprise at least one of an audio fingerprint recognition agent, a barcode recognition agent and an image watermark recognition agent.
6. The portable device of claim 1 in which the recognition agents comprise at least two of an audio fingerprint recognition agent, a barcode recognition agent and an image watermark recognition agent.
7. The portable device of claim 1 in which the recognition agents comprise an audio fingerprint recognition agent, a barcode recognition agent and an image watermark recognition agent.
8. The portable device of claim 1 in which the recognition agents include an image watermark recognition agent.
9. The portable device of claim 8 wherein the sensors further include an accelerometer, and the device is configured to capture an image, determine a luminance value related thereto, assess acceleration associated with the image, assess saturation of the image and, based thereon, assess an image quality, and thereafter decide whether to invoke a watermark reading operation on the image based on the assessed image quality.