Context-based smartphone sensor logic
Methods employ sensors in portable devices (e.g., smartphones) both to sense content information (e.g., audio and imagery) and context information. Device processing is desirably dependent on both. For example, some embodiments activate certain processor intensive operations (e.g., content recognition) based on classification of sensed content and context. The context can control the location where information produced from such operations is stored, or control an alert signal indicating, e.g., that sensed speech is being transcribed. Some arrangements post sensor data collected by one device to a cloud repository, for access and processing by other devices. Multiple devices can collaborate in collecting and processing data, to exploit advantages each may have (e.g., in location, processing ability, social network resources, etc.). A great many other features and arrangements are also detailed.
1. A portable user device comprising a processor, a memory and plural sensors, including at least a microphone or camera, the memory containing software instructions configuring the device to perform acts including:
(a) applying a classification procedure to received audio and/or visual information, sensed by the microphone and/or camera, to determine a type of said information from among plural possible types;
(b) determining a scenario type by processing context information other than said received audio and/or visual information, said processed other context information including one or more factors from the list: time of day, day of week, location, calendar data, clock alarm status, motion sensor data, orientation sensor data, and information from a social networking service; and
(c) based on a combination of both (i) the determined type of the received audio and/or visual information, and (ii) the determined type of scenario, selecting a group of one or more recognition technologies from a set of plural available recognition technologies, and applying the selected group of one or more recognition technologies to the received audio and/or visual information;
wherein configuration of said device by said software instructions enables the device to select and apply three or more different groups of recognition technologies to the received audio and/or visual information, in accordance with which of three or more different combinations of (i) information type, and (ii) scenario type, are respectively determined;
configuration of said device by said software instructions enables the device to include two or more recognition technologies in one of said selected and applied groups of recognition technologies; and
configuration of said device by said software instructions enables the device to include, in said two or more recognition technologies, at least one of a watermark-, fingerprint-, barcode-based, or optical character-recognition technology.
2. The device of claim 1 , configured by the software instructions to apply said classification procedure to audio information and not visual information, wherein said classification procedure determines a type of audio information from among plural possible types, including at least two from the list: speech, music and silence.
3. The device of claim 1 , configured by the software instructions to apply said classification procedure to visual information and not audio information.
4. The device of claim 1 , configured by the software instructions to determine a scenario type based at least on time of day.
5. The device of claim 1 , configured by the software instructions to determine a scenario type based at least on day of week.
6. The device of claim 1 , configured by the software instructions to determine a scenario type based at least on location.
7. The device of claim 1 , configured by the software instructions to determine a scenario type based at least on calendar data.
8. The device of claim 1 , configured by the software instructions to determine a scenario type based at least on motion sensor data.
9. The device of claim 1 , configured by the software instructions to determine a scenario type based at least on orientation sensor data.
10. The device of claim 1 , configured by the software instructions to determine a scenario type based at least on information from a social networking service.
11. The device of claim 1 , in which configuration of said device by said software instructions enables the device to select and apply a group of one or more recognition technologies including a watermark recognition technology, based on said combination of the determined type of received audio and/or visual information, and the determined type of scenario.
12. The device of claim 1 , in which configuration of said device by said software instructions enables the device to select and apply a group of one or more recognition technologies including a fingerprint recognition technology, based on said combination of the determined type of received audio and/or visual information, and the determined type of scenario.
13. The device of claim 1 , in which configuration of said device by said software instructions enables the device to select and apply a group of one or more recognition technologies including a barcode-based recognition technology, based on said combination of the determined type of received audio and/or visual information, and the determined type of scenario.
14. The device of claim 1 , in which configuration of said device by said software instructions enables the device to select and apply a group of one or more recognition technologies including an optical character recognition technology, based on said combination of the determined type of received audio and/or visual information, and the determined type of scenario.
15. A non-transitory computer readable medium containing software instructions operative to configure a device, including a processor, memory and plural sensors, including at least a microphone or camera, to perform acts including:
(a) applying a classification procedure to received audio and/or visual Information, sensed by the microphone and/or camera, to determine a type of said information from among plural possible types;
(b) determining a scenario type by processing context information other than said received audio and/or visual information, said processed other context information including, one or more factors from the list: time of day, day of week, location, calendar data, clock alarm status, motion sensor data, orientation sensor data, and information from a social networking service; and
(c) based on a combination of both (i) the determined type of the received audio and/or visual information, and (ii) the determined type of scenario, selecting a group of one or more recognition technologies from a set of plural available recognition technologies, and applying the selected group of one or more recognition technologies to the received audio and/or visual information;
wherein:
said software instructions enable a device configured thereby to select and apply three or more different groups of recognition technologies to the received audio and/or visual information, in accordance with which of three or more different combinations of (i) information type, and (ii) scenario type, are respectively determined;
said software instructions enable a device configured thereby to include two or more recognition technologies in one of said selected and applied groups of recognition technologies; and
said software instructions enable a device configured thereby to include, in said two or more recognition technologies, at least one of a watermark-, fingerprint-, barcode based-, or optical character-recognition technology.
16. A method of operating a portable user device that includes one or more sensors, including at least a microphone or camera, the method comprising the acts:
(a) applying a classification procedure to received audio and/or visual information, sensed by the microphone and/or camera, to determine a type of said information from among plural possible types;
(b) determining a scenario type by processing context information other than said received audio and/or visual information, said processed other context information including one or more factors from the list: time of day, day of week, location, calendar data, clock alarm status, motion sensor data, orientation sensor data, and information from a social networking service; and
(c) based on a combination of both (i) the determined type of the received audio and/or visual information, and (ii) the determined type of scenario, selecting a group of one or more recognition technologies from a set of plural available recognition technologies, and applying the selected group of one or more recognition technologies to the received audio and/or visual information;
wherein, in accordance with which of first, second and third different combinations of (i) information type, and (ii) scenario type, are respectively determined at first, second and third different times, the method includes (A) selecting and applying a first group of recognition technologies to the received audio and/or visual information at said first time; (B) selecting and applying a second group of recognition technologies to the received audio and/or visual information at said second time; and (C) selecting and applying a third group of recognition technologies to the received audio and/or visual information at said third time, where said first, second and third groups of recognition technologies are different;
one of said selected and applied groups of recognition technologies includes two or more recognition technologies, at least one of which comprises a watermark-, fingerprint-, barcode-based, or optical character-recognition technology.
17. The method of claim 16 in which act (a) comprises applying the classification procedure to received visual information sensed by the camera, and act (c) comprises:
based on a first combination of the determined type of received visual information and the determined type of scenario, selecting and applying a barcode recognition agent but not an OCR recognition agent; and
based on a second combination of the determined type of received visual information and the determined type of scenario, selecting and applying an OCR recognition agent but not a barcode recognition agent.
18. The method of claim 16 in which said classification procedure is a procedure based on supervised machine learning.
19. The method of claim 16 in which said classification procedure is a procedure based on clustering.
20. The method of claim 16 in which act (a) comprises applying the classification procedure to received audio information sensed by the microphone, and said classification procedure determines a type of the audio information as speech at one time, as music at another time, and as silence at still another time.