Detection and interpretation of visual indicators
Devices, systems and methods are disclosed that detect and interpret the meaning of visual and/or audio indicators of devices, and provide information, solutions to potential malfunctions, and/or automatic ordering of consumable products to users. For example, sequences of visual indicators (such as LEDs) and/or audio indicators (such as speakers that output beeps) generated by home electronics and other devices may be monitored using an image capture component (such as a camera) or audio capture component (such as a microphone), and the sequences may be interpreted to provide users with information of what is wrong with a device, how to fix a device, purchase a new device, offer repair services, etc. The systems and methods may also provide users with notifications relating to such indicators when the user is away from home.
1. A computer-implemented method, comprising:
receiving a device identifier corresponding to an image capture device associated with a user profile;
receiving, from the image capture device, image data representing an object including an illuminated indicator;
determining an object identifier corresponding to the object;
determining, based in part on the object identifier, stored data indicating a location of a known indicator with respect to the object and information associated with the known indicator;
determining, using the image data and the stored data, a meaning of the illuminated indicator based at least in part on a luminescence of a pixel within a bounded region of the image data, the bounded region corresponding to the location;
determining an action associated with the meaning; and
performing the action.
2. The computer-implemented method of claim 1 , wherein determining the action and performing the action includes:
determining that the meaning corresponds to an indication to replace a consumable product associated with the object identifier;
identifying a user address associated with the user profile; and
ordering the consumable product to be shipped to the user address.
3. The computer-implemented method of claim 1 , wherein determining the action and performing the action includes:
determining that the meaning corresponds to an indication to replace a consumable product associated with the object identifier;
identifying a user device associated with the user profile; and
sending options of consumable products to the user device for purchase.
4. The computer-implemented method of claim 1 , wherein determining the action and performing the action includes:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user device associated with the user profile; and
sending instruction corresponding to resolving the malfunction to the user device.
5. The computer-implemented method of claim 1 , wherein determining the action and performing the action includes:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user device associated with the user profile; and
sending options of replacement objects to the user device for purchase.
6. The computer-implemented method of claim 1 , wherein determining the action and performing the action includes:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user device associated with the user profile; and
sending contact information for repair services to the user device.
7. The computer-implemented method of claim 1 , wherein determining the action and performing the action includes:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user address associated with the user profile; and
contacting a repair service to resolve the malfunction.
8. The computer-implemented method of claim 1 , further comprising:
receiving, from the image capture device, audio data corresponding to an audible indicator;
determining an audio fingerprint of the audio data;
comparing the audio fingerprint to a stored audio fingerprint associated with a stored audio sequence associated with the object identifier; and
performing an action associated with the stored audio sequence.
9. The method of claim 1 , wherein the device identifier corresponds to a camera positioned with the object in the camera's field of view.
10. A system, comprising:
at least one processor; and
at least one memory including instructions operable to be executed by the at least one processor to perform a set of actions, configuring the at least one processor to:
receive a device identifier corresponding to an image capture device associated with a user profile;
receive, from the image capture device, image data representing an object including an illuminated indicator;
process the image data to identify the object;
determine an object identifier corresponding to the object;
determine, based in part on the object identifier, stored data indicating a location of a known indicator with respect to the object and information associated with the known indicator;
determine, using the image data and the stored data, a meaning of the illuminated indicator in the image data based at least in part on a luminescence of a pixel within a bounded region corresponding to the location;
determine an action associated with the meaning; and
perform the action.
11. The system of claim 10 , wherein the at least one processor is further configured to determine the action and perform the action by:
determining that the meaning corresponds to an indication to replace a consumable product associated with the object identifier;
identifying a user address associated with the user profile; and
ordering the consumable product to be shipped to the user address.
12. The system of claim 10 , wherein the at least one processor is further configured to determine the action and perform the action by:
determining that the meaning corresponds to an indication to replace a consumable product associated with the object identifier;
identifying a user device associated with the user profile; and
sending options of consumable products to the user device for purchase.
13. The system of claim 10 , wherein the at least one processor is further configured to determine the action and perform the action by:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user device associated with the user profile; and
sending instruction corresponding to resolving the malfunction to the user device.
14. The system of claim 10 , wherein the at least one processor is further configured to determine the action and perform the action by:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user device associated with the user profile; and
sending options of replacement objects to the user device for purchase.
15. The system of claim 10 , wherein the at least one processor is further configured to determine the action and perform the action by:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user device associated with the user profile; and
sending contact information for repair services to the user device.
16. The system of claim 10 , wherein the at least one processor is further configured to determine the action and perform the action by:
determining that the meaning corresponds to an indication of a malfunction of the object;
identifying a user address associated with the user profile; and
contacting a local repair service to resolve the malfunction.
17. The system of claim 10 , wherein the device identifier corresponds to a camera positioned with the object in the camera's field of view.