Systems and methods to identify page-turning using a portable device
Systems and methods are described in which a light beam generated by a handheld device is used to point toward a printed page containing visual contents, for example displayed in a book or magazine. The page being pointed at may be identified using computer vison applied to images acquired by a device camera pointed in the same direction as the beam. One or more actions may be performed by the device upon identifying that the user has turned to a new page or page section. Such actions may augment the visual content of a printed book to include audible elements, additional visual components, and/or haptic stimulation enacted by the handheld device. Systems and methods may provide simple and intuitive methods for human-machine interactions that may be particularly well-suited for children and other learners.
1. A device, comprising:
a device body configured to be manipulated by a device user;
electronic circuitry within the device body that includes a device processor;
a device light source affixed to the device body and configured to emit a projected light away from the device body to generate a light reflection on a viewable page viewable by the device user; and
a device camera affixed to the device body comprising a field of view that includes some or all of the light reflection,
wherein the device processor is configured to:
acquire a first camera image via the device camera;
identify a first page layout based on a match of the first camera image to one of two or more predetermined page layouts;
acquire a second camera image via the device camera after acquiring the first camera image;
identify a second page layout based on a match of the second camera image to one of the two or more predetermined page layouts;
determine that the second page layout differs from the first page layout; and
perform an action.
2. The device of claim 1 , wherein the device light source is operatively coupled to the device processor and wherein the action comprises turning the device light source off.
3. The device of claim 2 , wherein the device further comprises a device switch operatively coupled to the device processor and wherein the device processor is further configured to turn the device light source on when a currently acquired switch state of the device switch is determined to differ from a preceding acquired switch state of the device switch.
4. The device of claim 1 , wherein the device further comprises a device speaker operatively coupled to the device processor and wherein the action comprises playing, on the device speaker, one or more of one or more new page prompt sounds, one or more out-of-sequence page prompts, one or more words acquired from the second page layout, and one or more sounds acquired from the second page layout.
5. The device of claim 4 , wherein the device further comprises a device switch operatively coupled to the device processor and wherein the playing on the device speaker is stopped by the device processor when a currently acquired switch state of the device switch is determined to differ from a preceding acquired switch state of the device switch.
6. The device of claim 1 , wherein the device further comprises one or more device indicator lights operatively coupled to the device processor and wherein the action comprises one or more of turning on the one or more device indicator lights for one or more predetermined times, changing an intensity of the one or more device indicator lights, and changing a color of the one or more device indicator lights.
7. A device, comprising:
a device body configured to be manipulated by a device user;
electronic circuitry within the device body that includes a device processor;
a device light source affixed to the device body and configured to emit a projected light away from the device body to generate a light reflection on a viewable page viewable by the device user; and
a device camera affixed to the device body comprising a field of view that includes some or all of the light reflection,
wherein the device processor is configured to:
acquire a first camera image via the device camera;
identify a first page within the first camera image;
acquire a second camera image via the device camera after acquiring the first camera image;
identify a second page within the second camera image;
determine that the second page differs from the first page; and
perform an action.
8. The device of claim 7 , wherein the device is configured to be manipulated by one of a hand of the device user, a head of the device user, a wrist of the device user, an arm of the device user, a shoulder of the device user, a leg of the device user, and a chest of the device user.
9. The device of claim 7 , wherein the device light source is operatively coupled to the device processor and wherein the action comprises turning the device light source off.
10. The device of claim 9 , wherein the device further comprises a device switch operatively coupled to the device processor and wherein the device processor is further configured to turn the device light source on when a currently acquired switch state of the device switch is determined to differ from a preceding acquired switch state of the device switch.
11. The device of claim 9 , wherein the device further comprises a device speaker operatively coupled to the device processor and wherein the action comprises playing, on the device speaker, one or more of one or more new page prompts, one or more out-of-sequence page prompts, one or more textual words identified in the second camera image, and one or more descriptive words related to one or more identified objects in the second camera image.
12. The device of claim 11 , wherein the device further comprises a device switch operatively coupled to the device processor and wherein the playing on the device speaker is stopped by the device processor when a currently acquired switch state of the device switch is determined to differ from a preceding acquired switch state of the device switch.
13. The device of claim 11 , wherein the device light source is turned on upon completion of the playing on the device speaker.
14. The device of claim 7 , wherein the device further comprises a device haptic unit operatively coupled to the device processor and wherein the action comprises activating the device haptic unit for one or more predetermined times.
15. The device of claim 7 , wherein the device further comprises one or more device indicator lights operatively coupled to the device processor and wherein the action comprises one or more of turning on the one or more indicator lights for one or more predetermined times, changing an intensity of the one or more device indicator lights, and changing a color of the one or more device indicator lights.
16. A device, comprising:
a device body configured to be manipulated by a device user;
electronic circuitry within the device body that includes a device processor;
a device switch affixed to the device body;
a device light source affixed to the device body and configured to emit a projected light away from the device body to generate a light reflection on a viewable page viewable by the device user; and
a device camera affixed to the device body comprising a field of view that includes some or all of the light reflection,
wherein the device processor is configured to:
acquire a first camera image via the device camera;
identify a first page within the first camera image;
acquire a first switch state via the device switch;
acquire a second switch state via the device switch after acquiring the first switch state;
determine that the second switch state differs from the first switch state;
acquire a second camera image via the device camera;
identify a second page within the second camera image;
determine that the second page differs from the first page; and
perform an action.
17. The device of claim 16 , wherein the device switch comprises one of a pushbutton switch, a rocker switch, a contact switch and a proximity sensitive switch.
18. The device of claim 16 , wherein the device light source is operatively coupled to the device processor and wherein the action comprises turning the device light source off.
19. The device of claim 16 , wherein the device further comprises a device speaker operatively coupled to the device processor and wherein the action comprises playing, on the device speaker, one or more of one or more new page prompts, one or more out-of-sequence page prompts, one or more textual words identified in the second camera image, and one or more descriptive words related to one or more identified objects in the second camera image.
20. The device of claim 19 , wherein the playing on the device speaker is stopped by the device processor when a currently acquired switch state of the device switch is determined to differ from a preceding acquired switch state of the device switch.