IP Library Granted Patent US 12,125,407
Granted Patent B2
US 12,125,407 · App. 18/382,456 · Granted Oct 22, 2024

Systems and methods to specify interactive page locations by pointing a light beam using a handheld device

Inventors: Lewis James Marggraff (Lafayette, CA); Nelson George Publicover (Bellingham, WA)
Assignee: KIBEAM LEARNING, INC.
G09B5/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,125,407
App. No.
18/382,456
Granted
Oct 22, 2024
Kind
B2
Abstract

Systems and methods are described in which a light beam generated by a handheld device is used to point toward a selected object or location within a printed page containing visual content. The object(s) and/or location being pointed at may be determined using a device camera pointed in the same direction as the beam coupled with computer vision methods to determine locations of camera-acquired images within a database of page-content templates. The database may include interactive sequences and/or scenarios associated with page objects and locations, which may augment the visual content of a printed book to include audible elements, additional visual components, and/or haptic stimulation enacted by the handheld device. The device may also generate a beam image reflection on a viewable surface that is in focus at a predetermined distance to encourage a user to position the handheld device at the predetermined distance from the viewable surface.

Claims (35)

1. A handheld device, comprising:

a device body configured to be manipulated by a hand of a device user;

electronic circuitry within the device body that includes a device processor;

a device light beam source configured to emanate a light beam away from the device body to generate a beam image reflection creating an image on a viewable surface, the image in focus at a predetermined distance from the viewable surface; and

a device camera operatively coupled to the device processor and aligned such that a field-of-view of the device camera includes the beam image reflection, wherein the handheld device is configured to:

generate the beam image reflection on the viewable surface such that the image is viewable by the device user and is in focus at the predetermined distance to encourage the device user to position the handheld device at the predetermined distance from the viewable surface; and

acquire a camera image.

2. The handheld device of claim 1 , wherein the device light beam source is one of one or more light-emitting diodes, one or more micro-light emitting diodes, one or more lasing diodes, and one or more digital light processing projector elements, each operatively coupled to the device processor.

3. The handheld device of claim 1 , wherein the beam image reflection appears to the device user as one of a smiley face, a human face, an animal face, a toy, a cartoon character, a circle, a rectangle, a polygon, an arrow, a cross, a finger, a hand, one or more characters, one or more numbers, one or more symbols, one or more recognizable shapes, and an animation.

4. The handheld device of claim 1 , wherein a structured light pattern producing the beam image reflection is generated by one of one or more light blocking filters within a light path of the device light beam source; liquid crystal filter elements within the light path of the light beam source, each operatively coupled to the device processor; digital light processing projector elements, each operatively coupled to the device processor; and a plurality of light beam sources, each operatively coupled to the device processor.

5. The handheld device of claim 4 , wherein the device processor changes the structured light pattern based, at least in part, on content determined by the device processor within the camera image.

6. The handheld device of claim 1 , wherein the light beam source is operatively coupled to the device processor and further configured to control a light beam intensity by one or both of regulating a light beam driving current, and a pulse width modulation of the light beam driving current.

7. The handheld device of claim 6 , wherein the device processor is further configured to:

identify the image in the camera image; and

control the light beam intensity based on a detected light intensity measured in the camera image within one or both of a beam image reflection region and an ambient light image region not containing the image.

8. The handheld device of claim 1 , wherein the image comprises one of an object, symbol, or pattern on the viewable surface.

9. A method to encourage a device user to position a handheld device a predetermined distance from a viewable surface, wherein the handheld device comprises a device body configured to be manipulated by a hand of the device user; electronic circuitry within the device body that includes a device processor; a device light beam source configured to emit a light beam to generate a beam image reflection creating an image that is in focus at the predetermined distance; and a device camera operatively coupled to the device processor and aligned such that the device camera field-of-view includes the beam image reflection, the method comprising:

generating, by the device light beam source, the beam image reflection on the viewable surface such that the image is viewable by the device user and is in focus at the predetermined distance to encourage the device user to position the handheld device at the predetermined distance from the viewable surface; and

acquiring, by the device camera, a camera image.

10. The method of claim 9 , further comprising:

identifying, by the device processor, the image in the camera image; and

determining, by the device processor, a beam state of the beam image reflection within the camera image as one of in focus and out of focus, and performing an action by the device processor based, at least in part, on the beam state.

11. The method of claim 10 , wherein the determining the beam state includes computing, by the device processor applied to the camera image, one or more of a trained neural network, one or more kernel operations, one or more image contrast measurements, and a Fourier transform.

12. The method of claim 10 , wherein the light beam source is operatively coupled to the device processor, and the action comprises changing one or more of a beam intensity, a beam color, and a beam structured light pattern.

13. The method of claim 10 , wherein a haptic unit within the handheld device is operatively coupled to the device processor, and the action comprises activating the haptic unit.

14. The method of claim 10 , wherein a device speaker within the handheld device is operatively coupled to the device processor, and the action comprises playing an indicating sound on the device speaker.

15. The method of claim 9 , wherein the viewable surface is constructed from one or more of paper, cardboard, film, cloth, wood, plastic, glass, a painted surface, a printed surface, a textured surface, an enhanced surface enhanced with three-dimensional elements, a flexible surface, and an electronic display.

16. The method of claim 9 , wherein the viewable surface is a component of one or more of a book, a book cover, a brochure, a box, a sign, a newspaper, a magazine, a tablet, a printable surface, a tattoo, a mobile device, an electronic book reader, and a display screen.

17. The method of claim 9 , wherein the image comprises one of an object, symbol, or pattern that is viewable on the viewable surface.

18. A method to encourage a device user to position a handheld device a predetermined distance from a viewable surface, wherein the handheld device comprises a device body configured to be manipulated by a hand of the device user; electronic circuitry within the device body that includes a device processor; a device light beam source configured to emit a light beam to generate a beam image reflection creating an image that is viewable on the viewable surface; one or more moveable reflective surfaces within a device beam light path originating from the device light beam source, each operatively coupled to the device processor; and a device camera operatively coupled to the device processor and aligned such that the device camera field-of-view includes the beam image reflection, the method comprising:

generating, by the device light beam source, the image on the viewable surface;

moving, by the device processor, the one or more moveable reflective surfaces to make the beam image reflection of the image in focus at the predetermined distance to encourage the device user to position the handheld device at the predetermined distance from the viewable surface; and

acquiring, by the device camera, a camera image.

19. The method of claim 18 , wherein the one or more moveable reflective surfaces each comprises one or more moveable mirrors, one or more micro-electromechanical systems, and one or more moveable prisms.

20. The method of claim 18 , wherein the moving the one or more moveable reflective surfaces changes one or both of a light beam path distance and a light path direction travelled by a device light beam.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2023
From: PUBLICOVER, NELSON GEORGE; MARGGRAFF, LEWIS JAMES
To: KIBEAM LEARNING, INC.
Reel/Frame 065740/0876 →
Continuity (3)
Continuation In Part 18220738 · Jul 11, 2023
Provisional Application 63441731 · Jan 27, 2023
Related Publication 20240257658A1 · Aug 1, 2024
Cited By (2)
US 12,282,613 US 12,393,287