IP Library Granted Patent US 11,726,561
Granted Patent B2
US 11,726,561 · App. 17/278,350 · Granted Aug 15, 2023

Enhanced autonomous hands-free control in electronic visual aids

Inventors: David Watola (Irvine, CA); Jay E. Cormier (Laguna Niguel, CA); Brian Kim (San Clemente, CA)
Assignee: Eyedaptic, Inc.
G06F3/013A61F9/08G02B27/0101G06V20/10G09B21/008G02B2027/014G02B2027/0138G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,726,561
App. No.
17/278,350
Granted
Aug 15, 2023
Kind
B2
Abstract

Adaptive Control Systems and methods are provided for an electronic visual aid, which may be wearable, hand held or fixed mount, including autonomous hands free control integrated with artificial intelligence tunable to user preferences and environmental conditions.

Claims (54)

1. A method of providing autonomous enhancements to a video output of a visual aid device worn by a user, comprising:

obtaining real-time images of a scene with a camera of the visual aid device;

evaluating a motion component of the visual aid device to determine if the user of the visual aid device is engaged in a concentration state associated with very small angular velocities of the visual aid device maintained over a perceptible interval, a scanning movement state associated with intermediate angular velocities of the visual aid device, or an animated movement state associated with high angular velocities of the visual aid device;

if the user of the visual aid device is engaged in the animated movement state:

presenting first video images on a display of the visual aid device without additional enhancements or adjustments applied to the real-time images;

if the user of the visual aid device is engaged in a scanning movement state:

locating first structures of interest from the real-time images;

applying first visual enhancements to at least the first structures of interest; and

presenting second video images including the first visually enhanced first structures of interest on the display of the visual aid device;

if the user of the visual aid device is engaged in the concentration state:

locating second structures of interest from the real-time video images;

applying second visual enhancements to at least the second structures of interest; and

presenting third video images including the second visually enhanced second structures of interest on the display of the visual aid device.

2. The method of claim 1 , wherein the motion component comprises an angular velocity of the visual aid device.

3. The method of claim 1 , wherein the animated movement state indicates that the user of the visual aid device is not engaged in deliberate concentration or attempting to focus on something within a field of view of the user.

4. The method of claim 1 , wherein the concentration state indicates that the user of the visual aid device is deliberately concentrating on something within the field of view.

5. The method of claim 1 , wherein the scanning state indicates that the user of the visual aid device is reading.

6. The method of claim 1 , wherein the first or second structures of interest comprise text-like structures.

7. The method of claim 1 , wherein the first or second structures of interest comprise facial features.

8. The method of claim 1 , wherein the first or second visual enhancements comprise adjusting a magnification of at least the first or second structures of interest.

9. The method of claim 1 , wherein the first or second visual enhancements comprise adjusting a contrast component of at least the first or second structures of interest.

10. The method of claim 1 , wherein determining a motion state of the visual aid device comprises:

evaluating the motion component associated with recent frames of the real-time images over a first interval of approximately 50-250 milliseconds to categorize short-term movement of the user of the visual aid device; and

evaluating the motion component of recent frames of the real-time images over a second interval longer than the first interval to identify a specific action state of the user of the visual aid device.

11. The method of claim 1 , wherein obtaining real-time images of the scene with the visual aid device further comprises obtaining the real-time images with a camera of the visual aid device that is not rigidly fixed with respect to the user's eyes.

12. A method of providing enhanced vision for a low-vision user, comprising the steps of:

capturing real-time video images with a camera of a first electronic device;

transmitting the real-time video images from the first electronic device to a visual aid eyeglasses device separate from the first electronic device;

identifying a subset of each of the real-time video images corresponding to a focus of attention of the camera;

extracting structures of interest within the subset of each of the real-time video images;

identifying at least one group of structures of interest that are organized in a row or column;

determining a dimension of the at least one group of structures of interest;

adjusting a magnification of the at least one group of structures of interest to match a preferred structure dimension;

forming an enhanced video stream that includes the real-time images and the at least one group of structures of interest with adjusted magnification; and

displaying the enhanced video stream on a display of the visual aid eyeglasses device.

13. The method of claim 12 , wherein the subset comprises a fixed area of pixels within each of the real-time video images.

14. The method of claim 12 , wherein the structures of interest comprise text-like structures.

15. The method of claim 12 , wherein the structures of interest comprise facial structures.

16. The method of claim 12 , wherein the structures of interest are selected from the group consisting of letters, numerals, pictograms, glyphs and icons.

17. The method of claim 12 , wherein the extracting step further comprises extracting maximally stable extremal regions from the subset of real-time video images.

18. A method of providing enhanced vision for a low-vision user, comprising the steps of:

capturing real-time video images with a camera of an electronic device;

transmitting the real-time video images from the electronic device to a visual aid eyeglasses device worn by the low-vision user and separate from the electronic device;

identifying a subset of each of the real-time video images corresponding to a focus of attention of the camera;

extracting structures of interest within the subset of each of the real-time video images;

applying visual enhancements to the structures of interest; and

displaying video images including the visually enhanced structures of interest on one or more displays of the visual aid eyeglasses device.

19. The method of claim 18 , wherein the subset comprises a fixed area of pixels within each of the real-time video images.

20. The method of claim 18 , wherein the structures of interest comprise text-like structures.

21. The method of claim 18 , wherein the structures of interest comprise facial structures.

22. The method of claim 18 , wherein the structures of interest are selected from the group consisting of letters, numerals, pictograms, glyphs and icons.

23. The method of claim 18 , wherein the electronic device comprises a smartphone.

24. The method of claim 18 , wherein applying visual enhancements to the structures of interest further comprises adjusting a magnification of the structures of interest.

25. The method of claim 18 , wherein applying visual enhancements to the structures of interest further comprises adjusting a contrast of the structures of interest.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 10, 2023
From: WATOLA, DAVID; CORMIER, JAY E.; KIM, BRIAN
To: EYEDAPTIC, INC.
Reel/Frame 063273/0967 →
Continuity (2)
Provisional Application 62735643 · Sep 24, 2018
Related Publication 20210373656A1 · Dec 2, 2021
Cited By (3)
US 12,282,169 US 12,416,062 US 12,693,528