IP Library Granted Patent US 9,619,688
Granted Patent B2
US 9,619,688 · App. 14/048,215 · Granted Apr 11, 2017

Navigation using portable reading machine

Inventor: Rafael Maya Zetune (Chestnut Hill, MA)
Assignee: KNFB READER, LLC
G06K9/00G01C21/00G01C21/20G01C21/206G06K9/00664G06K9/469G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,619,688
App. No.
14/048,215
Granted
Apr 11, 2017
Kind
B2
Abstract

Navigation techniques including map based and object recognition based and especially adapted for use in a portable reading machine are described.

Claims (59)

1. A device comprising:

one or more processing devices;

memory to store instructions that are executable by the one or more processing devices to:

capture images of physical scenes, at least some of the images containing at least one or more of objects, landmarks, and/or signs in the images of the physical scenes;

process the captured images to recognize existence of the at least one or more objects, landmarks, and/or signs in the images of the physical scenes;

associate the recognized objects, landmarks and/or signs in the images of the physical scenes to nodes in a data structure, which correspond to the objects, landmarks, and/or signs;

associate, with the nodes, navigation descriptions of the corresponding one or more of the objects, landmarks, and/or signs; and

generate a construct relating the nodes to each another, the nodes being related based, at least in part, on the navigation descriptions.

2. The device of claim 1 , wherein to generate a construct comprises generating an edge between the nodes representing relationships between the corresponding objects.

3. The device of claim 1 , wherein to generate a construct comprises generating an edge between the nodes containing information about connectedness of the corresponding objects.

4. The device of claim 1 , wherein to generate a construct comprises generating an edge between the nodes containing information about proximity of the corresponding objects.

5. The device of claim 1 , wherein the nodes and the construct relating objects to each another are incorporated in a map for use in navigation, wherein the map is selected from the group consisting of a un-directed and a directed map.

6. The device of claim 1 , wherein the processor is further configured to cause navigation instructions to be conveyed to a user of the device, the navigation instructions based at least in part on navigation descriptions associated with the nodes.

7. The device of claim 1 , wherein the objects, landmarks, and/or signs comprise first and second sets with the first set of objects incorporated in a first layer of a map and the second set of objects incorporated in a second layer of the map.

8. The device of claim 1 , wherein the construct comprises a list that associates at least one node to one or more other nodes to which the at least one node is connected.

9. The device of claim 1 , wherein the construct comprises a matrix that associates at least one node to one or more other nodes to which the at least one node is connected with the matrix storing the navigation descriptions including parameters corresponding to distances between nodes.

10. The device of claim 1 , further comprising instructions that are executable by the one or more processing devices to:

perform a process to identify the route among the objects, the process identifying a path among the objects, the path having a predefined characteristic.

11. The device of claim 1 , wherein the process comprises a shortest path algorithm, and wherein the predefined characteristic comprises a length of the path.

12. The device of claim 1 , wherein the parameters relate to objects that are encountered along the route.

13. The device of claim 1 , further comprising providing a user navigation instructions to reach an object in the first layer of the map and providing to the user information about objects in the second layer of the map and each corresponding to a node having connectedness to a node corresponding to the object in the first layer.

14. The device of claim 1 , further comprising upon receiving starting and ending positions, processing information about the nodes and construct to select correct paths or sub paths between the starting and ending positions.

15. A computer program product tangibly stored on a computer readable hardware storage device, the computer program product for navigation of a physical environment, comprises instructions for causing a computer device to:

capture images of physical scenes, at least some of the images containing at least one or more of objects, landmarks, and/or signs in the images of the physical scenes;

process the captured images to recognize existence of the at least one or more objects, landmarks, and/or signs in the physical scenes;

associate the recognized objects, landmarks and/or signs in the images to nodes, the nodes corresponding to the objects, landmarks, and/or signs;

associate, with the nodes, navigation descriptions of the corresponding one or more of the objects, landmarks, and/or signs; and

generate a construct relating the nodes to each another, the nodes being related based, at least in part, on the navigation descriptions.

16. A computer implement method for navigation of a physical environment, comprises:

capturing images of physical scenes using an image input device, at least some of the images of the physical scenes containing at least one or more of objects, landmarks, and/or signs;

processing by a computing device the captured images to recognize existence of the at least one or more objects, landmarks, and/or signs in the physical scenes;

associating by the computing device the recognized objects, landmarks, and/or signs contained in the images to nodes that correspond to the objects, landmarks, and/or signs;

associating by the computing device navigation descriptions of the corresponding one or more of objects, landmarks, and/or signs; and

generating by the computing device a construct relating the nodes to each another, the nodes being related based, at least in part, on the navigation descriptions.

17. The computer program product of claim 15 , wherein to generate a construct comprises instructions to generate an edge between the nodes representing relationships between the corresponding objects.

18. The computer program product of claim 15 , wherein to generate a construct comprises instructions to generate an edge between the nodes containing information about connectedness of the corresponding objects.

19. The computer program product of claim 15 , wherein to generate a construct comprises instructions to generate an edge between the nodes containing information about proximity of the corresponding objects.

20. The computer program product of claim 15 , further comprising instructions to cause the navigation instructions to be conveyed to a user, the navigation instructions based at least in part on navigation descriptions associated with the nodes.

21. The method of claim 16 , wherein generating a construct comprises generating an edge between the nodes representing relationships between the corresponding objects.

22. The method of claim 16 , wherein generating a construct comprises generating an edge between the nodes containing information about connectedness of the corresponding objects.

23. The method of claim 16 , wherein generating a construct comprises generating an edge between the nodes containing information about proximity of the corresponding objects.

24. The method of claim 16 , further comprising conveying navigation instructions to a user, the navigation instructions based at least in part on navigation descriptions associated with the nodes.

25. A device comprising:

one or more processing devices;

memory to store instructions that are executable by the one or more processing devices to:

capture images of physical scenes associated with a user-planned path through the physical scene, at least some of the images containing at least one or more of objects, landmarks, and/or signs in the physical scenes;

process the captured images to recognize existence of the at least one or more objects, landmarks, and/or signs in the images of the physical scenes;

load a map associated with the physical scenes, which includes associated recognized objects that exist in the physical scenes;

load the user-planned path through the physical scene;

determine based on the captured images of the physical scene and recognized objects in the physical scenes, a present location of the user along the planned path;

generate navigation description instructions; and

render the navigation description instructions to the user.

26. The device of claim 25 , wherein to load a map comprises accessing a generated construct nodes representing relationships between adjacent objects.

27. The device of claim 26 , wherein the nodes and the construct relating objects to each another are incorporated in the map for use in the navigation, wherein the map is selected from the group consisting of a un-directed and a directed map.

28. The device of claim 27 , wherein the processor is further configured to cause navigation instructions to be conveyed to a user of the device, the navigation instructions based at least in part on navigation descriptions associated with the nodes.

29. The device of claim 27 , wherein the processor is further configured to provide user navigation instructions to reach an object in a first layer of the map and provide to the user information about objects in a second layer of the map and each corresponding to a node having connectedness to a node corresponding to the object in the first layer.

30. The device of claim 25 , wherein the processor is further configured to:

receive starting and ending positions, and

process information about the nodes and constructs to select correct paths or sub paths between the starting and ending positions.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2022
From: KNFB READER, LLC
To: THE NATIONAL FEDERATION OF THE BLIND
Reel/Frame 058577/0914 →
RELEASE AND TERMINATION OF LIENS Recorded Jan 27, 2016
From: FISH & RICHARDSON P.C.
To: KNFB READER, LLC
Reel/Frame 037603/0342 →
LICENSE Recorded May 1, 2015
From: KNFB READER, LLC
To: DIMENSIONAL STACK ASSETS LLC
Reel/Frame 035546/0143 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2015
From: K-NFB READING TECHNOLOGY, INC.
To: KNFB READER, LLC
Reel/Frame 035522/0289 →
LIEN Recorded Dec 30, 2014
From: K-NFB HOLDING TECHNOLOGY, IMC.
To: FISH & RICHARDSON P.C.
Reel/Frame 034599/0860 →
Continuity (3)
Continuation 12433846 · Apr 30, 2009
Provisional Application 61049311 · Apr 30, 2008
Related Publication 20140037149A1 · Feb 6, 2014