IP Library › Granted Patent US 9,953,461
Granted Patent B2
US 9,953,461 · App. 15/256,791 · Granted Apr 24, 2018

Navigation system applying augmented reality

Inventors: Chung-Hao Huang (Taoyuan, TW); Lun-Hui Lee (Taoyuan, TW); Chian-Cheng Ho (Taipei, TW); Bo-Kai Wang (Tainan, TW)
Assignees: Institute of Nuclear Energy Research Atomic Energy Council, Executive Yuan; National Yunlin University of Science & Technology
G06T19/003G01C21/18G01C21/206G06T19/006G06K9/00691G06T2219/004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,953,461
App. No.
15/256,791
Granted
Apr 24, 2018
Kind
B2
Abstract

A navigation system includes a measurement unit that outputs a positioning signal associated with positioning coordinates, a navigation unit that obtains guidance indication and that outputs a navigation signal, and an image unit that includes a capturing module capturing a series of planar images, a computation module and an augmented reality module. The computation module generates an optical flow signal for obtaining the positioning coordinates, and further outputs a registration signal. The augmented reality module performs image registration processing based on the navigation signal and the registration signal, so that the guidance indication is superimposed on the series of planar images of real-world scene for subsequent display.

Claims (40)

1. A navigation system which applies augmented reality, comprising:

a measurement unit including a positioning module that is configured to output a positioning signal associated with positioning coordinates of said navigation system;

a navigation unit coupled to said measurement unit, and configured to perform a path planning algorithm according to the positioning signal, so as to obtain guidance indication for guidance of a user of said navigation system, said navigation unit being further configured to output a navigation signal associated with the guidance indication; and

an image unit coupled to said measurement unit and said navigation unit, and including

a capturing module that is configured to capture a series of planar images of real-world scene,

a computation module that is configured to perform an optical flow analysis upon the series of planar images so as to generate an optical flow signal associated with a result of performance of the optical flow analysis and used to obtain the positioning coordinates, said computation module being further configured to perform plane detection processing upon each planar image of the series of planar images so as to output a registration signal associated with a result of performance of the plane detection processing, and

an augmented reality module that is configured to receive the navigation signal and the registration signal, and to perform image registration processing based on the navigation signal and the registration signal, so that the guidance indication is superimposed on the series of planar images of real-world scene for subsequent display.

2. The navigation system of claim 1 , wherein to realize the optical flow analysis for generation of the optical flow signal, said computation module is further configured to:

obtain a plurality of analysis features for each planar image of the series of planar images, and recording pixel coordinates for each of the analysis features;

for each planar image, calculate a sum of x-coordinates of the analysis features and a sum of y-coordinates of the analysis features; and

subtract the sum of x-coordinates of the analysis features on a current planar image of the series of planar images from the sum of x-coordinates of the analysis features on a previous planar image of the series of planar images, divide a result of subtraction of the sums of x-coordinates over a number of the analysis features, subtract the sum of y-coordinates of the analysis features on the current planar image of the series of planar images from the sum of y-coordinates of the analysis features on the previous planar image of the series of planar images, and divide a result of subtraction of the sums of y-coordinates over the number of the analysis features, so as to generate the optical flow signal.

3. The navigation system of claim 1 , wherein to realize the plane detection processing for outputting the registration signal, said computation module is further configured to:

convert each planar image into a grayscale image;

perform an algorithm associated with Wiener deconvolution for enhancing edges in the grayscale image;

detect the edges thus enhanced by using a Canny detector;

connect the edges in a same straight line using the Hough transform;

determine if each of the edges is a wall-floor boundary based on a position and a direction in which the edge is located in the grayscale image; and

define a floor plane in the spatial environment based on pairwise wall-floor boundary so as to output the registration signal associated with the floor plane.

4. The navigation system of claim 1 ,

wherein said measurement unit further includes a posture module which is configured to generate a posture signal associated with angles of rotation of said posture module; and

wherein said image unit further includes a display interface, and to realize the image registration processing, said augmented reality module is further configured to

generate, according to the posture signal and the registration signal, a mapping matrix which is associated with a mapping relationship between different coordinate systems, and

perform an algorithm associated with Affine transformation upon the guidance indication by using the mapping matrix thus generated so as to superimpose the guidance indication on the series of planar images of real-world scene for subsequent display on said display interface.

5. The navigation system of claim 1 ,

wherein said measurement unit stores map data associated with the spatial environment;

wherein said computation module is configured to determine a displacement vector associated with displacement of said capturing module according to the optical flow signal;

wherein said positioning module is configured to perform, according to the displacement vector determined by said computation module, position estimation processing with reference to a coordinate system of the map data, so as to output the positioning signal associated with the positioning coordinates of said navigation system located at said coordinate system of the map data.

6. The navigation system of claim 5 , wherein to realize the position estimation processing for outputting the positioning signal, said positioning module is further configured to:

obtain starting coordinates of said navigation system by detecting a positioning landmark in the spatial environment; and

obtain estimated coordinates by mapping the displacement vector to said coordinate system of the map data, and by adding the displacement vector thus mapped to the starting coordinates, so as to output the positioning signal which is associated with the estimated coordinates, the estimated coordinates serving as the positioning coordinates.

7. The navigation system of claim 6 , wherein said positioning module is further configured to:

determine if there exist the estimated coordinates;

when it is determined that there do not exist the estimated coordinates, obtain the starting coordinates by detecting the positioning landmark; and

when it is determined that there exist the estimated coordinates, obtain updated estimated coordinates by adding the displacement vector, which is determined by said computation module according to the optical flow signal, to the estimated coordinates, and by mapping a result of the addition to said coordinate system of the map data, so as to continuously update the estimated coordinates and to output the positioning signal associated with the estimated coordinates.

8. The navigation system of claim 5 , wherein to realize the path planning algorithm for obtaining the guidance indication, said navigation unit is further configured to:

establish an adjacency list based on the map data;

map a current position, which is associated with the positioning coordinates contained in the positioning signal, and a target position to two nodes of the adjacency list, respectively; and

find a path which has a shortest distance or time between the two nodes according to the Dijkstra's algorithm, so as to obtain the guidance indication associated with the path thus found.

9. The navigation system of claim 8 , wherein the guidance indication superimposed on the series of planar images of real-world scene is represented in a manner of at least one of a graphic or a word for guiding a user of said navigation system to move from the current position to the target position along the path thus found.

10. The navigation system of claim 1 , wherein said measurement unit further includes a posture module which generates a posture signal associated with angles of rotation of said posture module, said posture module includes a gyroscope which outputs a posture angle signal associated with each axis of a self-defined coordinate system of said gyroscope, and said posture module, based on the posture angle signal and a gravitational acceleration, outputs the posture signal associated with angles of rotation of said posture module at respective axes of a world coordinate system.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2023
From: INSTITUTE OF NUCLEAR ENERGY RESEARCH ATOMIC ENERGY COUNCIL, EXECUTIVE YUAN
To: NATIONAL YUNLIN UNIVERSITY OF SCIENCE & TECHNOLOGY
Reel/Frame 063994/0628 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INVENTOR CHIAN-CHENG HO WAS INADVERTENTLY LEFT OFF PREVIOUSLY RECORDED ON REEL 041328 FRAME 0423. ASSIGNOR(S) HEREBY CONFIRMS THE INVENTOR CHIAN-CHENG HO IS AN INVENTOR NEEDING TO BE PART OF ASSIGNMENT. Recorded Apr 5, 2017
From: HUANG, CHUNG-HAO; LEE, LUN-HUI; HO, CHIAN-CHENG; WANG, BO-KAI
To: INSTITUTE OF NUCLEAR ENERGY RESEARCH ATOMIC ENERGY COUNCIL, EXECUTIVE YUAN; NATIONAL YUNLIN UNIVERSITY OF SCIENCE & TECHNOLOGY
Reel/Frame 042164/0732 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SPELLING OF THE FIRST ASSIGNEE'S NAME PREVIOUSLY RECORDED ON REEL 039634 FRAME 0278. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 11, 2017
From: HUANG, CHUNG-HAO; LEE, LUN-HUI; WANG, BO-KAI
To: INSTITUTE OF NUCLEAR ENERGY RESEARCH ATOMIC ENERGY COUNCIL, EXECUTIVE YUAN; NATIONAL YUNLIN UNIVERSITY OF SCIENCE & TECHNOLOGY
Reel/Frame 041328/0423 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 040884 FRAME: 0982. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 11, 2017
From: HUANG, CHUNG-HAO; LEE, LUN-HUI; HO, CHIAN-CHENG; WANG, BO-KAI
To: INSTITUTE OF NUCLEAR ENERGY RESEARCH ATOMIC ENERGY COUNCIL, EXECUTIVE YUAN; NATIONAL YUNLIN UNIVERSITY OF SCIENCE & TECHNOLOGY
Reel/Frame 041328/0640 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 039634 FRAME: 0278. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 9, 2016
From: HUANG, CHUNG-HAO; LEE, LUN-HUI; HO, CHIAN-CHENG; WANG, BO-KAI
To: INSTITUTE OF NUCLEAR ENERGY RESEARCH ATOMIC ENERGY COUNCIL, EXECUTIVE YUAN
Reel/Frame 040884/0982 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2016
From: HUANG, CHUNG-HAO; LEE, LUN-HUI; HO, CHIAN-CHENG; WANG, BO-KAI
To: INSTITUTE OF NUCLEAR ENERGY RESEARCH ATOMIC ENERGY COUNCIL; NATIONAL YUNLIN UNIVERSITY OF SCIENCE & TECHNOLOGY
Reel/Frame 039634/0278 →
Priority Claims (1)
TW 104135036 A · Oct 26, 2015 · national
Continuity (1)
Related Publication 20170116783A1 · Apr 27, 2017