IP Library Granted Patent US 10,217,021
Granted Patent B2
US 10,217,021 · App. 15/540,369 · Granted Feb 26, 2019

Method for determining the position of a portable device

Inventors: Kimmo Roimela (Tampere, FI); Lixin Fan (Tampere, FI)
Assignee: Nokia Technologies Oy
G06K9/4671G01C21/3679G06K9/00664G06T7/001G06T7/20G06T7/73G06T7/74G06T19/00G06T2207/10016G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,217,021
App. No.
15/540,369
Granted
Feb 26, 2019
Kind
B2
Abstract

An augmented reality method for displaying information comprises: capturing a sequence of image frames by a camera of a portable device, obtaining movement data by using an inertial sensor of the portable device, determining a sampling frame rate according to the movement data, selecting a tracking image from the captured image frames according to the sampling frame rate, determining a position of the portable device by comparing the tracking image with a point cloud, and displaying information on a display of the device based on the determined position.

Claims (57)

1. A method, comprising:

obtaining a sequence of captured image frames from a camera of a portable device;

obtaining movement data by using an inertial sensor of the portable device;

determining a sampling frame rate according to the movement data;

selecting a tracking image from the captured image frames according to the sampling frame rate;

determining a position of the portable device by comparing the tracking image with a point cloud;

determining a blurring value for a point of the point cloud based on a position of said point and based on the movement data;

comparing the blurring value with a predetermined limit;

selecting said point to a subset of the point cloud or rejecting said point from said subset of the point cloud based on the comparison;

determining the position of the portable device by comparing the tracking image with the subset of the point cloud; and

displaying information on a display of the portable device based on the determined position.

2. The method according to claim 1 , wherein the sequence is captured at a first frame rate, and the sampling frame rate is equal to the first frame rate divided by an integer.

3. The method according to claim 2 , wherein said integer is greater than or equal to two when an angular velocity of the portable device is lower than a first predetermined limit and an acceleration of the portable device is lower than a second predetermined limit.

4. The method according to claim 1 , wherein the sampling frame rate is equal to a first frame rate when an angular velocity of the portable device is equal to or higher than a third predetermined limit and/or an acceleration of the portable device is equal to or higher than a fourth predetermined limit.

5. The method according to claim 1 further comprising:

determining a change of position of the portable device by using the movement data; and

determining a corrected position of the portable device by using said change and a previous determined position of the portable device.

6. The method according to claim 1 further comprising:

determining an indicator value indicative of a number of matching features appearing in an image frame; and

increasing the sampling frame rate when the number of the matching features is lower than a predetermined limit.

7. A portable device, comprising:

a display;

a camera caused to capture a sequence of image frames;

an intertial sensor caused to provide movement data; and

a processor caused to:

determine a sampling frame rate according to the movement data, to determine a position of the portable device by comparing the tracking image with a a point cloud;

determine a blurring value for a point of the point cloud based on a position of said point and based on the movement data;

compare the blurring value with a predetermined limit;

select said point to a subset of the point cloud or reject said point from said subset of the point cloud based on the comparison;

determine the position of the portable device by comparing the tracking image with the subset of the point cloud; and

control displaying information on the display based on the determined position.

8. The portable device according to claim 7 , wherein the camera is caused to capture the sequence of image frames at a first frame rate, and the sampling frame rate is equal to the first frame rate divided by an integer.

9. The portable device according to claim 8 , wherein the processor is caused to determine the sampling frame rate such that said integer is greater than or equal to two when an angular velocity of the portable device is lower than a first predetermined limit and an acceleration of the portable device is lower than a second predetermined limit.

10. The portable device according to claim 7 , wherein the camera is caused to capture the sequence of image frames at a first frame rate, and the processor is configured to determine the sampling frame rate such that the sampling frame rate is equal to the first frame rate when an angular velocity of the portable device is equal to or higher than a third predetermined limit and/or an acceleration of the portable device is equal to or higher than a fourth predetermined limit.

11. The portable device according to claim 7 , wherein the processor is further caused to:

determine a change of position of the portable device by using the movement data; and

determine a corrected position of the portable device by using said change and a previous determined position of the portable device.

12. The portable device according to claim 7 , wherein the processor is further caused to:

determine an indicator value indicative of the number of matching features appearing in an an image frame; and

increase the sampling frame rate when a number of the matching features is lower than a predetermined limit.

13. A computer program product embodied on a non-transitory computer readable medium, comprising computer program code configured to, when executed on at least one processor, cause an apparatus or a system to:

obtain a sequence of captured image frames from a camera of a portable device;

obtain movement data from an inertial sensor of the portable device;

determine a sampling frame rate according to the movement data;

select a tracking image from the captured image frames according to the sampling frame rate;

determine a position of the portable device by comparing the tracking image with a point cloud;

determine a blurring value for a point of the point cloud based on a position of said point and based on the movement data;

compare the blurring value with a predetermined limit;

select said point to a subset of the point cloud or reject said point from said subset of the point cloud based on the comparison;

determine the position of the portable device by comparing the tracking image with the subset of the point cloud; and

display information on a display of the portable device based on the determined position.

14. The computer program product according to claim 13 , wherein the camera is caused to capture the sequence of image frames at a first frame rate, and the sampling frame rate is equal to the first frame rate divided by an integer.

15. The computer program product according to claim 14 , wherein the processor is caused to determine the sampling frame rate such that said integer is greater than or equal to two when an angular velocity of the portable device is lower than a first predetermined limit and an acceleration of the portable device is lower than a second predetermined limit.

16. The computer program product according to claim 13 , wherein the camera is caused to capture the sequence of image frames at a first frame rate, and the processor is configured to determine the sampling frame rate such that the sampling frame rate is equal to the first frame rate when an angular velocity of the portable device is equal to or higher than a third predetermined limit and/or an acceleration of the portable device is equal to or higher than a fourth predetermined limit.

17. The computer program product according to claim 13 , wherein the processor is further caused to:

determine a change of position of the portable device by using the movement data; and

determine a corrected position of the portable device by using said change and a previous determined position of the portable device.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2017
From: ROIMELA, KIMMO; FAN, LIXIN
To: NOKIA CORPORATION
Reel/Frame 042845/0841 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2017
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 042845/0942 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2017
From: ROIMELA, KIMMO; FAN, LIXIN
To: NOKIA CORPORATION
Reel/Frame 042846/0086 →
Priority Claims (1)
GB 1423327.4 · Dec 30, 2014 · national
Continuity (1)
Related Publication 20170357873A1 · Dec 14, 2017