IP Library Granted Patent US 9,619,863
Granted Patent B2
US 9,619,863 · App. 14/395,962 · Granted Apr 11, 2017

Method, apparatus and computer program product for generating panorama images

Inventors: Veldandi Muninder (Bangalore, IN); Basavaraja S V (Bangalore, IN)
Assignee: Nokia Technologies Oy
G06T3/4038G06K9/4609G06T3/0068H04N5/23238G06T2200/32
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,619,863
App. No.
14/395,962
Granted
Apr 11, 2017
Kind
B2
Abstract

In accordance with an example embodiment a method, apparatus and computer program product are provided. The method comprise assigning weights to at least one first feature and at least one second feature. The at least one first feature may be associated with a first image and the at least one second feature may be associated with a second image. The weights are assigned based on a distance of the at least one first feature and the at least one second feature from a central portion of the corresponding one of the first image and the second image. The method further includes registering the first image and the second image based at least on the assigned weights to determine transforms between the first image and the second image. The first image and the second image may be based on the determined transform.

Claims (60)

1. A method comprising:

assigning, by an image processor, weights to at least one first feature and at least one second feature, the at least one first feature being associated with a first image and the at least one second feature being associated with a second image, the weights being assigned based on a distance of the at least one first feature and the at least one second feature from a center of an overlapping portion of the first image and the second image, wherein the weights decrease exponentially from the center of the overlapping portion;

registering, by the image processor, the first image and the second image based at least on the assigned weights to determine transforms between the first image and the second image, wherein features closer to the center of the overlapping portion of the first image and the second image are preferentially used for determining the transforms; and

transforming, by the image processor, the first image and the second image based on the determined transform.

2. The method as claimed in claim 1 , wherein registering at least the first image and the second image comprises:

performing feature matching between the at least one first feature and the at least one second feature to generate a plurality of matching features; and

computing a plurality of matching weights for the plurality of matching features based on the weights assigned to the at least one first feature and the at least one second feature.

3. The method as claimed in claim 2 further comprising:

estimating a plurality of transforms between a set of matching features of the plurality of matching features, the set of matching features being selected from the plurality of matching features based on the plurality of matching weights; and

selecting a transform of the plurality of transforms based on a total matching weight associated with the transform, the total matching weight of the transform being determined based on a plurality of inliers associated with the transform.

4. The method as claimed in claim 1 , further comprising computing the at least one first feature and the at least one second feature prior to assigning weights to the at least one first feature and the at least one second feature.

5. The method as claimed in claim 1 , further comprising determining a distance of the at least one first feature from a central portion of the first image, and determining a distance of the at least one second feature from a central portion of the second image, prior to assigning weights to the at least one first feature and the at least one second feature.

6. The method as claimed in claim 3 , wherein the set of matching features is selected from the plurality of matching features by:

sorting the plurality of matching features in an order of the matching weights associated with each of the plurality of matching features; and

selecting the matching features of the plurality of matching features that have a matching weight greater than a predetermined matching weight as the set of matching features.

7. The method as claimed in claim 6 , wherein estimating the plurality of transforms from the set of matching features comprises:

selecting random pairwise matching features from the set of matching features;

computing the total matching weight for each of the selected random pairwise matching features; and

determining a transform corresponding to a maximum total matching weight from the computed total matching weight, the determined transform being utilized for panorama generation.

8. The method as claimed in claim 6 , wherein the total matching weight is determined by:

computing a probable transform for each of the random pairwise matching features;

determining an inlier corresponding to each of the probable transform; and

computing the total matching weight associated with each of the probable transforms for the determined inlier.

9. An apparatus comprising:

at least one processor; and

at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform:

assign weights to at least one first feature and at least one second feature, the at least one first feature being associated with a first image and the at least one second feature being associated with a second image, the weights being assigned based on a distance of the at least one first feature and the at least one second feature from a center of an overlapping portion of the first image and the second image, wherein the weights decrease exponentially from the center of the overlapping portion;

register the first image and the second image based at least on the assigned weights to determine transforms between the first image and the second image, wherein features closer to the center of the overlapping portion of the first image and the second image are preferentially used for determining the transforms; and

transform the first image and the second image based on the determined transform.

10. The apparatus as claimed in claim 9 , wherein, to register at least the first image and the second image, the apparatus is further caused, at least in part to:

perform feature matching between the at least one first feature and the at least one second feature to generate a plurality of matching features; and

compute a plurality of matching weights for the plurality of matching features based on the weights assigned to the at least one first feature and the at least one second feature.

11. The apparatus as claimed in claim 10 , wherein the apparatus is further caused, at least in part to perform:

estimate a plurality of transforms between a set of matching features of the plurality of matching features, the set of matching features being selected from the plurality of matching features based on the plurality of matching weights; and

select a transform of the plurality of transforms based on a total matching weight associated with the transform, the total matching weight of the transform being determined based on a plurality of inliers associated with the transform.

12. The apparatus as claimed in claim 9 , wherein the apparatus is further caused, at least in part to compute the at least one first feature and the at least one second feature prior to assigning weights to the at least one first feature and the at least one second feature.

13. The apparatus as claimed in claim 9 , wherein the apparatus is further caused, at least in part to determine a distance of the at least one first feature from a central portion of the first image, and determine a distance of the at least one second feature from a central portion of the second image, prior to assigning weights to the at least one first feature and the at least one second feature.

14. The apparatus as claimed in claim 11 , wherein, to select the set of matching features from the plurality of matching features, the apparatus is further caused, at least in part to:

sort the plurality of matching features in an order of the matching weights associated with each of the plurality of matching features; and

select the matching features of the plurality of matching features that have a matching weight greater than a predetermined matching weight as the set of matching features.

15. The apparatus as claimed in claim 14 , wherein, to estimate the plurality of transforms from the set of matching features, the apparatus is further caused, at least in part to:

select random pairwise matching features from the set of matching features;

compute the total matching weight for each of the selected random pairwise matching features; and

determine a transform corresponding to a maximum total matching weight from the computed total matching weight, the determined transform being utilized for panorama generation.

16. The apparatus as claimed in claim 13 , wherein, to determine the total matching weight, the apparatus is further caused, at least in part to:

compute a probable transform for each of the random pairwise matching features;

determine an inlier corresponding to each of the probable transform; and

compute the total matching weight associated with each of the probable transforms for the determined inlier.

17. A computer program product comprising at least one non-transitory computer-readable storage medium, the computer-readable storage medium comprising a set of instructions, which, when executed by one or more processors, cause an apparatus at least to perform:

assign weights to at least one first feature and at least one second feature, the at least one first feature being associated with a first image and the at least one second feature being associated with a second image, the weights being assigned based on a distance of the at least one first feature and the at least one second feature from a center of an overlapping portion of the first image and the second image, wherein the weight sdecrease exponentially from the center of the overlapping portion;

register the first image and the second image based at least on the assigned weights to determine transforms between the first image and the second image, wherein features closer to the center of the overlapping portion of the first image and the second image are preferentially used for determining the transforms; and

transform the first image and the second image based on the determined transform.

18. The computer program product as claimed in claim 17 , wherein, to register at least the first image and the second image, the apparatus is further caused, at least in part to:

perform feature matching between the at least one first feature and the at least one second feature to generate a plurality of matching features; and

compute a plurality of matching weights for the plurality of matching features based on the weights assigned to the at least one first feature and the at least one second feature.

19. The computer program product as claimed in claim 18 , wherein the apparatus is further caused, at least in part to perform:

estimate a plurality of transforms between a set of matching features of the plurality of matching features, the set of matching features being selected from the plurality of matching features based on the plurality of matching weights; and

select a transform of the plurality of transforms based on a total matching weight associated with the transform, the total matching weight of the transform being determined based on a plurality of inliers associated with the transform.

20. The computer program product as claimed in claim 17 , wherein the apparatus is further caused, at least in part to compute the at least one first feature and the at least one second feature prior to assigning weights to the at least one first feature and the at least one second feature.

21. The computer program product as claimed in claim 17 , wherein the apparatus is further caused, at least in part to determine a distance of the at least one first feature from a central portion of the first image, and determine a distance of the at least one second feature from a central portion of the second image, prior to assigning weights to the at least one first feature and the at least one second feature.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 11, 2017
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 040946/0547 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MUNINDER, VELDANDI; S V, BASAVARAJA
To: NOKIA CORPORATION
Reel/Frame 034443/0937 →
Priority Claims (1)
IN 1632/CHE/2012 · Apr 25, 2012 · national
Continuity (1)
Related Publication 20150131909A1 · May 14, 2015