IP Library Granted Patent US 9,202,288
Granted Patent B2
US 9,202,288 · App. 14/078,767 · Granted Dec 1, 2015

Method, apparatus and computer program product for processing of image frames

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,202,288
App. No.
14/078,767
Granted
Dec 1, 2015
Kind
B2
Abstract

In accordance with an example embodiment a method, apparatus and computer program product are provided. The method comprises receiving color image frames and panchromatic image frames associated with a scene. The color image frames correspond to the panchromatic image frames. The method further comprises computing registration information based on a panchromatic image frame and one or more panchromatic image frames from among the panchromatic image frames. A color image frame corresponding to the panchromatic image frame is modified based on the registration information.

Claims (59)

1. A method comprising:

receiving colour image frames and panchromatic image frames associated with a scene, wherein the colour image frames correspond to the panchromatic image frames;

computing registration information based on a panchromatic image frame and one or more panchromatic image frames from among the panchromatic image frames; and

modifying a colour image frame corresponding to the panchromatic image frame based on the registration information, wherein modifying the colour image frame further comprises adding pixel values of pixels of one or more colour image frames to pixel values of corresponding pixels of the colour image frame based on the registration information.

2. The method as claimed in claim 1 , further comprising:

determining registration information from the colour image frame and the one or more colour image frames from among the colour image frames, wherein the one or more colour image frames correspond to the one or more panchromatic image frames.

3. The method as claimed in claim 2 , wherein modifying the colour image frame comprises updating pixel values of pixels of the colour image frame with average pixel values obtained from averaging of pixel values of pixels in the one or more colour image frames and the pixel values of the corresponding pixels of the colour image frame based on the registration information.

4. The method as claimed in claim 2 , further comprising computing a global registration information by performing:

extracting features from the panchromatic image frame and the one or more panchromatic image frames;

matching the features extracted from the panchromatic image frame and the one or more panchromatic image frames; and

determining homography transformation between the panchromatic image frame and the one or more panchromatic image frames based on the matching of the extracted features.

5. The method as claimed in claim 2 , further comprises computing a local registration information by performing:

generating a compensated panchromatic image frame corresponding to at least one panchromatic image frame of the one or more panchromatic image frames based on the homography transformation associated with the at least one panchromatic image frame;

computing a difference image frame based on the compensated panchromatic image frame and the panchromatic image frame; and

comparing a pixel value attribute associated with each block of pixels of a plurality of blocks of pixels configuring the difference image frame with a pre-defined threshold value.

6. The method as claimed in claim 5 , wherein computing the local registration information comprises:

performing motion compensation for one or more blocks of pixels from among the plurality of blocks of pixels configuring the difference image frame, wherein the pixel value attribute associated with each of the one or more blocks of pixels is determined to exceed the pre-defined threshold value.

7. The method as claimed in claim 6 , further comprises determining, based on tone mapping, a number of colour image frames in the one or more colour image frames for modifying the colour image frame.

8. An apparatus comprising:

at least one processor; and

at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform:

receive colour image frames and panchromatic image frames associated with a scene, wherein the colour image frames correspond to the panchromatic image frames;

compute registration information based on a panchromatic image frame and one or more panchromatic image frames from among the panchromatic image frames;

modify a colour image frame corresponding to the panchromatic image frame based on the registration information; and

add pixel values of pixels of one or more colour image frames to pixel values of corresponding pixels of the colour image frame based on the registration information for modifying the colour image frame.

9. The apparatus as claimed in claim 8 , wherein the apparatus is further caused, at least in part, to:

determine registration information from the colour image frame and the one or more colour image frames from among the colour image frames, wherein the one or more colour image frames correspond to the one or more panchromatic image frames.

10. The apparatus as claimed in claim 9 , wherein the apparatus is further caused, at least in part, to:

update pixel values of pixels of the colour image frame with average pixel values obtained from averaging of pixel values of pixels in the one or more colour image frames and the pixel values of the corresponding pixels of the colour image frame based on the registration information.

11. The apparatus as claimed in claim 9 , wherein the apparatus is further caused, at least in part, to: compute a global registration information by:

extracting features from the panchromatic image frame and the one or more panchromatic image frames;

matching the features extracted from the panchromatic image frame and the one or more panchromatic image frames; and

determining homography transformation between the panchromatic image frame and the one or more panchromatic image frames based on the matching of the extracted features.

12. The apparatus as claimed in claim 9 , wherein the apparatus is further caused, at least in part, to: compute a local registration information by:

generating a compensated panchromatic image frame corresponding to at least one panchromatic image frame of the one or more panchromatic image frames based on the homography transformation associated with the at least one panchromatic image frame;

computing a difference image frame based on the compensated panchromatic image frame and the panchromatic image frame; and

comparing a pixel value attribute associated with each block of pixels of a plurality of blocks of pixels configuring the difference image frame with a pre-defined threshold value.

13. The apparatus as claimed claim 12 , wherein computing the local registration information further comprises:

performing motion compensation for one or more blocks of pixels from among the plurality of blocks of pixels configuring the difference image frame, wherein the pixel value attribute associated with each of the one or more blocks of pixels is determined to exceed the pre-defined threshold value.

14. The apparatus as claimed in claim 13 , wherein the apparatus is further caused, at least in part, to: determine a number of colour image frames in the one or more colour image frames for modifying the colour image frame based on tone mapping.

15. A computer program product comprising at least one computer-readable storage medium, the computer-readable storage medium comprising a set of instructions, which, when executed by one or more processors, cause an apparatus to at least perform:

receive colour image frames and panchromatic image frames associated with a scene, wherein the colour image frames correspond to the panchromatic image frames;

compute registration information based on a panchromatic image frame and one or more panchromatic image frames from among the panchromatic image frames;

modify a colour image frame corresponding to the panchromatic image frame based on the registration information; and

add pixel values of pixels of one or more colour image frames to pixel values of corresponding pixels of the colour image frame based on the registration information for modifying the colour image frame.

16. The computer program product as claimed in claim 15 , wherein the apparatus is further caused, at least in part, to:

determine registration information from the colour image frame and the one or more colour image frames from among the colour image frames, wherein the one or more colour image frames correspond to the one or more panchromatic image frames.

17. The computer program product as claimed in claim 16 , wherein the apparatus is further caused, at least in part, to:

update pixel values of pixels of the colour image frame with average pixel values obtained from averaging of pixel values of pixels in the one or more colour image frames and the pixel values of the corresponding pixels of the colour image frame based on the registration information.

18. The computer program product as claimed in claim 16 , wherein the apparatus is further caused, at least in part, to: compute a global registration information by:

extracting features from the panchromatic image frame and the one or more panchromatic image frames;

matching the features extracted from the panchromatic image frame and the one or more panchromatic image frames; and

determining homography transformation between the panchromatic image frame and the one or more panchromatic image frames based on the matching of the extracted features.

19. The computer program product as claimed in claim 16 , wherein the apparatus is further caused, at least in part, to: compute a local registration information by:

generating a compensated panchromatic image frame corresponding to at least one panchromatic image frame of the one or more panchromatic image frames based on the homography transformation associated with the at least one panchromatic image frame;

computing a difference image frame based on the compensated panchromatic image frame and the panchromatic image frame; and

comparing a pixel value attribute associated with each block of pixels of a plurality of blocks of pixels configuring the difference image frame with a pre-defined threshold value.

20. The computer program product as claimed in claim 19 , wherein computing the local registration information further comprises:

performing motion compensation for one or more blocks of pixels from among the plurality of blocks of pixels configuring the difference image frame, wherein the pixel value attribute associated with each of the one or more blocks of pixels is determined to exceed the pre-defined threshold value.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2015
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 035215/0973 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 13, 2013
From: MUNINDER, VELDANDI
To: NOKIA CORPORATION
Reel/Frame 031591/0148 →