IP Library Granted Patent US 9,076,269
Granted Patent B2
US 9,076,269 · App. 13/813,187 · Granted Jul 7, 2015

Method and electronic device for creating a combined image

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,076,269
App. No.
13/813,187
Granted
Jul 7, 2015
Kind
B2
Abstract

An object of the present invention is to provide a way of improving the stitching procedure. The object is achieved by a method in an electronic device for creating a combined image. The method comprises obtaining ( 701 ) a first set of at least two sub-images recorded by an array camera. The method further comprises obtaining ( 702 ) a second set of at least two sub-images recorded by the array camera. A first depth map is calculated for the first set of sub-images. The first depth map comprises information about a first plane of the sub-images and a second plane of the sub-images. The first plane being at a first distance from the point where the sub-images have been recorded, and the second plane being at a second distance from the point where the sub-images have been recorded. A second depth map is calculated for the second set of sub-images. The second depth map comprises information about the first plane and the second plane. The method further comprises stitching the first set of sub-images and the second set of sub-images in the first plane, and thereby obtaining a first stitched image. The first set of sub-images and the second set of sub-images are stitched in the second plane, and thereby obtaining a second stitched image. Finally the method comprises combining the first stitched image with the second stitched image and thereby creating the combined image.

Claims (35)

1. A method in an electronic device for creating a combined image, the method comprising:

obtaining a first set of at least two sub-images recorded by an array of image recording units;

obtaining a second set of at least two sub-images recorded by the array of image recording units;

calculating a first depth map for the first set of sub-images, the first depth map comprises information about a first plane of the sub-images and a second plane of the sub-images, the first plane being at a first distance from the point where the sub-images have been recorded, and the second plane being at a second distance from the point where the sub-images have been recorded:

calculating a second depth map for the second set of sub-images, wherein the second depth map comprising information about the first plane and the second plane;

stitching the first set of sub-images and the second set of sub-images in the first plane, to obtain a first stitched image;

stitching the first set of sub-images and the second set of sub-images in the second plane to obtain a second stitched image; and

combining the first stitched image with the second stitched image to create the combined image.

2. The method according to claim 1 wherein the distance between the first plane to the point where the sub-images have been recorded is larger than the distance between the second plane and the point where the sub-images have been recorded.

3. The method according to claim 2 wherein combining the first stitched image with the second stitched image comprises placing the second stitched image in front of the first stitched image.

4. The method according to claim 1 wherein the calculating of the first depth map and the second depth map further is based on information about a third plane, the third plane being at a third distance from the point where the sub-images have been recorded, the method further comprising:

stitching the first set of sub-images and the second set of sub-images in the third plane, to obtain a third stitched image, and wherein the combining of the first stitched image and the second stitched image further comprises combining the first stitched image and the second stitched image with the third stitched image.

5. The method according to claim 1 wherein calculating the first depth map and the second depth map further is based on information about n planes, where n is a natural number, the n planes being at n different distances from the point where the sub-images have been recorded,

wherein stitching the first set of sub-images and the second set of sub-images comprises stitching in the respective n planes, to obtain n stitched images, and wherein the combining comprises:

combining the first stitched image and the second stitched image with the n stitched images.

6. The method according to claim 1 wherein the array of image recording units comprises at least two sub-cameras adapted to produce one sub-image each.

7. The method according to claim 1 wherein the array of image recording units comprises at least two sub-images taken by a single camera at least at two different locations.

8. The method according to claim 1 , wherein combining further comprises recreating information in the first stitched image using information from the first set of sub-images and second set of sub-images, the information in the first plane being hidden by objects in the second plane by one of the first set of sub-images and the second set of sub-images.

9. A non-transitory computer-readable memory device comprising one or more instructions for performing one or more actions of claim 1 when being executed.

10. An electronic device for creating a combined image, comprising:

a first sub-camera to obtain a first set of at least two sub-images recorded by an array of cameras,

a second sub-camera to obtain a second set of at least two sub-images recorded by the array of cameras, and

one or more processors to:

calculate a first depth map for the first set of sub-images, the first depth map comprises information about a first plane of the sub-images and a second plane of the sub-images, the first plane being at a first distance from the point where the sub-images have been recorded, and the second plane being at a second distance from the point where the sub-images have been recorded,

calculate a second depth map for the second set of sub-images, wherein the second depth map comprises information about the first plane and the second plane,

stitch the first set of sub-images and the second set of sub-images in the first plane, to generate a first stitched image,

stitch the first set of sub-images and the second set of sub-images in the second plane, to generate a second stitched image, and

combine the first stitched image with the second stitched image and to create the combined image.

11. The electronic device according to claim 10 , wherein the distance between the first plane to the point where the sub-images have been recorded is larger than the distance between the second plane and the point where the sub-images have been recorded.

12. The electronic device according to claim 11 , wherein the one or more processors to combine the first stitched image with the second stitched image is further configured to place the second stitched image in front of the first stitched image.

13. The electronic device according to claim 11 , wherein the one or more processors is further configured to recreate information in the first stitched image using information from the first set of sub-images and second set of sub-images, the information in the first plane being hidden by objects in the second plane by one of the first set of sub-images and the second set of sub-images.

14. The electronic device according to claim 10 , wherein the one or more processors to calculate the first depth map and the second depth map is further configured to calculate information about n planes, where n is a natural number, the n planes being at n distances from the point where the sub-images have been recorded,

wherein the one or more processors to stitch the first set of sub-images and the second set of sub-images is further configured to stitch the respective n planes, to generate n stitched images, and

wherein the one or more processors to combine the first stitched image and the second stitched image is further configured to combine the first stitched image and the second stitched image with the n stitched images.

15. The electronic device according to claim 10 wherein the array of cameras comprises at least two sub-cameras adapted to produce one sub-image each.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 25, 2019
From: SONY MOBILE COMMUNICATIONS, INC.
To: SONY CORPORATION
Reel/Frame 048691/0134 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2017
From: SONY MOBILE COMMUNICATIONS AB
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 043951/0529 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2017
From: SONY CORPORATION
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 043943/0631 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2015
From: SONY MOBILE COMMUNICATIONS AB
To: SONY CORPORATION; SONY MOBILE COMMUNICATIONS AB
Reel/Frame 035766/0518 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2013
From: ASTRAND, PER; LANDQVIST, MAGNUS; THORN, KARL OLA; WERNERSSON, MATS; MARTENSSON, LINUS; BENGTSSON, HENRIK
To: SONY MOBILE COMMUNICATIONS AB
Reel/Frame 029719/0370 →