IP Library › Granted Patent US 10,085,006
Granted Patent B2
US 10,085,006 · App. 15/585,042 · Granted Sep 25, 2018

Three hundred sixty degree video stitching

Inventors: Tuan Ho (Garland, TX); Madhukar Budagavi (Plano, TX)
Assignee: Samsung Electronics Co., Ltd.
H04N13/111G06T7/85G11B27/031G11B27/28H04N13/0011H04N13/0055H04N13/0239H04N13/0271H04N13/189H04N13/239H04N13/271H04N21/4223H04N21/854G06F3/04815G06T7/97
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,085,006
App. No.
15/585,042
Granted
Sep 25, 2018
Kind
B2
Abstract

An electronic device configured to render a three hundred and sixty (360) degree image includes a communication unit and a processor. The communication unit is configured to receive a first image, a second image, and metadata from a camera. The processor is configured to render the 360 degree image by transforming the first image and the second image to compensate for a light fall-off, unwarping the transformed first image and second image, aligning the unwarped first image and second image, and blending the aligned unwarped first image and second image.

Claims (60)

1. An electronic device configured to render a three hundred and sixty (360) degree image, the electronic device comprising:

a communication unit configured to receive a first image, a second image, and metadata from a camera; and

a processor configured to render the 360 degree image by:

transforming the first image and the second image to compensate for a light fall-off;

unwarping the transformed first image and second image;

aligning the unwarped first image and second image; and

blending an overlapping portion of the aligned unwarped first image and second image to render the 360 degree image.

2. The electronic device of claim 1 , wherein the communication unit is configured to transmit the rendered 360 degree image to a user equipment (UE).

3. The electronic device of claim 1 , wherein transforming the first image and the second image to compensate for a light fall-off is based on a polynomial equation included in the metadata.

4. The electronic device of claim 1 , wherein aligning the unwarped first image and second image comprises:

transforming the unwarped first image and second image to compensate for a mechanical misalignment of the camera; and

transforming the unwarped first image and second image based on at least one of an object or a scene in the unwarped first image and second image.

5. The electronic device of claim 4 , wherein transforming the unwarped first image and second image to compensate for a mechanical misalignment is based on an affine matrix included in the metadata.

6. The electronic device of claim 4 , wherein transforming the unwarped first image and second image based on at least one of an object or a scene in the unwarped first image and second image comprises:

determining a first template and a reference to match in at least one boundary shared between the unwarped first image and second image;

matching the first template to return a correlation matrix;

determining a displacement between the first template and the reference;

determining a second template based on the displacement;

determining an affine matrix based on the first template and the second template; and

warping at least one of the first image or the second image based on the affine matrix.

7. The electronic device of claim 6 , wherein matching the first template is based on a normalized cross-correlation.

8. The electronic device of claim 6 , wherein the affine matrix is determined based on a plurality of vertices in the first template and the second template.

9. A method for rendering a three hundred and sixty (360) degree image in an electronic device having a processor, the method comprising:

receiving a first image, a second image, and metadata from a camera;

transforming, by the processor, the first image and the second image to compensate for a light fall-off;

unwarping, by the processor, the transformed first image and second image;

aligning, by the processor, the unwarped first image and second image; and

blending, by the processor, an overlapping portion of the aligned unwarped first image and second image to render the 360 degree image.

10. The method of claim 9 , further comprising transmitting the rendered 360 degree image to a user equipment (UE).

11. The method of claim 9 , wherein transforming the first image and the second image to compensate for a light fall-off is based on a polynomial equation included in the metadata.

12. The method of claim 9 , wherein aligning the unwarped first image and second image comprises:

transforming the unwarped first image and second image to compensate for a mechanical misalignment of the camera; and

transforming the unwarped first image and second image based on at least one of an object or a scene in the unwarped first image and second image.

13. The method of claim 12 , wherein transforming the unwarped first image and second image to compensate for a mechanical misalignment is based on an affine matrix included in the metadata.

14. The method of claim 12 , wherein transforming the unwarped first image and second image based on at least one of an object or a scene in the unwarped first image and second image comprises:

determining a first template and a reference to match in at least one boundary shared between the unwarped first image and second image;

matching the first template to return a correlation matrix;

determining a displacement between the first template and the reference;

determining a second template based on the displacement;

determining an affine matrix based on the first template and the second template; and

warping at least one of the first image or the second image based on the affine matrix.

15. The method of claim 14 , wherein matching the template is based on a normalized cross-correlation.

16. The method of claim 14 , wherein the affine matrix is determined based on a plurality of vertices in the first template and the second template.

17. A non-transitory computer readable medium embodying a computer program, the computer program comprising computer readable program code that when executed causes at least one processing device to:

receive a first image, a second image, and metadata from a camera; and

transform the first image and the second image to compensate for a light fall-off;

unwarp the transformed first image and second image;

align the unwarped first image and second image; and

blend an overlapping portion of the aligned unwarped first image and second image to render a 360 degree image.

18. The non-transitory computer readable medium of claim 17 , wherein the at least one processing device transforms the first image and the second image to compensate for a light fall-off based on a polynomial equation included in the metadata.

19. The non-transitory computer readable medium of claim 17 , wherein the at least one processing device aligns the unwarped first image and second image by:

transforming the unwarped first image and second image to compensate for a mechanical misalignment of the camera; and

transforming the unwarped first image and second image based on at least one of an object or a scene in the unwarped first image and second image.

20. The non-transitory computer readable medium of claim 19 , wherein the at least one processing device transforms the unwarped first image and second image based on at least one of an object or a scene in the unwarped first image and second image by:

determining a first template and a reference to match in at least one boundary shared between the unwarped first image and second image;

matching the first template to return a correlation matrix;

determining a displacement between the first template and the reference;

determining a second template based on the displacement;

determining an affine matrix based on the first template and the second template; and

warping at least one of the first image or the second image based on the affine matrix.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 29, 2018
From: HO, TUAN; BUDAGAVI, MADHUKAR
To: SAMSUNG ELECTRONICS CO., LTD
Reel/Frame 045921/0969 →
Continuity (3)
Provisional Application 62384981 · Sep 8, 2016
Provisional Application 62411806 · Oct 24, 2016
Related Publication 20180070070A1 · Mar 8, 2018
Cited By (1)
US 12,201,403