IP Library Granted Patent US 10,096,115
Granted Patent B2
US 10,096,115 · App. 14/683,400 · Granted Oct 9, 2018

Building a depth map using movement of one camera

Inventors: Thomas Guillaume Grandin (Kitchener, CA); Sui Tong Tang (Waterloo, CA); Sung Ho Hong (Waterloo, CA)
Assignee: BlackBerry Limited
G06T7/0071G06T7/579H04N13/221H04N13/264G06T2207/10016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,096,115
App. No.
14/683,400
Granted
Oct 9, 2018
Kind
B2
Abstract

A method for building a depth map is operable on a mobile device having a single camera integrated therewith. The method includes capturing a plurality of images of a given view using movement of the mobile device between images, capturing data regarding the movement of the mobile device during capture of the plurality of images, determining a relative position of the mobile device corresponding to each of the plurality of images, and building a depth map using the plurality of images and the relative position corresponding to each of the plurality of images.

Claims (30)

1. A method operable on a mobile device, comprising:

capturing, with a single camera integrated with the mobile device, at least three images of a given view using movement of the mobile device between images, wherein the at least three images comprise an initial image taken from an initial position, a first subsequent image taken from a first subsequent position offset from the initial position in both vertical and horizontal directions, a second subsequent image taken from a second subsequent position further from the initial position, and the initial position, the first subsequent position, and the second subsequent position are substantially in a plane parallel to the given view being captured;

capturing data regarding the movement of the mobile device during capture of the at least three images;

after capturing the at least three images, determining a relative position of the mobile device corresponding to each of the captured at least three images; and

building a depth map for both close and far objects using the at least three images and the relative position corresponding to each of the at least three images.

2. The method as recited in claim 1 wherein determining the relative position of the mobile device corresponding to a given image comprises determining a distance moved by the mobile device since a previous image of the at least three images.

3. The method as recited in claim 2 wherein determining the relative position of the mobile device corresponding to the given image further comprises determining a direction of movement of the mobile device since the previous image of the at least three images.

4. The method as recited in claim 3 wherein determining the relative position of the mobile device corresponding to the given image uses information from an accelerometer.

5. The method as recited in claim 4 further comprising determining a rotation of the mobile device during movement of the mobile device between images and indicating the rotation to a user of the mobile device.

6. The method as recited in claim 1 further comprising determining a rotation of the mobile device during movement of the mobile device between images and compensating for the rotation when creating the depth map.

7. The method as recited in claim 1 further comprising using the depth map to perform one of measuring distances in the initial of the at least three images and measuring the size of objects in the initial image.

8. The method as recited in claim 1 further comprising using the depth map to perform post-processing of the initial image of the at least three images, the post-processing comprising separating objects from background in the initial image and changing a focus on specific objects of the initial image.

9. A non-transitory computer-readable medium containing instructions that when executed by a processor cause the processor to perform a method comprising:

capturing, with a single camera integrated with a mobile device, at least three images of a given view using movement of the mobile device between images, wherein the at least three images comprise an initial image taken from an initial position, a first subsequent image taken from a first subsequent position offset from the initial position in both vertical and horizontal directions, a second subsequent image taken from a second subsequent position further from the initial position, and the initial position, the first subsequent position, and the second subsequent position are substantially in a plane parallel to the given view being captured;

capturing data regarding the movement of the mobile device during capture of the at least three images;

after capturing the at least three images, determining a relative position of the mobile device corresponding to each of the captured at least three images; and

building a depth map for both close and far objects using the at least three images and the relative position corresponding to each of the at least three images.

10. The non-transitory computer-readable medium as recited in claim 9 wherein determining the relative position of the mobile device corresponding to a given image comprises determining a direction of movement of the mobile device and a distance moved by the mobile device since a previous image of the at least three images.

11. The non-transitory computer-readable medium as recited in claim 9 wherein determining the relative position of the mobile device corresponding to the given image uses information from an accelerometer integrated with the mobile device.

12. The non-transitory computer-readable medium as recited in claim 9 further comprising determining a rotation of the mobile device during movement of the mobile device between images and compensating for the rotation when creating the depth map.

13. The non-transitory computer-readable medium as recited in claim 9 further comprising using the depth map to perform one of measuring distances in the initial image of the at least three images, measuring objects in the initial image, separating objects from background in the initial image and changing a focus on specific objects of the initial image.

14. A mobile device comprising:

a camera that is connected to a memory and configured to store images created by the camera in the memory;

an accelerometer operable to detect movement of the mobile device in three dimensions; and

a processor operably coupled to the accelerometer and to the memory, the memory storing instructions that when executed by the processor, causes the camera to capture at least three images of a given view using movement of the mobile device between images, wherein the at least three images comprise an initial image taken from an initial position, a first subsequent image taken from a first subsequent position offset from the initial position in both vertical and horizontal directions, and a second subsequent image taken from a second subsequent position further from the initial position, and the initial position, the first subsequent position, and the second subsequent position are substantially in a plane parallel to the given view being captured, causes the accelerometer to capture data regarding the movement of the mobile device during capture of the at least three images, determines a relative position of the mobile device corresponding to each of the captured at least three images after the at least three images are captured, and builds a depth map for both close and far objects using the at least three images and the relative position corresponding to each of the at least three images.

15. The mobile device as recited in claim 14 wherein determining the relative position of the mobile device corresponding to a given image comprises determining a direction of movement of the mobile device and a distance moved by the mobile device since a previous image of the at least three images.

16. The mobile device as recited in claim 14 further comprising determining a rotation of the mobile device during movement of the mobile device between images and indicating the rotation to a user of the mobile device.

17. The mobile device as recited in claim 14 further comprising determining a rotation of the mobile device during movement of the mobile device between images and compensating for the rotation when creating the depth map.

18. The mobile device as recited in claim 14 further comprising using the depth map to perform one of measuring distances in initial image of the at least three images and measuring objects in the first image.

19. The mobile device as recited in claim 14 comprising using the depth map to perform post-processing of the initial image of the at least three images, the post-processing comprising one of separating objects from background in the initial image and changing a focus of the initial image.

Assignments (5)
NUNC PRO TUNC ASSIGNMENT Recorded Jun 19, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064271/0199 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064104/0103 →
CHANGE OF NAME Recorded Aug 2, 2018
From: RESEARCH IN MOTION LIMITED
To: BLACKBERRY LIMITED
Reel/Frame 046702/0259 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 26, 2015
From: TANG, SUI TONG; HONG, SUNG HO
To: BLACKBERRY LIMITED
Reel/Frame 036426/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 26, 2015
From: GRANDIN, THOMAS
To: RESEARCH IN MOTION LIMITED
Reel/Frame 036426/0403 →
Priority Claims (1)
CA 2848794 · Apr 11, 2014 · national
Continuity (1)
Related Publication 20150294474A1 · Oct 15, 2015