IP Library › Granted Patent US 10,438,373
Granted Patent B2
US 10,438,373 · App. 15/807,263 · Granted Oct 8, 2019

Method and system for determining a pose of camera

Inventors: Lejing Wang (Munich, DE); Peter Meier (Munich, DE)
Assignee: Apple Inc.
G06T7/74G06T7/70G06T2207/10016G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,438,373
App. No.
15/807,263
Granted
Oct 8, 2019
Kind
B2
Abstract

The invention is related to a method and system for determining a pose of a first camera, comprising providing or receiving a spatial relationship (Rvc 1 ) between a visual content displayed on a display device and the first camera, receiving image information associated with an image (B 1 ) of at least part of the displayed visual content captured by a second camera, and determining a pose of the first camera according to the image information associated with the image (B 1 ) and the spatial relationship (Rvc 1 ).

Claims (49)

1. A method comprising:

obtaining a spatial relationship between a first camera and visual content displayed on a display device;

receiving first image data corresponding to a first image captured by a second camera at a first time, the first image depicting at least part of the visual content displayed on the display device;

receiving second image data corresponding to a second image captured by the second camera at a second time, the second image depicting at least part of the visual content displayed on the display device; and

determining camera motion data indicating a change in orientation of the first camera between the first time and the second time based on the spatial relationship, the first image data, and the second image data.

2. The method of claim 1 , wherein the first camera is oriented in a first pose at the first time and is oriented in a second pose at the second time, the method further comprising:

receiving third image data corresponding to a third image captured by the first camera while the first camera is oriented in the first pose, wherein the third image depicts an object;

receiving fourth image data corresponding to a fourth image captured by the first camera while the first camera is oriented in the second pose, wherein the fourth image depicts the object; and

generating a model of the object based on the third image data, the fourth image data, and the camera motion data.

3. The method of claim 2 , further comprising initiating capture of the third image by the first camera while the display device displays the visual content.

4. The method of claim 2 , further comprising using the model to initialize a simultaneous localization and mapping (SLAM) system for tracking the first camera in a real environment.

5. The method of claim 1 , further comprising:

determining a planar homography based on the visual content as depicted in the first image and the visual content as depicted in the second image; and

determining the camera motion data by decomposing the planar homography.

6. The method of claim 1 , wherein the first camera and the display device are included in a mobile device.

7. The method of claim 6 , wherein the second camera is included in a head mounted device.

8. An apparatus comprising:

one or more processors; and

a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

obtaining a spatial relationship between a first camera and visual content displayed on a display device;

receiving first image data corresponding to a first image captured by a second camera at a first time, the first image depicting at least part of the visual content displayed on the display device;

receiving second image data corresponding to a second image captured by the second camera at a second time, the second image depicting at least part of the visual content displayed on the display device; and

determining camera motion data indicating a change in orientation of the first camera between the first time and the second time based on the spatial relationship, the first image data, and the second image data.

9. The apparatus of claim 8 , wherein the first camera is oriented in a first pose at the first time and is oriented in a second pose at the second time, the operations further comprising:

receiving third image data corresponding to a third image captured by the first camera while the first camera is oriented in the first pose, wherein the third image depicts an object;

receiving fourth image data corresponding to a fourth image captured by the first camera while the first camera is oriented in the second pose, wherein the fourth image depicts the object; and

generating a model of the object based on the third image data, the fourth image data, and the camera motion data.

10. The apparatus of claim 9 , wherein the operations further include initiating capture of the third image by the first camera while the display device displays the visual content.

11. The apparatus of claim 9 , wherein the operations further include using the model to initialize a simultaneous localization and mapping (SLAM) system for tracking the first camera in a real environment.

12. The apparatus of claim 8 , wherein the operations further include:

determining a planar homography based on the visual content as depicted in the first image and the visual content as depicted in the second image; and

determining the camera motion data by decomposing the planar homography.

13. The apparatus of claim 8 , wherein the first camera and the display device are included in a mobile device.

14. The apparatus of claim 13 , wherein the second camera is included in a head mounted device.

15. The apparatus of claim 13 , wherein the one or more processors are included in the mobile device.

16. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

obtaining a spatial relationship between a first camera and visual content displayed on a display device;

receiving first image data corresponding to a first image captured by a second camera at a first time, the first image depicting at least part of the visual content displayed on the display device;

receiving second image data corresponding to a second image captured by the second camera at a second time, the second image depicting at least part of the visual content displayed on the display device; and

determining camera motion data indicating a change in orientation of the first camera between the first time and the second time based on the spatial relationship, the first image data, and the second image data.

17. The computer-readable storage medium of claim 16 , wherein the first camera is oriented in a first pose at the first time and is oriented in a second pose at the second time, the operations further comprising:

receiving third image data corresponding to a third image captured by the first camera while the first camera is oriented in the first pose, wherein the third image depicts an object;

receiving fourth image data corresponding to a fourth image captured by the first camera while the first camera is oriented in the second pose, wherein the fourth image depicts the object; and

generating a model of the object based on the third image data, the fourth image data, and the camera motion data.

18. The computer-readable storage medium of claim 17 , wherein the operations further include initiating capture of the third image by the first camera while the display device displays the visual content.

19. The computer-readable storage medium of claim 17 , wherein the operations further include using the model to initialize a simultaneous localization and mapping (SLAM) system for tracking the first camera in a real environment.

20. The computer-readable storage medium of claim 16 , wherein the operations further include:

determining a planar homography based on the visual content as depicted in the first image and the visual content as depicted in the second image; and

determining the camera motion data by decomposing the planar homography.

Continuity (2)
Continuation 15028474
Related Publication 20180075620A1 · Mar 15, 2018
Cited By (1)
US 12,353,617