IP Library Granted Patent US 10,545,717
Granted Patent B2
US 10,545,717 · App. 15/763,379 · Granted Jan 28, 2020

Methods and systems of automatic calibration for dynamic display configurations

Inventor: Tatu V. J. Harviainen (Helsinki, FI)
Assignee: PCMS Holdings, Inc.
G06F3/147G02B27/017G06F3/011H04N13/156H04N13/332H04N13/344H04N13/361H04N13/363G02B2027/014G02B2027/0134G02B2027/0138G09G2320/0693
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,545,717
App. No.
15/763,379
Granted
Jan 28, 2020
Kind
B2
Abstract

Systems and methods are described for capturing, using a forward-facing camera associated with a head-mounted augmented reality (AR) head-mounted display (HMD), images of portions of first and second display devices in an environment, the first and second display devices displaying first and second portions of content related to an AR presentation, and displaying a third portion of content related to the AR presentation on the AR HMD, the third portion determined based upon the images of portions of the first and second display devices captured using the forward-facing camera. Moreover, the first and second display devices may be active stereo display, and the AR HMD may simultaneously function as shutter glasses.

Claims (34)

1. A method comprising:

capturing, at a forward-facing camera associated with an augmented reality (AR) head-mounted display (HMD), images coming from a portion of a first screen;

responsively determining that the captured images comprise a first portion of three-dimensional (3-D) content as displayed on the first screen as alternating images of a stereoscopic image sequence, the first portion of 3-D content being related to an AR presentation;

detecting a frequency of the captured images by analyzing sequential images of the captured images; and

shuttering, at the detected frequency, the AR HMD in synchronization with the images of the stereoscopic image sequence while also presenting left-eye imagery of a second portion of content in a portion of the AR HMD associated with the left-eye display, and right-eye imagery of the second portion of content in a portion of the AR HMD associated with the right-eye display.

2. The method of claim 1 , wherein the detection is done through image analysis.

3. The method of claim 1 , wherein the detection is done through occlusion analysis.

4. The method of claim 1 , further comprising:

capturing, using the forward-facing camera associated with the AR HMD, second images of a portion of a second screen; and

determining that the captured second images comprise a third portion of content as displayed on the second screen, the third portion of content being related to the AR presentation,

wherein the AR HMD determines the second portion of content for presentation by the AR HMD based on the captured images of the portion of the first screen and the captured second images of the portion of the second screen.

5. The method of claim 4 , wherein the AR HMD determines the second portion of content based on portions of the content related to the AR presentation that are associated with portions of a field of view that are not occupied by the first or second screens.

6. An apparatus comprising:

a network module configured to connect to an external display;

a processor running a mobile application, the mobile application executing instructions comprising:

intercepting a set of 3D content; and

partitioning the set of 3D content into a first virtual element and a second virtual element;

a streaming module configured to display the first virtual element on the external display;

a head-mounted display configured to display the second virtual element, the head-mounted display having a portion associated with a left-eye display and having a portion associated with a right-eye display; and

a forward-facing camera associated with the head-mounted display, the camera being configured to capture images coming from a portion of the external display,

wherein the instructions executed by the mobile application further comprise: responsively determining that the captured images comprise a first portion of three-dimensional (3-D) content as displayed on the external display as alternating images of a stereoscopic image sequence, the first portion of 3-D content being related to an AR presentation; detecting a frequency of the captured images by analyzing sequential images of the captured images; and shuttering, at the detected frequency, the head-mounted display in synchronization with the images of the stereoscopic image sequence while also presenting left-eye imagery of a second portion of content related to the AR presentation in the portion of the head-mounted display associated with the left-eye display, and right-eye imagery of the second portion of content related to the AR presentation in the portion of the head-mounted display associated with the right-eye display.

7. The apparatus of claim 6 , wherein the streaming module is configured to display a third virtual element on a second external display, the third virtual element partitioned from the 3D content.

8. The apparatus of claim 7 , wherein the second external display is an active stereo display having a second display frequency.

9. An apparatus, wherein the apparatus is configured to connect to an external display and is further configured to display a first virtual element on the external display, the apparatus comprising:

a processor;

a non-transitory computer-readable medium storing instructions that are operative, when executed by the processor, to cause the apparatus to:

intercept a set of three-dimensional (3D) content; and

partition the set of 3D content into the first virtual element and a second virtual element;

a head-mounted display configured to display the second virtual element, the head-mounted display having a portion associated with a left-eye display and having a portion associated with a right-eye display; and

a forward-facing camera associated with the head-mounted display, the camera being configured to capture images coming from a portion of the external display;

wherein the instructions are further operative, when executed by the processor, to further cause the apparatus to:

responsively determine that the captured images comprise a first portion of 3D content as displayed on the external display as alternating images of a stereoscopic image sequence, the first portion of 3D content being related to an AR presentation;

detect a frequency of the captured images by images by analyzing sequential images of the captured images; and

shutter, at the detected frequency, the head-mounted display in synchronization with the images of the stereoscopic image sequence while also presenting left-eye imagery of a second portion of content related to the AR presentation in the portion of the head-mounted display associated with the left-eye display, and right-eye imagery of the second portion of content related to the AR presentation in the portion of the head-mounted display associated with the right-eye display.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2023
From: PCMS HOLDINGS, INC.
To: INTERDIGITAL VC HOLDINGS, INC.
Reel/Frame 062900/0645 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2018
From: HARVIAINEN, TATU V. J.
To: PCMS HOLDINGS, INC.
Reel/Frame 047061/0973 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2018
From: TEKNOLOGIAN TUTKIMUSKESKUS VTT OY
To: PCMS HOLDINGS, INC.
Reel/Frame 046854/0640 →
Cited By (1)
US 12,260,493