IP Library Granted Patent US 8,988,343
Granted Patent B2
US 8,988,343 · App. 13/853,767 · Granted Mar 24, 2015

Method of automatically forming one three-dimensional space with multiple screens

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,988,343
App. No.
13/853,767
Granted
Mar 24, 2015
Kind
B2
Abstract

Multiple electronic displays are arranged to define an ensemble of the user's choosing. The processor generates and displays a uniquely encoded emblem encoding a unique identifier for that display. A camera captures at least one image of the ensemble so that the encoded emblem for each display is captured. The captured images are then processed to extract the following attributes: a) unique identifier of the display; b) position of the encoded emblem relative to the reference coordinate system; and c) pointing direction of the encoded emblem relative to the reference coordinate system. The processor collectively processes the attributes for each of the emblem images to compute for each display a transformation matrix that selects a portion of the image data for display on that monitor and performs pointing direction correction so that the information presented on that display appears spatially consistent with the information presented on the other displays.

Claims (54)

1. A method for presenting information across multiple electronic displays, comprising:

arranged to define a non-coplanar ensemble;

generating and displaying on each display a uniquely encoded emblem of predetermined size and shape that encodes a unique identifier for that display;

optically capturing at least one digital image of the ensemble so that the encoded emblem for each display is captured from a vantage point spaced apart from the ensemble, the optically captured image having a captured size that varies from the predetermined size based on distance from vantage point to ensemble and having a captured shape that has foreshortened dimensions across the line of sight between vantage point and ensemble;

defining a reference coordinate system corresponding to the space occupied by the ensemble of electronic displays;

processing each of the captured encoded emblem images to extract the following attributes:

a) unique identifier of the display;

b) position of the encoded emblem relative to the reference coordinate system; and

c) three-dimensional orientation of the encoded emblem relative to the reference coordinate system, where the three-dimensional orientation is computed using the captured size and shape;

supplying the information to be presented as image data to be displayed across the ensemble of displays; and

collectively processing the attributes for each of the encoded emblem images to compute for each display a transformation matrix that selects a portion of the image data for display on that monitor and that performs three-dimensional orientation correction so that the information presented on that display appears spatially consistent with the information presented on the other displays.

2. The method of claim 1 wherein said generating and displaying on each display a uniquely encoded emblem step is performed by generating and displaying an encoded emblem of known size and shape.

3. The method of claim 1 wherein said unique identifier is expressed using a machine-readable code.

4. The method of claim 1 wherein said unique identifier is expressed using a QR code.

5. The method of claim 1 wherein said step of capturing at least one digital image is performed using a still camera.

6. The method of claim 1 wherein said step of capturing at least one digital image is performed using a video camera.

7. The method of claim 1 wherein said step of capturing at least one digital image is performed by using a processor programmed to control a camera.

8. The method of claim 1 wherein the position attribute is extracted using graphical processing to locate the center of the emblem.

9. The method of claim 1 wherein the three-dimensional orientation attribute is extracted using graphical processing to locate and measure angles defined by edges of the emblem.

10. The method of claim 1 wherein the position attribute is extracted using graphical processing to locate the center of the emblem and using this center as x,y coordinates that are projected into said reference coordinate system.

11. The method of claim 1 further comprising generating and displaying on each display a uniquely encoded emblem of predetermined fixed size and shape.

12. The method of claim 11 further comprising:

using a camera to capture at least one image of the encoded emblem;

using graphical processing to determine the size of the captured encoded emblem;

comparing the size of the captured encoded emblem with said predetermined fixed size to determine a parameter indicative of distance from the camera to the displayed encoded emblem; and

using said parameter indicative of distance as a z coordinate of said position attribute.

13. The method of claim 1 further comprising:

using a camera to capture multiple images of the encoded emblem for each display, where at least some of the multiple images are taken from different vantage points relative to the space occupied by the ensemble; and

processing the multiple images by triangulation to determine the position attribute in the reference coordinate system.

14. The method of claim 1 wherein the reference coordinate system is a three-dimensional reference system.

15. The method of claim 1 wherein said step of supplying information to be presented comprises generating different information for each of a viewer's left and right eyes.

16. The method of claim 15 further comprising providing said displays with at least one associated camera that captures information indicative of the location of a user while viewing said ensemble and using said user location information to modify said data streams to account for the position of the user's left and right eyes.

17. The method of claim 15 further comprising providing at least a plurality of said displays each with an associated camera that captures information indicative of the location of a user while viewing said ensemble and combining the information indicative of the location of a user from said associated cameras into a single user head-tracking estimate.

18. The method of claim 17 wherein said combining step is performed by generating weighting factors based on head-tracking confidence from each associated camera and applying said weighting factors when combining the information indicative of user location from each of said associated cameras.

19. The method of claim 1 wherein said step of supplying information to be presented comprises generating different information data streams for each of a viewer's left and right eyes; and applying said transformation matrix separately to each of said data streams.

20. A method for ascertaining the positions and orientations of a plurality of electronic displays arranged in a non-coplanar ensemble, comprising:

generating and displaying on each display a uniquely encoded emblem of predetermined size and shape that encodes a unique identifier for that display;

optically capturing at least one digital image of the ensemble so that the encoded emblem for each display is captured from a vantage point spaced apart from the ensemble, the optically captured image having a captured size that varies from the predetermined size based on distance from vantage point to ensemble and having a captured shape that has foreshortened dimensions across the line of sight between vantage point and ensemble;

defining a reference coordinate system corresponding to the space occupied by the ensemble of electronic displays;

processing each of the captured encoded emblem images to extract the following attributes:

a) unique identifier of the display;

b) position of the encoded emblem relative to the reference coordinate system; and

c) three-dimensional orientation of the encoded emblem relative to the reference coordinate system, where the three-dimensional orientation is computed using the captured size and shape; and

supplying the information to be presented as image data to be displayed across the ensemble of displays.

21. An apparatus for presenting information across multiple electronic displays that are arranged as a non-coplanar ensemble, comprising:

a processor programmed to generate a uniquely encoded emblem of predetermined size and shape that encodes a unique identifier for each display, the processor having associated graphics circuitry to send the encoded emblem to the display for visual presentation;

a camera positioned to optically capture at least one digital image of the ensemble so that the encoded emblem for each display is captured from a vantage point spaced apart from the ensemble, the optically captured image having a captured size that varies from the predetermined size based on distance from vantage point to ensemble and having a captured shape that has foreshortened dimensions across the line of sight between vantage point and ensemble;

the processor being programmed to define a reference coordinate system corresponding to the space occupied by the ensemble of electronic displays;

the processor being further programmed to process each of the captured encoded emblem images to extract the following attributes:

a) unique identifier of the display;

b) position of the encoded emblem relative to the reference coordinate system; and

c) three-dimensional orientation of the encoded emblem relative to the reference coordinate system, where the three-dimensional orientation is computed using the captured size and shape;

the processor having an input port to receive the information to be presented as image data to be displayed across the ensemble of displays; and

the processor being programmed to collectively process the attributes for each of the encoded emblem images to compute for each display a transformation matrix that selects a portion of the image data for display on that monitor and that performs three-dimensional orientation correction so that the information presented on that display appears spatially consistent with the information presented on the other displays.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE ERRONEOUSLY FILED APPLICATION NUMBERS 13/384239, 13/498734, 14/116681 AND 14/301144 PREVIOUSLY RECORDED ON REEL 034194 FRAME 0143. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 24, 2020
From: PANASONIC CORPORATION
To: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Reel/Frame 056788/0362 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2014
From: PANASONIC CORPORATION
To: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Reel/Frame 034194/0143 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2013
From: FEI, YUE; MELLE, ANDREA; KRYZE, DAVID
To: PANASONIC CORPORATION
Reel/Frame 031238/0953 →