IP Library Granted Patent US 9,811,908
Granted Patent B2
US 9,811,908 · App. 14/896,859 · Granted Nov 7, 2017

Head-mountable apparatus and systems

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,811,908
App. No.
14/896,859
Granted
Nov 7, 2017
Kind
B2
Abstract

A head mountable display (HMD) is provided that has an upward facing camera associated with a wide angle (such as a fisheye) lens arranged to capture images of the environment of the HMD. The HMD also includes an image comparator arranged to derive an indication of yaw of the HMD from images captured by the upward facing camera at different points in time. The image comparator may be configured to perform two or more detection techniques to detect HMD yaw, and to combine the results of the two or more detection techniques. The HMD may also include a tilt detector. Here, the image comparator may be configured to ignore one or more portions of a captured image in dependence upon a current tilt of the HMD as detected by the tilt detector.

Claims (51)

1. A head mountable display (HMD) comprising:

an upward facing camera associated with a wide angle lens configured to capture images of an environment of the HMD;

an image comparator operatively coupled to the upward facing camera, the image comparator being arranged to derive an indication of yaw of the HMD from images captured by the upward facing camera at different points in time; and

a processor configured to identify feature points in the images captured at different points in time, and to detect positions in the images of the identified feature points so as to provide an indication of rotational movement of the HMD between two respective images, in which the processor is configured to detect indications of the rotational movement from a plurality of pairs of image positions, each pair representing opposite viewpoints of the camera, and in which the processor is further configured to derive the indication of rotational movement from one or more pairs exhibiting a lowest difference between the viewpoints of that pair.

2. An HMD according to claim 1 , in which the lens is a fisheye lens.

3. An HMD according to claim 2 , in which the lens has a field of view of at least 140°.

4. An HMD according to claim 1 , in which the images are successively captured images.

5. An HMD according claim 1 , configured to apply an image warping process to convert the captured images into a linear image format.

6. An HMD according to claim 1 , wherein the processor is further configured to apply a correlation process as between the images captured at different points in time by the upward facing camera.

7. An HMD according to claim 1 , in which the image comparator is configured to perform two or more detection techniques to detect HMD yaw, and to combine the results of the two or more detection techniques.

8. An HMD according to claim 1 , further comprising:

a tilt detector;

and in which the image comparator is configured to ignore one or more portions of a captured image in dependence upon a current tilt of the HMD.

9. A system, comprising:

a head mountable display (HMD) including:

an upward facing camera associated with a wide angle lens configured to capture images of an environment of the HMD;

an image comparator operatively coupled to the upward facing camera, the image comparator being arranged to derive an indication of yaw of the HMD from images captured by the upward facing camera at different points in time; and

a processor configured to identify feature points in the images captured at different points in time, and to detect positions in the images of the identified feature points so as to provide an indication of rotational movement of the HMD between two respective images, in which the processor is configured to detect indications of the rotational movement from a plurality of pairs of image positions, each pair representing opposite viewpoints of the camera, and in which the processor is further configured to derive the indication of rotational movement from one or more pairs exhibiting a lowest difference between the viewpoints of that pair; and

one or more of a base device and an intermediate device operatively coupled to the HMD.

10. A method of operation of a head mountable display (HMD) having an upward facing camera associated with a wide angle lens arranged to capture images of the environment of the HMD, the method comprising:

deriving, by one or more processors, an indication of yaw of the HMD from images captured by the upward facing camera at different points in time;

identifying feature points in the images captured at the different points in time;

detecting positions in the images of the identified feature points so as to provide an indication of rotational movement of the HMD between two respective images;

detecting indications of the rotational movement from a plurality of pairs of image positions, each pair representing opposite viewpoints of the camera; and

deriving the indication of rotational movement from one or more pairs exhibiting a lowest difference between the viewpoints of that pair.

11. A non-transitory computer-readable storage medium on which computer readable instructions are stored, the instructions, when executed by one or more processors, causing the one or more processors to execute a method of operation of a head mountable display (HMD) having an upward facing camera associated with a wide angle lens arranged to capture images of the environment of the HMD, the method comprising:

deriving an indication of yaw of the HMD from images captured by the upward facing camera at different points in time;

identifying feature points in the images captured at the different points in time;

detecting positions in the images of the identified feature points so as to provide an indication of rotational movement of the HMD between two respective images;

detecting indications of the rotational movement from a plurality of pairs of image positions, each pair representing opposite viewpoints of the camera; and

deriving the indication of rotational movement from one or more pairs exhibiting a lowest difference between the viewpoints of that pair.

12. A head mountable display (HMD) comprising:

an upward facing camera associated with a wide angle lens configured to capture images of an environment of the HMD; and

an image comparator operatively coupled to the upward facing camera, the image comparator being arranged to derive an indication of yaw of the HMD from images captured by the upward facing camera at different points in time;

in which the image comparator is configured to detect inter-image scaling in two or more regions of the captured images, and not to use a region, in the detection of HMD yaw, in which inter-image scaling of more than a threshold scaling has been detected.

13. An HMD according to claim 12 , comprising a processor configured to apply a correlation process as between the images captured at different points in time by the upward facing camera.

14. An HMD according to claim 12 , further comprising:

a tilt detector;

and in which the image comparator is configured to ignore one or more portions of a captured image in dependence upon a current tilt of the HMD.

15. An HMD according to claim 12 , in which the images are successively captured images.

16. An HMD according claim 12 , configured to apply an image warping process to convert the captured images into a linear image format.

17. A system, comprising:

a head mountable display (HMD) including:

an upward facing camera associated with a wide angle lens configured to capture images of an environment of the HMD; and

an image comparator operatively coupled to the upward facing camera, the image comparator being arranged to derive an indication of yaw of the HMD from images captured by the upward facing camera at different points in time;

in which the image comparator is configured to detect inter-image scaling in two or more regions of the captured images, and not to use a region, in the detection of HMD yaw, in which inter-image scaling of more than a threshold scaling has been detected; and

one or more of a base device and an intermediate device operatively coupled to the HMD.

18. A method of operation of a head mountable display (HMD) having an upward facing camera associated with a wide angle lens arranged to capture images of the environment of the HMD, the method comprising:

deriving, by one or more processors, an indication of yaw of the HMD from images captured by the upward facing camera at different points in time;

detecting inter-image scaling in two or more regions of the captured images; and

not using a region, in the detection of HMD yaw, in which inter-image scaling of more than a threshold scaling has been detected.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2020
From: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 052167/0398 →
CHANGE OF NAME Recorded Jul 14, 2017
From: SONY COMPUTER ENTERTAINMENT EUROPE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 043198/0110 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2015
From: RAGHOEBARDAJAL, SHARWIN WINESH; BICKERSTAFF, IAN HENRY
To: SONY COMPUTER ENTERTAINMENT EUROPE LIMITED
Reel/Frame 037250/0026 →