IP Library › Granted Patent US 11,526,004
Granted Patent B2
US 11,526,004 · App. 16/818,698 · Granted Dec 13, 2022

Head-mounted display device and operating method of the same

Inventors: Bonkon Koo (Suwon-si, KR); Jaewoo Ko (Suwon-si, KR); Wonwoo Lee (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G02B27/0093G02B27/017G06F3/013G06T7/521G06T7/593G06T19/006G06V10/25G02B2027/0178G06T2207/10028G06T2215/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,526,004
App. No.
16/818,698
Granted
Dec 13, 2022
Kind
B2
Abstract

A head-mounted display device includes an eye tracking sensor configured to obtain eye information by tracking both eyes of a user, a depth sensor configured to obtain depth information about one or more objects, and a processor configured to obtain information about a gaze point based on the eye information, and determine a measurement parameter of the depth sensor based on the information about the gaze point.

Claims (30)

1. A head-mounted display device comprising:

an eye tracking circuitry;

a depth sensor including a light emitter, wherein the light emitter comprises light sources disposed in a plurality of regions; and

one or more processors configured to:

control the eye tracking circuitry to obtain a direction of a left eye of a user and a direction of a right eye of the user,

obtain 2D location of a gaze point of the user based on the obtained direction of the left eye of the user and the obtained direction of the right eye of the user,

determine a region, among the plurality of regions, corresponding to the obtained 2D location of the gaze point of the user,

control the light sources disposed in the determined region to output light, and

obtain depth information about one or more objects, based on the output light by the light sources disposed in the determined region.

2. The head-mounted display device of claim 1 , wherein the one or more processors is further configured to obtain the depth measurement parameter of the depth sensor including at least one of a width of a pulse or a magnitude of light emitted by the light emitter based on the obtained depth information about the one or more objects, and

wherein the depth sensor is further configured to re-obtain the depth information about the one or more objects based on the obtained depth measurement parameter.

3. The head-mounted display device of claim 1 , wherein the depth sensor is further configured to obtain the depth information about the one or more objects by using at least one of a time of flight (TOF) method, a structured light (SL) method, or a stereo image (SI) method.

4. The head-mounted display device of claim 3 , wherein, when the depth sensor comprises a TOF depth sensor, the one or more processors is further configured to determine the depth information based on the 2D location information of the gaze point such that the light sources included in the light emitter disposed in the determined region are driven, and

wherein the depth sensor is further configured to obtain the depth information about the one or more objects by driving the light sources.

5. The head-mounted display device of claim 1 , further comprising a display displaying a real space comprising one or more objects, and

wherein the one or more processors is further configured to control the display to display at least one virtual object on the one or more objects based on the depth information about the one or more objects.

6. A method of operating an electronic device, the method comprising:

obtaining, via an eye tracking circuitry, a direction of a left eye of a user and a direction of a right eye of the user;

obtaining 2D location of a gaze point of the user based on the obtained direction of the left eye of the user and the obtained direction of the right eye of the user;

determining a region, among a plurality of regions, corresponding to the obtained 2D location of the gaze point of the user;

outputting light by driving light sources included in a light emitter of a depth sensor disposed in a region, among the plurality of regions, corresponding to the 2D location of the gaze point of the user; and

obtaining depth information about one or more objects, based on the output light by the light sources of the depth sensor disposed in the determined region.

7. The method of claim 6 , further comprising:

obtaining a depth measurement parameter of the depth sensor including at least one of a width of a pulse or a magnitude of light emitted by the light emitter based on the obtained depth information about the one or more objects; and

re-obtaining the depth information about the one or more objects based on the obtained depth measurement parameter.

8. The method of claim 6 , wherein the obtaining of the depth information about the one or more objects comprises obtaining the depth information about the one or more objects by using at least one of a time of flight (TOF) method, a structured light (SL) method, or a stereo image (SI) method.

9. The method of claim 8 , wherein the obtaining of the depth information comprises when obtaining the depth information using the TOF method, determining the depth information based on the 2D location information of the gaze point such that the light sources included in the light emitter disposed in the region corresponding to the 2D location information of the gaze point of the user are driven, and

the obtaining of the depth information about the one or more objects comprises obtaining the depth information about the one or more objects by driving the light sources.

10. A non-transitory computer-readable recording medium having recorded thereon a program for executing the method of claim 6 .

11. The head-mounted display device of claim 2 , wherein the depth measurement parameter of the depth sensor further includes a sensing speed of a reflected light by the depth sensor and a sensing sensitivity of depth sensor.

Priority Claims (1)
KR 10-2019-0106133 · Aug 28, 2019 · national
Continuity (3)
Continuation 16800414 · Feb 25, 2020
Provisional Application 62832544 · Apr 11, 2019
Related Publication 20200326536A1 · Oct 15, 2020
Cited By (1)
US 12,375,627