IP Library › Granted Patent US 10,885,663
Granted Patent B2
US 10,885,663 · App. 16/302,110 · Granted Jan 5, 2021

Method for setting a viewing direction in a representation of a virtual environment

Inventors: Christopher Mutschler (Cadolzburg, DE); Tobias Feigl (Dachsbach, DE); Christian Daxer (Happurg, DE); Stephan Otto (Heroldsberg, DE); Bercea Cosmin-Ionut (Nuremberg, DE)
Assignee: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e. v.
G06T7/73G02B27/0172G06F3/011G06F3/012G06T3/0006G06T7/74G02B2027/0138G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,885,663
App. No.
16/302,110
Granted
Jan 5, 2021
Kind
B2
Abstract

A method for setting a direction of view in a representation of a virtual environment is disclosed. The method includes recording a known object in a real environment using a recording device. Further, the method includes determining a rotational offset of the direction of view in the representation of the virtual environment around a yaw axis of the representation of the virtual environment based on the recording of the object, a known position of the recording device in the real environment and a current direction of view in the representation of the virtual environment. The method further includes rotating the direction of view in the representation of the virtual environment by the rotational offset.

Claims (62)

1. A method for setting a direction of view in a representation of a virtual environment, comprising:

recording a known object in a real environment using a recording device;

determining a rotational offset of the direction of view in the representation of the virtual environment around a yaw axis of the representation of the virtual environment based on the recording of the object, a known position of the recording device in the real environment and a current direction of view in the representation of the virtual environment for a time instant t 0 ; and

rotating the direction of view in the representation of the virtual environment by the rotational offset,

wherein the current direction of view in the representation of the virtual environment for the time instant t 0 is based on measurement values of a gyroscope, a magnetometer and/or an accelerometer, and

wherein determining the rotational offset of the direction of view in the representation of the virtual environment around the yaw axis of the representation of the virtual environment comprises:

determining an orientation of the recording device in the real environment based on the recording of the object and the known position of the recording device in the real environment;

determining a target direction of view in the representation of the virtual environment for the time instant t 0 based on the orientation of the recording device in the real environment; and

determining the rotational offset of the direction of view in the representation of the virtual environment around the yaw axis of the representation of the virtual environment from the target direction of view in the representation of the virtual environment for the time instant to and the current direction of view in the representation of the virtual environment for the time instant t 0 .

2. The method according to claim 1 , wherein determining the orientation of the recording device in the real environment comprises determining a transformation which correlates at least a part of the recording of the object with at least a part of a comparison recording.

3. The method according to claim 1 , wherein determining the orientation of the recording device in the real environment further comprises:

detecting a feature of the object in the recording;

determining a position of the feature in the recording;

identifying a comparison feature from a plurality of comparison features of a database which corresponds to the feature of the object in the recording, wherein a position in the real environment is associated with each of the plurality of comparison features; and

determining the orientation of the recording device in the real environment based on the known position of the recording device in the real environment, the position of the feature in the recording and the position in the real environment associated with the identified comparison feature.

4. The method according to claim 3 , wherein when determining the orientation of the recording device in the real environment several features of the object are detected, wherein an orientation of the recording device in the real environment is determined for each of the several detected features of the object, and wherein the one of the orientations of the recording device in the real environment determined for the several detected features of the object which fulfils a quality criterion is determined to be the orientation of the recording device in the real environment.

5. The method according to claim 3 , wherein the plurality of comparison features comprises different features of the object.

6. The method according to claim 1 , wherein the object is a pattern attached to a vertical plane.

7. The method according to claim 1 , wherein the method further comprises outputting a representation of the virtual environment to a user and wherein the recording device is aligned in the straight direction of view of the user in the real environment.

8. The method according to claim 7 , wherein outputting the representation of the virtual environment to the user is executed via a display device mounted to a head of the user and wherein the display device further comprises the recording device.

9. The method according to claim 8 , wherein the display device comprises a mobile communications device and wherein the recording device is a camera of the mobile communication device.

10. The method according to claim 1 , wherein determining the orientation of the recording device in the real environment comprises:

determining, based on the known position of the recording device in the real environment, a comparison recording from a plurality of comparison recordings of a database, wherein an orientation of the comparison recording in the real environment is known;

determining a rotation of the recording of the object relative to the comparison recording;

determining the orientation of the recording device in the real environment based on the orientation of the comparison recording in the real environment and the rotation of the recording of the object relative to the comparison recording.

11. The method according to claim 10 , wherein the plurality of comparison recordings are binary recordings and wherein determining the rotation of the recording of the object relative to the comparison recording comprises:

converting the recording of the object into a binary recording of the object; and

determining the rotation of the binary recording of the object relative to the comparison recording.

12. The method according to claim 10 , wherein the method further comprises outputting the representation of the virtual environment to a user and wherein the recording device is aligned vertically to a straight direction of view of the user in the real environment.

13. The method according to claim 12 , wherein the object extends vertically exclusively above or exclusively below the user.

14. A method for setting a direction of view in a representation of a virtual environment, comprising:

recording a known object in a real environment using a recording device;

determining a rotational offset of the direction of view in the representation of the virtual environment around a yaw axis of the representation of the virtual environment based on the recording of the object and a current direction of view in the representation of the virtual environment for a time instant t 0 ; and

rotating the direction of view in the representation of the virtual environment by the rotational offset,

wherein the current direction of view in the representation of the virtual environment for the time instant t 0 is based on measurement values of a gyroscope, a magnetometer and/or an accelerometer, and

wherein determining the rotational offset of the direction of view in the representation of the virtual environment around the yaw axis of the representation of the virtual environment comprises:

determining an orientation of the recording device in the real environment based on an orientation of the object in the recording and a reference direction;

determining a target direction of view in the representation of the virtual environment for the time instant t 0 based on the orientation of the recording device in the real environment; and

determining the rotational offset of the direction of view in the representation of the virtual environment around the yaw axis of the representation of the virtual environment from the target direction of view in the representation of the virtual environment for the time instant t 0 and the current direction of view in the representation of the virtual environment for the time instant t 0 .

15. The method according to claim 14 , wherein determining the orientation of the recording device in the real environment comprises:

converting the recording of the object into a binary recording of the object;

detecting candidates for the object in the binary recording of the object;

determining a respective eccentricity of the candidates for the object; and

determining an orientation of a main axis of the one candidate whose eccentricity is above a threshold value and whose main axis is longer than main axes of the other candidates for the object with an eccentricity above the threshold value as an orientation of the object in the recording.

16. The method according to claim 14 , wherein determining the orientation of the recording device in the real environment further comprises:

converting the recording of the object into a binary recording of the object;

detecting circular objects in the binary recording of the object, wherein respective radii of the circular objects are included in a predetermined value range;

determining distances of the circular objects from one another;

determining the orientation of the object in the recording on the basis of the distances of the circular objects from one another.

17. The method according to claim 14 , wherein the method further comprises outputting the representation of the virtual environment to a user and wherein the recording device is aligned vertically to a straight direction of view of the user in the real environment.

18. The method according to claim 17 , wherein outputting the representation of the virtual environment to the user is executed via a display device mounted to a head of the user and wherein the display device further comprises the recording device.

19. The method according to claim 18 , wherein the display device comprises a mobile communications device and wherein the recording device is a camera of the mobile communications device.

20. The method according to claim 17 , wherein the object extends vertically exclusively above or exclusively below the user.

21. A method for setting a direction of view in a representation of a virtual environment, comprising:

recording an object arranged in a real environment at the body of a user using a recording device arranged at a head of the user at a first time instant and at a later second time instant;

determining a rotational offset of the direction of view in the representation of the virtual environment around a yaw axis of the representation of the virtual environment based on the recordings of the object at the first time instant and at the second time instant and measurement values of at least one further sensor mounted to the head of the user; and

rotating the direction of view in the representation of the virtual environment by the rotational offset.

22. The method according to claim 21 , wherein determining the rotational offset of the direction of view in the representation of the virtual environment around the yaw axis of the representation of the virtual environment further comprises:

determining a first rotation of the recording device around a yaw axis of the head of the user between the first time instant and the second time instant based on the recordings of the object at the first time instant and at the second time instant;

determining a second rotation of the recording device around the yaw axis of the head of the user between the first time instant and the second time instant based on the measurement values of at least one further sensor mounted to the head of the user; and

determining the rotational offset between the first rotation and the second rotation as the rotational offset of the direction of view in the representation of the virtual environment around the yaw axis of the representation of the virtual environment.

23. The method according to claim 21 , wherein the method further comprises outputting the representation of the virtual environment to the user via a display device mounted to the head of the user and wherein the display device further comprises the recording device and the at least one further sensor.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2018
From: MUTSCHLER, CHRISTOPHER; FEIGL, TOBIAS; DAXER, CHRISTIAN; OTTO, STEPHAN; COSMIN-IONUT, BERCEA
To: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 047520/0670 →
Priority Claims (1)
DE 10 2016 109 153 · May 18, 2016 · national
Continuity (1)
Related Publication 20190180471A1 · Jun 13, 2019