VR content sickness evaluating apparatus using deep learning analysis of motion mismatch and method thereof

1. A virtual reality (VR) content sickness evaluating apparatus, the apparatus comprising:

a memory; and

at least one processor connected to the memory, and configured to execute computer readable instructions included in the memory,

wherein the at least one processor is configured to

analyze visual recognition information according to a visual recognition motion feature based on a change in motion of VR content;

analyze posture recognition information according to a posture recognition motion feature based on a change in motion of a user, the change being received from a sensing module; and

determine a degree of sickness induced by the VR content from a difference between the visual recognition information and the posture recognition information,

wherein the at least one processor is configured to:

analyze a change in motion of the VR content the user recognizes with his or her eyes, using a convolutional neural network (CNN) and a convolutional long short-term memory (conv LSTM); and

extract the visual recognition information of motion information according to the visual recognition motion feature of a temporal factor and a spatial factor using the CNN and the conv LSTM.

2. The apparatus of claim 1 , wherein the at least one processor is configured to:

analyze a temporal change in head location information received from the sensing module including at least one or more of a motion sensor configured to obtain motion information of the user and a VR player worn by the user, using an LSTM; and

extract the posture recognition motion feature using the LSTM.

3. A virtual reality (VR) content sickness evaluating apparatus, the apparatus comprising:

a memory; and

at least one processor connected to the memory, and configured to execute computer readable instructions included in the memory,

wherein the at least one processor is configured to

analyze visual recognition information according to a visual recognition motion feature based on a change in motion of VR content;

analyze posture recognition information according to a posture recognition motion feature based on a change in motion of a user, the change being received from a sensing module; and

determine a degree of sickness induced by the VR content from a difference between the visual recognition information and the posture recognition information,

wherein the at least one processor is configured to:

analyze a temporal change characteristic of three-dimensional (3D) space coordinates received from the sensing module depending on a change in motion of the user and the posture recognition motion feature; and

extract the posture recognition information of motion information.

4. The apparatus of claim 3 , wherein the at least one processor is configured to:

generate a visual recognition motion map which is a motion map from an extracted visual recognition motion feature, and

wherein the at least one processor is configured to:

generate a posture recognition motion map which is a motion map from an extracted posture recognition motion feature.

5. The apparatus of claim 4 , wherein the at least one processor is configured to:

generate a motion mismatch map through a difference between the visual recognition motion map and the posture recognition motion map.

6. The apparatus of claim 5 , wherein the at least one processor is configured to:

discriminate real motion or fake motion for the generated motion mismatch map by discriminating between the generated motion mismatch map and a real motion mismatch map.

7. The apparatus of claim 6 , wherein the at least one processor is configured to:

quantify and evaluate a sickness degree obtained from difference data between the visual recognition information and the posture recognition information and a sickness obtained from the generated motion mismatch map.

8. A method of evaluating a VR content sickness evaluating apparatus, the method comprising:

analyzing visual recognition information according to a visual recognition motion feature based on a change in motion of VR content;

analyzing posture recognition information according to a posture recognition motion feature based on a change in motion of a user, the change being received from a sensing module; and

determining a degree of sickness induced by the VR content from a difference between the visual recognition information and the posture recognition information,

wherein the analyzing of the visual recognition information comprises:

analyzing a change in motion of the VR content the user recognizes with his or her eyes, using a CNN and a conv LSTM; and

extracting the visual recognition information of motion information according to the visual recognition motion feature of a temporal factor and a spatial factor using the CNN and the conv LSTM.

9. The method of claim 8 , wherein the analyzing of the posture recognition information comprises:

analyzing a temporal change in head location information received from the sensing module including at least one or more of a motion sensor configured to obtain motion information of the user and a VR player worn by the user, using an LSTM; and

extracting the posture recognition motion feature using the LSTM.

10. The method of claim 9 , wherein the analyzing of the posture recognition information comprises:

analyzing a temporal change characteristic of 3D space coordinates received from the sensing module depending on a change in motion of the user and the posture recognition motion feature; and

extracting the posture recognition information of motion information.

11. The method of claim 8 , wherein the determining of the degree of the sickness induced by the VR content comprises:

generating a motion mismatch map through a difference between a visual recognition motion map and a posture recognition motion map;

discriminating between the generated motion mismatch map and a real motion mismatch map; and

quantifying and evaluating a sickness degree obtained from difference data between the visual recognition information and the posture recognition information and a sickness obtained from the generated motion mismatch map.

12. The method of claim 11 , wherein the discriminating between the generated motion mismatch map and the real motion mismatch map comprises:

discriminating real motion or fake motion for the generated motion mismatch map.