Content creation platform for XR devices
A content creation system for extended Reality (XR) systems. The content creation system receives motion data of an XR device and generates trajectory data of a trajectory within a 3D environment model of a real-world scene based on the motion data where the trajectory simulates the motion of the XR device within the real-world scene. The content creation system receives user interaction event data and generates simulated sensor data based on the trajectory data, the 3D environment model, and the user interaction event data. The content creation system generates simulated tracking data based on the simulated sensor data and determines simulated power consumption data and thermal condition data based on operation of the computer vision component while generating the simulated tracking data. The content creation system generates a display from a user's perspective of the 3D environment model along with the simulated power consumption and thermal data.
1. A computer-implemented method comprising:
receiving, by a content creation platform, motion data of motion of an XR device;
generating, by the content creation platform, trajectory data of a trajectory of the XR device within a 3D environment model of a 3D environment based on the motion data, the data of the trajectory simulating the motion of the XR device within the 3D environment;
receiving, by the content creation platform, user interaction event data;
generating, by a data simulation platform, simulated sensor data based on the data of the trajectory of the XR device, the 3D environment model, and the user interaction event data, the simulated sensor data including simulated image data and simulated pose data of one or more sensors of the XR device;
generating, by a computer vision component within an operating system emulator, simulated tracking data of the trajectory simulating the motion of the XR device within the 3D environment using the simulated sensor data, one or more simulated operational conditions, and computer vision methodologies selected from the group consisting of Harris corner detection, Shi-Tomasi corner detection, Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB);
determining, by an operating system emulator, simulated energy consumption data of one or more computing devices of the computer vision component and the one or more sensors of the XR device based on operation of the computer vision component while generating the simulated tracking data; and
determining, by the operating system emulator using the energy consumption data, simulated thermal condition data including a thermal load and a consequent thermal condition including temperatures of portions of the XR device associated with the one or more computing devices and the one or more sensors; and
providing, by the content creation platform to a user, the simulated energy consumption data and the simulated thermal condition data of the XR device.
2. The computer-implemented method of claim 1 , wherein receiving the motion data of the XR device comprises:
capturing the data of the motion of the XR device as the XR device is moved through a real-world scene by a user.
3. The computer-implemented method of claim 1 , wherein receiving the motion data of the XR device comprises:
selecting the data of the motion from a listing of stored data.
4. The computer-implemented method of claim 1 , wherein the user interaction event is associated with a time event.
5. The computer-implemented method of claim 1 , wherein the user interaction event is associated with a location within the the 3D environment.
6. The computer-implemented method of claim 1 , wherein the user interaction event is a gesture made by a user.
7. The computer-implemented method of claim 1 , wherein generating the trajectory data of the trajectory within the 3D environment model further comprises:
changing an illumination parameter of the 3D environment model.
8. The computer-implemented method of claim 1 , wherein the simulated sensor data includes camera data, Inertial Motion Unit data, and Global Positioning Sensor data.
9. The computer-implemented method of claim 1 , wherein the simulated tracking data includes device pose data and hand recognition result data.
10. The computer-implemented method of claim 1 , wherein generating, by the computer vision component within the operating system emulator, simulated tracking data further comprises:
generating, by the operating system emulator, simulated operating system events affecting an operation of the computer vision component based on simulated environmental changes over time.
11. The computer-implemented method of claim 10 , wherein the simulated operating system events include a thermal throttling event, a network condition change event, an ambient noise event and a display adjustment event.
12. A machine comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
receiving, by a content creation platform, motion data of motion of an XR device;
generating, by the content creation platform, trajectory data of a trajectory of the XR device within a 3D environment model of a 3D environment based on the motion data, the data of the trajectory simulating the motion of the XR device within the 3D environment;
receiving, by the content creation platform, user interaction event data;
generating, by a data simulation platform, simulated sensor data based on the data of the trajectory of the XR device, the 3D environment model, and the user interaction event data, the simulated sensor data including simulated image data and simulated pose data of one or more sensors of the XR device;
generating, by a computer vision component within an operating system emulator, simulated tracking data of the trajectory simulating the motion of the XR device within the 3D environment using the simulated sensor data, one or more simulated operational conditions, and computer vision methodologies selected from the group consisting of Harris corner detection, Shi-Tomasi corner detection, Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB);
determining, by an operating system emulator, simulated energy consumption data of one or more computing devices of the computer vision component and the one or more sensors of the XR device on operation of the computer vision component while generating the simulated tracking data; and
determining by the operating system emulator using the energy consumption data, simulated thermal condition data including a thermal load and a consequent thermal condition including temperatures of portions of the XR device associated with the one or more computing devices and the one or more sensors; and
providing, by the content creation platform to a user, the simulated energy consumption data and the simulated thermal condition data of the XR device.
13. The machine of claim 12 , wherein receiving data of the motion of the XR device comprises:
capturing the data of the motion of the XR device as the XR device is moved through a real-world scene by a user.
14. The machine of claim 12 , wherein receiving data of the motion of the XR device comprises:
selecting the data of the motion from a listing of stored data.
15. The machine of claim 12 , wherein the user interaction event is associated with a time event.
16. The machine of claim 12 , wherein the user interaction event is associated with a location within the the 3D environment.
17. The machine of claim 12 , wherein the user interaction event is a gesture made by a user.
18. The machine of claim 12 , wherein generating the trajectory data of the trajectory within the 3D environment model further comprises:
changing an illumination parameter of the 3D environment model.
19. The machine of claim 12 , wherein the simulated sensor data includes camera data, Inertial Motion Unit data, and Global Positioning Sensor data.
20. A non-transitory machine-readable storage medium, the machine-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:
receiving, by a content creation platform, motion data of motion of an XR device;
generating, by the content creation platform, trajectory data of a trajectory of the XR device within a 3D environment model of a 3D environment based on the motion data, the data of the trajectory simulating the motion of the XR device within the 3D environment;
receiving, by the content creation platform, user interaction event data;
generating, by a data simulation platform, simulated sensor data based on the data of the trajectory of the XR device, the 3D environment model, and the user interaction event data, the simulated sensor data including simulated image data and simulated pose data of one or more sensors of the XR device;
generating, by a computer vision component within an operating system emulator, simulated tracking data of the trajectory simulating the motion of the XR device within the 3D environment using the simulated sensor data, one or more simulated operational conditions, and computer vision methodologies selected from the group consisting of Harris corner detection, Shi-Tomasi corner detection, Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB);
determining, by an operating system emulator, simulated energy consumption data of one or more computing devices of the computer vision component and the one or more sensors of the XR device based on operation of the computer vision component while generating the simulated tracking data; and
determining by the operating system emulator using the energy consumption data, simulated thermal condition data including a thermal load and a consequent thermal condition including temperatures of portions of the XR device associated with the one or more computing devices and the one or more sensors; and
providing, by the content creation platform to a user, the simulated energy consumption data and the simulated thermal condition data of the XR device.