Rendering a virtual object in spatial alignment with a pose of an electronic device
A method of rendering a virtual object by a rendering device includes establishing a communication channel with a mobile device; determining or receiving initial pose information of the mobile device, the pose information defining an initial position and an initial orientation; receiving device information via the communication channel, the device information including orientation, motion, and/or position information for updating the initial pose of the mobile device; using the initial pose information and the device information to determine a current pose of the mobile device, the current pose defining a current position and orientation; receiving first content data from a server system, the first content data representing a virtual object; and, rendering the first content data in the viewport of the rendering device based on the current pose information, the current pose information being used to render the virtual object in spatial alignment with the mobile device.
1. Method of rendering a virtual object by a rendering device, the method comprising:
establishing by the rendering device a communication channel with a mobile electronic device, the mobile electronic device being an electronic display device, the electronic display device having a physical display and a screen plane, the electronic display device comprising an inertial measurement unit, IMU;
determining or receiving initial pose information of the electronic display device, the initial pose information defining an initial position and an initial orientation;
receiving device information via the communication channel, the device information including orientation information, motion information, and/or position information for updating the initial pose of the electronic display device, at least part of the orientation information and/or motion information being generated by the IMU, the device information further including at least one of information about physical dimensions of the electronic display device and/or physical dimensions of the physical display of the electronic display device;
using the initial pose information and the device information to determine a current pose of the electronic display device, the current pose defining a current position and orientation;
receiving first content data from a server system, the first content data representing a virtual object, the virtual object being a virtual plane; and,
rendering the first content data in a viewport of the rendering device based on the current pose, wherein the current pose is used to spatially align the virtual object in co-planar spatial alignment with the screen plane of the electronic display device for extending the physical display of the electronic display device.
2. Method according to claim 1 wherein the device information further includes at least one of: information about a screen ratio of the electronic display device, color information associated with the physical display of the electronic display device, a model number of the electronic display device, information of a presentation time and/or clock time when the orientation, motion and/or position of the electronic display device was measured.
3. Method according to claim 1 wherein determining initial pose information further includes:
using a depth sensor or a depth camera to determine a distance between the electronic display device and the rendering device; and,
determine the initial position of the electronic display device based on the distance.
4. Method according to claim 3 wherein determining initial pose information further includes:
determining an initial orientation of the electronic display device based on tracking the electronic display device in pictures generated by a camera of the rendering device and based on image processing of the tracked electronic display device in the pictures; or,
receiving an initial orientation of the electronic display device via the communication channel, the initial orientation being determined by the inertial measurement unit of the electronic display device.
5. Method according to claim 1 wherein the communication channel is a wireless communication channel between the electronic display device and the rendering device.
6. Method according to claim 1 wherein the communication channel between the electronic display device and the rendering device is established via a server system.
7. Method according to claim 1 wherein the virtual object represents a virtual display extension of the physical display of the electronic display device, wherein the virtual display extension provides an extended view of content that is played back by the electronic display device.
8. A rendering device comprising:
a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform executable operations comprising:
establishing a communication channel with a mobile electronic device, the mobile electronic device being an electronic display device, the electronic display device comprising an inertial measurement unit, IMU;
determining or receiving initial pose information of the electronic display device, the initial pose information defining an initial position and an initial orientation;
receiving device information via the communication channel, the device information including orientation information, motion information, and/or position information for updating the initial pose of the electronic display device, the device information further including at least one of information about the physical dimensions of the electronic display device and/or physical dimensions of the display of the electronic display device;
using the initial pose information and the device information to determine a current pose of the electronic display device, the current pose defining a current position and orientation, at least part of the orientation information and/or motion information being generated by the IMU;
receiving first content data from a server system, the first content data representing a virtual object, the virtual object being a virtual plane; and,
rendering the first content data in a viewport of the rendering device based on the current pose, wherein the current pose is used to spatially align the virtual object in co-planar spatial alignment with the screen plane of the electronic display device for extending the physical display of the electronic display device.
9. Method according to claim 6 , wherein the method comprises receiving synchronization information via the communication channel for synchronization of the rendering of the first display extension and the content.
10. Computer program product comprising a non-transitory computer readable medium containing software code portions configured for, when run in memory of a computer, executing the method steps according to claim 1 .
11. Rendering device according to claim 8
wherein the device information further includes at least one of: information about a screen ratio of the electronic display device, color information associated with the physical display of the electronic display device, a model number of the electronic display device, information of a presentation time and/or clock time when the orientation, motion and/or position of the electronic display device was measured.
12. Rendering device according to claim 8 wherein determining initial pose information further includes:
using a depth sensor or a depth camera to determine a distance between the electronic display device and the rendering device; and,
determine the initial position of the electronic display device based on the distance.
13. Rendering device according to claim 12 wherein determining initial pose information further includes:
determining an initial orientation of the electronic display device based on tracking the electronic display device in pictures generated by a camera of the rendering device and based on image processing of the tracked electronic display device in the pictures; or,
receiving an initial orientation of the electronic display device via the communication channel, the initial orientation being determined by the inertial measurement unit of the electronic display device.
14. Rendering device according to claim 8 wherein the communication channel is a wireless communication channel between the electronic display device and the rendering device.
15. Rendering device according to claim 6 , wherein the virtual object represents a virtual display extension of the physical display of the electronic display device, wherein the virtual display extension provides an extended view of content that is played back by the mobile display device, the rendering device further configured for receiving synchronization information via the communication channel for synchronization of the rendering of the first display extension and the content.