Sensor emulation
Various implementations disclosed herein include devices, systems, and methods that are capable of executing an application on a head-mounted device (HMD) having a first image sensor in a first image sensor configuration. In some implementations, the application is configured for execution on a device including a second image sensor in a second image sensor configuration different than the first image sensor configuration. In some implementations, a request is received from the executing application for image data from the second image sensor. Responsive to the request at the HMD, a pose of a virtual image sensor is determined, image data is generated based on the pose of the virtual image sensor, and the generated image data is provided to the executing application.
1 . A method comprising:
at a processor:
executing an application on a device having a first image sensor in a first image sensor configuration, the application configured for execution on a device comprising a second image sensor in a second image sensor configuration different than the first image sensor configuration;
receiving a request from the executing application for image data from the second image sensor in the second image sensor configuration; and
responsive to the request,
determining a pose of a virtual image sensor;
generating image data based on the pose of the virtual image sensor and image data from the first image sensor; and
providing the generated image data to the executing application.
2 . The method of claim 1 , wherein generating image data comprises:
modifying the image data from the first image sensor based on the pose of the virtual image sensor to provide the generated image data.
3 . The method of claim 2 , wherein modifying the obtained image data comprises performing point of view correction based on a pose of the first image sensor and the pose of the virtual image sensor.
4 . The method of claim 1 , wherein the generated image data simulates optical properties of the second image sensor.
5 . The method of claim 1 , wherein generating image data comprises generating an avatar based on the image data from the first image sensor.
6 . The method of claim 5 , further comprising sizing the avatar based on a size of a physical environment in which the device having the first image sensor is operating.
7 . The method of claim 1 , wherein the first image sensor comprises an inward facing image sensor or a downward facing image sensor.
8 . The method of claim 1 , further comprising:
presenting user selectable input controls in a 3D representation of the second image sensor in the second image sensor configuration.
9 . The method of claim 1 , further comprising:
presenting an operable 3D representation of an electronic device comprising the second image sensor in the second image sensor configuration.
10 . The method of claim 9 , further comprising:
generating a preview image of the generated image data near the 3D representation of the electronic device.
11 . The method of claim 9 , further comprising:
generating a preview image of the generated image data on the 3D representation of the electronic device.
12 . The method of claim 1 , wherein the second image sensor comprises a front-facing image sensor or a rear-facing image sensor.
13 . The method of claim 1 , wherein the second image sensor comprises a front-facing image sensor, and wherein generating image data comprises generating an avatar based on the image data from the first image sensor.
14 . The method of claim 1 , wherein the second image sensor comprises a rear-facing image sensor, and wherein generating image data comprises modifying the image data from the first image sensor based on the pose of the virtual image sensor to provide the generated image data.
15 . The method of claim 1 , wherein the application is executing in an extended reality (XR) environment.
16 . The method of claim 15 , wherein the generated image data comprises a virtual object from the XR environment as viewed by the virtual image sensor.
17 . The method of claim 1 , wherein executing the application comprises presenting a visual representation of the application, and wherein the pose of the virtual image sensor is based on a pose of the visual representation of the application.
18 . The method of claim 1 , wherein the request from the executing application for image data comprises a request for depth data.
19 . The method of claim 1 , further comprising receiving a request from the executing application for audio data.
20 . The method of claim 1 , wherein the executing application is an image processing application providing multiple segments of a communication session with a second device.
21 . The method of claim 1 , wherein the generated image data provides a perspective viewpoint of a user of the device having the first image sensor.
22 . The method of claim 21 , wherein the perspective viewpoint of the user of the device having the first image sensor comprises an avatar that dynamically imitates facial expressions of the user of the device having the first image sensor based on the image data from the first image sensor.
23 . The method of claim 1 , wherein the device having the first image sensor is a head-mounted device (HMD).
24 . A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
executing an application on a head-mounted device (HMD) having a first image sensor in a first image sensor configuration, the application configured for execution on a device comprising a second image sensor in a second image sensor configuration different than the first image sensor configuration;
receiving a request from the executing application for image data from the second image sensor in the second image sensor configuration; and
responsive to the request,
determining a pose of a virtual image sensor;
generating image data based on the pose of the virtual image sensor and image data from the first image sensor; and
providing the generated image data to the executing application.
25 . A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:
executing an application on a head-mounted device (HMD) having a first image sensor in a first image sensor configuration, the application configured for execution on a device comprising a second image sensor in a second image sensor configuration different than the first image sensor configuration;
receiving a request from the executing application for image data from the second image sensor in the second image sensor configuration; and
responsive to the request,
determining a pose of a virtual image sensor;
generating image data based on the pose of the virtual image sensor and image data from the first image sensor; and
providing the generated image data to the executing application.