Method for generating pass-through view according to tracker status and host

1 . A method for generating a pass-through view according to a tracker status, applied to a host, comprising:

in response to determining that the tracker status of a tracker satisfies a predetermined condition, generating, by the host, a target depth map based on a first depth map and a tracker information, wherein the first depth map is associated with a field of view (FOV) of the host, and wherein the tracker information is associated with the tracker, wherein the generating, by the host, the target depth map further comprising:

obtaining the first depth map associated with the FOV of the host, wherein the first depth map comprises at least one valid region;

determining the tracker information associated with the tracker;

determining scene information based on the at least one valid region within the first depth map and the tracker information;

generating a second depth map based on the scene information; and

determining the second depth map as the target depth map; and

rendering, by the host, the pass-through view based on an image associated with the FOV of the host, a camera parameter, and the target depth map.

2 . The method according to claim 1 , wherein the tracker is a trackable object, and the method further comprises:

in response to determining that the tracker is not being used, determining that the tracker status of the tracker satisfies the predetermined condition.

3 . The method according to claim 1 , wherein the tracker is a wearable device, and the method further comprises:

in response to determining that the tracker is being put on or taken off, determining that the tracker status of the tracker satisfies the predetermined condition.

4 . The method according to claim 1 , further comprising:

in response to determining that a tracker distance between the tracker and the host is less than a distance threshold, determining that the tracker status of the tracker satisfies the predetermined condition.

5 . The method according to claim 1 , wherein determining the tracker information associated with the tracker comprises:

tracking a tracker pose of a reference point on the tracker;

obtaining a tracker model of the tracker, wherein the tracker model comprises a plurality of model points comprising the reference point;

determining a pose of each of the plurality of model points based on the tracker pose of the reference point and a relative position between the reference point and each of the plurality of model points; and

determining the plurality of model points with the corresponding pose as the tracker information associated with the tracker.

6 . The method according to claim 1 , wherein the tracker information comprises a plurality of tracker points, the scene information comprises a plurality of scene points, and determining the scene information based on the at least one valid region within the first depth map and the tracker information comprises:

determining a plurality of first points via projecting the at least one valid region within the first depth map into a space;

determining a plurality of second points via removing an interfering point from the plurality of first points; and

determining the plurality of second points as the plurality of scene points.

7 . The method according to claim 6 , wherein before determining the plurality of second points as the plurality of scene points, the method further comprises:

adding a plurality of historical scene points into the plurality of second points.

8 . The method according to claim 6 , wherein before determining the plurality of second points as the plurality of scene points, the method further comprises:

applying a temporal filter to the plurality of second points.

9 . The method according to claim 1 , wherein the tracker information comprises a plurality of tracker points, the scene information comprises a plurality of scene points, and generating the second depth map based on the scene information comprises:

generating an intermediate depth map via projecting the plurality of scene points to a reference position; and

determining the second depth map via smoothing the intermediate depth map.

10 . The method according to claim 1 , wherein before determining the scene information based on the at least one valid region within the first depth map and the tracker information, the method further comprises:

updating the first depth map based on the tracker information.

11 . The method according to claim 10 , wherein the tracker information comprises a plurality of tracker points, and updating the first depth map based on the tracker information comprises:

determining a tracker region within the first depth map via projecting the plurality of tracker points onto the first depth map; and

updating the first depth map via setting the tracker region as invalid.

12 . A method for generating a pass-through view according to a tracker status, applied to a host, comprising:

in response to determining that the tracker status of a tracker satisfies a predetermined condition, generating, by the host, a target depth map based on based on a first depth map and a tracker information, wherein the first depth map is associated with a field of view (FOV) of the host, and wherein the tracker information is associated with the tracker, wherein the generating, by the host, the target depth map further comprising:

obtaining the first depth map associated with the FOV of the host;

determining the tracker information associated with the tracker;

modifying the first depth map into a second depth map according to the tracker information, wherein the second depth map comprises at least one valid region; and

determining the target depth map based on the second depth map; and

rendering, by the host, the pass-through view based on an image associated with the FOV of the host, a camera parameter, and the target depth map.

13 . The method according to claim 12 , wherein the tracker information comprises a plurality of tracker points, and modifying the first depth map into the second depth map according to the tracker information comprises:

determining a tracker region within the first depth map via projecting the plurality of tracker points onto the first depth map;

determining another tracker region within the first depth map based on the tracker region, and modifying the first depth map into the second depth map via setting the another tracker region as invalid.

14 . The method according to claim 12 , wherein determining the target depth map based on the second depth map comprises:

determining scene information based on the at least one valid region within the second depth map;

generating an intermediate depth map based on the scene information;

determining the target depth map via smoothing the intermediate depth map.

15 . The method according to claim 14 , wherein the tracker information comprises a plurality of tracker points, the scene information comprises a plurality of scene points, wherein generating an intermediate depth map based on the scene information comprises:

generating the intermediate depth map via projecting the plurality of scene points to a reference position.

16 . The method according to claim 14 , wherein the scene information comprises a plurality of scene points, and determining the scene information based on the at least one valid region within the second depth map comprises:

determining the plurality of scene points via projecting the at least one valid region within the second depth map into a space.

17 . The method according to claim 12 , wherein determining the target depth map based on the second depth map comprises:

determining a transforming function based on a reference position and the camera parameter;

transforming the second depth map into an intermediate depth map via applying the transforming function to the second depth map; and

determining the target depth map via smoothing the intermediate depth map.

18 . The method according to claim 1 , wherein generating the target depth map based on the first depth map and the tracker information comprises:

determining a predetermined depth map as the target depth map, wherein the predetermined depth map has a uniform depth value.

19 . A host, comprising:

a non-transitory storage circuit, storing a program code; and

a processor, coupled to the non-transitory storage circuit and accessing the program code to perform:

in response to determining that a tracker status of a tracker satisfies a predetermined condition, generating a target depth map based on a first depth map and a tracker information, wherein the first depth map is associated with a field of view (FOV) of the host, and wherein the tracker information is associated with the tracker;

obtaining the first depth map associated with the FOV of the host, wherein the first depth map comprises at least one valid region;

determining the tracker information associated with the tracker;

determining scene information based on the at least one valid region within the first depth map and the tracker information;

generating a second depth map based on the scene information;

determining the second depth map as the target depth map; and

rendering a pass-through view based on an image associated with the FOV of the host, a camera parameter, and the target depth map.