Granted Patent B2

US 12,737,982 · App. 18/588,325 · Granted Sep 15, 2026

Inventors: Sergio Cantero Clares (New Taipei City, TW); Shih-Hao Lin (New Taipei City, TW)

Assignee: ACER INCORPORATED

G06T17/20G06F3/012G06T7/50G06T15/04G06T2200/24G06T2207/10016

Abstract

Claims (137)

1 . A method for generating a three-dimensional (3D) interactive scene, the method comprising:

executing a first process based on a two-dimensional (2D) source image, wherein the 2D source image is loaded from a storage unit; and

generating the 3D interactive scene based on the 2D source image, wherein the first process comprises:

estimating a corresponding depth map for the 2D source image;

creating a 3D mesh based on the estimated depth map; and

executing a first loop iteratively to present the 3D interactive scene based on the 3D mesh and head pose information; and

executing a second process based on a 2D source video loaded from the storage unit, wherein the second process comprises:

determining whether a depth video corresponding to the 2D source video is stored in the storage unit, wherein the depth video comprises a depth frame sequence;

in an event that the depth video is stored in the storage unit, initializing the 3D mesh, and executing a second loop iteratively to present the 3D interactive scene based on the head pose information; and

in an event that the depth video is not stored in the storage unit, starting a background inference thread for generating the depth video and storing the generated depth video in the storage unit.

2 . The method as claimed in claim 1 , wherein the first loop comprises:

obtaining the head pose information;

calculating a set of head coordinates based on the head pose information;

constructing a transformation matrix based on the calculated set of head coordinates;

projecting each vertex of the 3D mesh onto a screen coordinate of an onscreen image using the transformation matrix;

coloring the onscreen image based on texture information corresponding to the 2D source image; and

causing a display unit to display the onscreen image.

3 . The method as claimed in claim 2 , wherein the first loop further comprises:

detecting a pause request;

in response to the pause request, pausing the iteration of the first loop and waiting for a resume request; and

in response to the resume request, resuming the next iteration of the first loop.

4 . The method as claimed in claim 3 , wherein the pause request is triggered in an event that an area proportion of the onscreen image obscured by windows of other programs exceeds a specified percentage.

5 . The method as claimed in claim 1 , wherein the second loop comprises:

loading a video frame of the 2D source video from the storage unit;

loading the corresponding depth frame for the video frame from the storage unit;

updating the 3D mesh based on the depth frame;

obtaining the head pose information;

calculating the set of head coordinates based on the head pose information;

constructing the transformation matrix based on the calculated set of head coordinates;

projecting each vertex of the updated 3D mesh onto the screen coordinate of the onscreen image using the transformation matrix;

coloring the onscreen image based on the texture information corresponding to the loaded video frame; and

causing the display unit to display the onscreen image.

6 . The method as claimed in claim 5 , wherein the second loop further comprises:

detecting a pause request;

in response to the pause request, pausing the iteration of the second loop and waiting for a resume request; and

in response to the resume request, resuming the next iteration of the second loop.

7 . The method as claimed in claim 6 , wherein the pause request is triggered in an event that an area proportion of the onscreen image obscured by windows of other programs exceeds a specified percentage.

8 . The method as claimed in claim 1 , wherein in an event that the depth video is not stored in the storage unit, the second process further comprises:

starting the background inference thread while initializing the 3D mesh; and

executing the second loop iteratively to present the 3D interactive scene based on the head pose information, wherein the second loop further comprises:

determining whether the depth video is stored in the storage unit;

in an event that the depth video is stored in the storage unit, proceeding to the step of loading the corresponding depth frame for the video frame from the storage unit; and

in an event that the depth video is not stored in the storage unit, starting a real-time inference thread to estimate the corresponding depth frame for the video frame.

9 . The method as claimed in claim 8 , wherein the real-time inference thread comprises:

loading the video frame from the storage unit;

estimating the corresponding depth frame for the video frame; and

performing intraframe depth stabilization on the estimated depth frame, wherein the background inference thread comprises:

executing a third loop to estimate the corresponding depth video for the 2D source video; and

performing global depth stabilization on the stored depth video;

wherein each iteration of the third loop comprises:

loading the video frame from the storage unit;

estimating the corresponding depth frame;

performing intraframe depth stabilization on the estimated depth frame; and

storing the stabilized depth frame in the storage unit;

wherein the background inference thread, relative to the real-time inference thread, adopts parameter settings with higher accuracy and computational complexity in depth estimation and intraframe depth stabilization.

10 . The method as claimed in claim 1 , wherein the first process further comprises:

detecting a source-change request; and

in response to the source-change request, checking a file format of a user-specified source;

in an event that the user-specified source is a static image, exiting the first loop, and loading the static image from the storage unit as another 2D source image;

in an event that the user-specified source is a video, switching to the second process where the video is loaded as the 2D source video, and

wherein the second process further comprises:

detecting the source-change request; and

in response to the source-change request, checking the file format of the user-specified source;

in an event that the user-specified source is a video, loading the video frame of the video from the storage unit as the video frame of another 2D source video; and

in an event that the user-specified source is a static image, switching to the first process where the static image is loaded as the 2D source image.

11 . The method as claimed in claim 1 , further comprising:

acquiring a handle to a desktop wallpaper, wherein the 3D interactive scene is presented as the desktop wallpaper.

12 . A computer system for generating a 3D interactive scene, comprising:

a storage unit, storing a 2D source image, 2D source video, and a program;

a display unit; and

a processing unit, communicable with the storage unit and the display unit, loading the program from the storage unit to execute a first process based on the two-dimensional (2D) source image loaded from the storage unit;

wherein the first process comprises:

estimating a corresponding depth map for the 2D source image;

creating a 3D mesh based on the estimated depth map; and

executing a first loop iteratively to present the 3D interactive scene based on the 3D mesh and head pose information;

wherein the processing unit further executes a second process based on the 2D source video loaded from the storage unit, wherein the second process comprises:

determining whether a depth video corresponding to the 2D source video is stored in the storage unit, wherein the depth video comprises a depth frame sequence;

in an event that the depth video is stored in the storage unit, initializing the 3D mesh, and executing a second loop iteratively to present the 3D interactive scene based on the head pose information; and

in an event that the depth video is not stored in the storage unit, starting a background inference thread for generating the depth video and storing the generated depth video in the storage unit.

13 . The computer system as claimed in claim 12 , wherein the first loop comprises:

obtaining the head pose information;

calculating a set of head coordinates based on the head pose information;

constructing a transformation matrix based on the calculated set of head coordinates;

projecting each vertex of the 3D mesh onto a screen coordinate of an onscreen image using the transformation matrix;

coloring the onscreen image based on texture information corresponding to the 2D source image; and

causing the display unit to display the onscreen image.

14 . The computer system as claimed in claim 13 , wherein the first loop further comprises:

detecting a pause request;

in response to the pause request, pausing the iteration of the first loop and waiting for a resume request; and

in response to the resume request, resuming the next iteration of the first loop.

15 . The computer system as claimed in claim 14 , wherein the pause request is triggered in an event that an area proportion of the onscreen image obscured by windows of other programs exceeds a specified percentage.

16 . The computer system as claimed in claim 12 , wherein the second loop comprises:

loading a video frame of the 2D source video from the storage unit;

loading the corresponding depth frame for the video frame from the storage unit;

updating the 3D mesh based on the depth frame;

obtaining the head pose information;

calculating the set of head coordinates based on the head pose information;

constructing the transformation matrix based on the calculated set of head coordinates;

projecting each vertex of the updated 3D mesh onto the screen coordinate of the onscreen image using the transformation matrix;

coloring the onscreen image based on the texture information corresponding to the loaded video frame; and

causing the display unit to display the onscreen image.

17 . The computer system as claimed in claim 16 , wherein the second loop further comprises:

detecting a pause request;

in response to the pause request, pausing the iteration of the second loop and waiting for a resume request; and

in response to the resume request, resuming the next iteration of the second loop.

18 . The computer system as claimed in claim 17 , wherein the pause request is triggered in an event that an area proportion of the onscreen image obscured by windows of other programs exceeds a specified percentage.

19 . The computer system as claimed in claim 12 , wherein in an event that the depth video is stored in the storage unit, the second process further comprises:

starting the background inference thread while initializing the 3D mesh; and

executing the second loop iteratively to present the 3D interactive scene based on the head pose information, wherein the second loop further comprises:

determining whether the depth video is stored in the storage unit;

in an event that the depth video is stored in the storage unit, proceeding to the step of loading the corresponding depth frame for the video frame from the storage unit; and

in an event that the depth video is not stored in the storage unit, starting a real-time inference thread to estimate the corresponding depth frame for the video frame.

20 . The computer system as claimed in claim 19 , wherein the real-time inference thread comprises:

loading the video frame from the storage unit;

estimating the corresponding depth frame for the video frame; and

performing intraframe depth stabilization on the estimated depth frame;

wherein the background inference thread comprises:

executing a third loop to estimate the corresponding depth video for the 2D source video; and

performing global depth stabilization on the stored depth video;

wherein each iteration of the third loop comprises:

loading the video frame from the storage unit;

estimating the corresponding depth frame;

performing intraframe depth stabilization on the estimated depth frame; and

storing the stabilized depth frame in the storage unit;

wherein the background inference thread, relative to the real-time inference thread, adopts parameter settings with higher accuracy and computational complexity in depth estimation and intraframe depth stabilization.

21 . The computer system as claimed in claim 12 , wherein the first process further comprises:

detecting a source-change request; and

in response to the source-change request, checking a file format of a user-specified source;

in an event that the user-specified source is a static image, exiting the first loop, and loading the static image from the storage unit as another 2D source image;

in an event that the user-specified source is a video, switching to the second process where the video is loaded as the 2D source video;

wherein the second process further comprises:

detecting the source-change request; and

in response to the source-change request, checking the file format of the user-specified source;

in an event that the user-specified source is a video, loading the video frame of the video from the storage unit as the video frame of another 2D source video; and

in an event that the user-specified source is a static image, switching to the first process where the static image is loaded as the 2D source image.

22 . The computer system as claimed in claim 12 , wherein the 3D interactive scene is presented as a desktop wallpaper; and

wherein the processing unit further acquires a handle to the desktop wallpaper.

Assignments (1)

Continuity (1)

References Cited (24)