Granted Patent B2

US 12,730,498 · App. 18/932,857 · Granted Sep 8, 2026

Inventors: Youngki Lee (Seoul, KR); Hyuna Seo (Seoul, KR); Juheon Yi (Seoul, KR)

Assignee: SEOUL NATIONAL UNIVERSITY R & DB FOUNDATION

G06F3/011G06T11/00

Abstract

Claims (47)

1 . An electronic apparatus for supporting interaction between a virtual environment and a real world, the electronic apparatus comprising:

a memory; and

a controller configured to determine an interaction state for at least one physical object among a plurality of physical objects in a real world and provide a virtual environment according to a blending technique selected based on determination results,

wherein the controller is configured to determine the interaction state by tracking a movement of at least one of a user, a user's hand, and an object, and by tracking at least one of (i) an interaction between the user and the object, (ii) an interaction between the user's hand and the object, and (iii) an interaction between a plurality of objects, and

wherein the controller is configured to:

analyze at least one of a distance between the user and the object, a distance between the user's hand and the object, and a distance between the plurality of objects;

classify the plurality of physical objects into a target object with which the user interacts and a non-target object with which the user does not interact;

set an interaction state flow for the target object and an interaction state flow for the non-target object differently; and

determine an avoid state, which is a state activated when the user needs to avoid the non-target object near the target object, based on a distance between the target object and the non-target object.

2 . The electronic apparatus of claim 1 , wherein the controller determines a type of the at least one physical object, and selects one of a perceive state, an approach state and a manipulate state when the at least one physical object is the target object and selects one of the perceive state and the avoid state when the at least one physical object is the non-target object.

3 . The electronic apparatus of claim 1 , wherein the controller determines whether the interaction state for the at least one physical object is a simple manipulate state or a complex manipulate state among detailed states of manipulate states.

4 . The electronic apparatus of claim 1 , wherein the blending technique is any one of Virtual Proxy, Affordance Contour, Pass-Through, and Boundary Box.

5 . The electronic apparatus of claim 1 , wherein the controller overlays an affordance contour on an object for which the at least one physical object has been processed into a virtual proxy when the interaction state is one of an approach state and a simple manipulate state.

6 . The electronic apparatus of claim 1 , wherein the controller overlays an affordance contour on an object for which the at least one physical object has been processed into a virtual proxy, and visually processes a grasping or manipulating area on the object processed into the virtual proxy.

7 . The electronic apparatus of claim 1 , wherein the controller overlays an affordance contour on a virtual object corresponding to the at least one physical object when a distance between the at least one physical object and a user is closer than a predetermined value, and deactivates the affordance contour overlay when the distance is equal to or farther than the predetermined value.

8 . The electronic apparatus of claim 1 , wherein the controller determines the non-target object, located within a predetermined radius from the at least one physical object, to be in the avoid state when it is determined that the at least one physical object is in a manipulate state.

9 . The electronic apparatus of claim 8 , wherein the controller applies a boundary box to the non-target object that is determined to be in the avoid state.

10 . A method of supporting interaction between a virtual environment and a real world, the method being performed by an electronic apparatus, the method comprising:

determining an interaction state for at least one physical object among a plurality of physical objects in a real world; and

providing a virtual environment according to a blending technique selected based on determination results,

wherein the interaction state is determined by tracking a movement of at least one of a user, a user's hand, and an object, and by tracking at least one of (i) an interaction between the user and the object, (ii) an interaction between the user's hand and the object, and (iii) an interaction between a plurality of objects, and

wherein determining the interaction state comprises:

analyzing at least one of a distance between the user and the object, a distance between the user's hand and the object, and a distance between the plurality of objects;

classifying the plurality of physical objects into a target object with which the user interacts and a non-target object with which the user does not interact;

setting an interaction state flow for the target object and an interaction state flow for the non-target object differently; and

determining an avoid state, which is a state activated when the user needs to avoid the non-target object near the target object, based on a distance between the target object and the non-target object.

11 . The method of claim 10 , wherein determining the interaction state comprises:

determining a type of the at least one physical object; and

selecting one of a perceive state, an approach state and a manipulate state when the at least one physical object is the target object, and selecting one of the perceive state and the avoid state when the at least one physical object is the non-target object.

12 . The method of claim 10 , wherein determining the interaction state comprises determining whether the interaction state for the at least one physical object is a simple manipulate state or a complex manipulate state among detailed states of manipulate states.

13 . The method of claim 10 , wherein providing the virtual environment comprises overlaying an affordance contour on an object for which the at least one physical object has been processed into a virtual proxy when the interaction state is one of an approach state and a simple manipulate state.

14 . The method of claim 10 , wherein providing the virtual environment comprises:

overlaying an affordance contour on an object for which the at least one physical object has been processed into a virtual proxy; and

visually processing a grasping or manipulating area on the object processed into the virtual proxy.

15 . The method of claim 10 , wherein providing the virtual environment comprises:

overlaying an affordance contour on a virtual object corresponding to the at least one physical object when a distance between the at least one physical object and a user is closer than a predetermined value; and

deactivating the affordance contour overlay when the distance is equal to or farther than the predetermined value.

16 . The method of claim 10 , wherein determining the interaction state comprises determining the non-target object, located within a predetermined radius from the at least one physical object, to be in an avoid state when it is determined that the at least one physical object is in a manipulate state.

17 . The method of claim 16 , wherein providing the virtual environment comprises applying a boundary box to the non-target object that is determined to be in the avoid state.

18 . A non-transitory computer-readable storage medium having stored thereon a program that, when executed by a processor, causes the processor to execute a method of supporting interaction between a virtual environment and a real world, the method being performed by an electronic apparatus, the method comprising:

determining an interaction state for at least one physical object among a plurality of physical objects in a real world; and

providing a virtual environment according to a blending technique selected based on determination results, wherein the interaction state is determined by tracking a movement of at least one of a user, a user's hand, and an object, and by tracking at least one of (i) an interaction between the user and the object, (ii) an interaction between the user's hand and the object, and (iii) an interaction between a plurality of objects, and

wherein determining the interaction state comprises:

analyzing at least one of a distance between the user and the object, a distance between the user's hand and the object, and a distance between the plurality of objects;

classifying the plurality of physical objects into a target object with which the user interacts and a non-target object with which the user does not interact;

setting an interaction state flow for the target object and an interaction state flow for the non-target object differently; and

determining an avoid state, which is a state activated when the user needs to avoid the non-target object near the target object, based on a distance between the target object and the non-target object.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (25)