IP Library Granted Patent US 11,941,315
Granted Patent B2
US 11,941,315 · App. 17/893,605 · Granted Mar 26, 2024

Wearable electronic device for displaying augmented reality object and method for operating the same

Inventors: Jinhong Jeong (Suwon-si, KR); Eunsu Jeong (Suwon-si, KR); Minji Cho (Suwon-si, KR); Seokhyun Kim (Suwon-si, KR); Gajin Song (Suwon-si, KR); Sunkey Lee (Suwon-si, KR); Chaigil Lim (Suwon-si, KR)
Assignee: Samsung Electronics Co., Ltd.
G06F3/14G02B27/0172G02B27/0179G06F3/013G06F3/017G02B2027/0138G02B2027/014G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,941,315
App. No.
17/893,605
Granted
Mar 26, 2024
Kind
B2
Abstract

A wearable electronic device is provided. The electronic device includes a camera, a communication circuit, a display including a transparent lens and displaying content through the lens, and a processor, wherein the processor may be configured to obtain first information about a first device indicated by a first object through the communication circuit when the first object is selected from among a plurality of objects for controlling devices displayed on an external electronic device, identify a first position for displaying the first information corresponding to the first object as an augmented reality image, based on a gaze direction of a user identified through the camera, determine a property of the augmented reality image, based on a user input, and control the display to display the augmented reality image having the property at the first position.

Claims (71)

1. A wearable electronic device comprising:

a camera;

a communication circuit;

a display; and

a processor configured to:

in response to a first object being selected from among a plurality of objects being displayed by an external electronic device, obtain, through the communication circuit, first information about a first device indicated by the first object, the plurality of objects being objects used for controlling a plurality of devices, respectively,

identify a first position in a specific space identified through the camera after the first object is selected, wherein the first position corresponds to one of a wall, a floor, a specific object, or a specific device in the specific space,

identify coordinate information corresponding to the first position with respect to the specific space to display an augmented reality object for controlling the first device, wherein the augmented reality object is obtained based on the first information,

control the display to display, at the first position, the augmented reality object corresponding to an augmented reality image of the first object, based on the coordinate information,

based on a user input, identify an attribute of the augmented reality object, and

control the display to display the augmented reality object having the attribute.

2. The wearable electronic device of claim 1 , wherein the processor is further configured to:

based on the user input, adjust at least one of a size, a form, a shape, a level of transparency, a slope, a direction, a color, or a position of the augmented reality object.

3. The wearable electronic device of claim 2 , wherein the processor is further configured to:

based on a gesture of a user identified through a sensor of the wearable electronic device, identify the user input.

4. The wearable electronic device of claim 2 , wherein the processor is further configured to:

based on an input received by to the external electronic device, determine the user input.

5. The wearable electronic device of claim 1 , wherein the processor is further configured to:

identify whether a user gazes at the first object for a specified period of time through the camera, and

in response to identifying that the user gazes at the first object for the specified period of time, select the first object.

6. The wearable electronic device of claim 5 , wherein the processor is further configured to:

identify an identifier of the first object through the camera, the identifier being comprised in the first object displayed on the external electronic device, and

identify whether the user gazes at the first object using the identified identifier.

7. The wearable electronic device of claim 1 , wherein the processor is further configured to:

in response to the first object being selected by an input received by the external electronic device, receive information indicating selection of the first object and the first information about the first device indicated by the first object from the external electronic device.

8. The wearable electronic device of claim 1 , wherein the processor is further configured to:

obtain three-dimensional (3D) space coordinate information about an area indicated by a gaze direction of a user, and

identify the first position based on a first coordinate corresponding to the gaze direction among coordinates of a closed space comprised in the 3D space coordinate information.

9. The wearable electronic device of claim 1 , wherein the processor is further configured to:

control the display to display an image showing a path between a position indicated by a first coordinate and the wearable electronic device.

10. The wearable electronic device of claim 1 , wherein the processor is further configured to:

control the display to display the augmented reality object fixed at the first position.

11. The wearable electronic device of claim 1 ,

wherein the first device comprises one of a smart light bulb, a smart air conditioner, a robot vacuum cleaner, a smart window, a smart door, a smart curtain, a smart home appliance, or a smart light,

wherein the first information comprises state information about the first device, information about a command to control the first device, and user interface information about a control menu, and

wherein the plurality of objects are representations of the plurality of devices, respectively, and include a respective icon corresponding to a respective device.

12. An operating method of a wearable electronic device, the operating method comprising:

in response to a first object being selected from among a plurality of objects being displayed by an external electronic device, obtaining, through a communication circuit of the wearable electronic device, first information about a first device indicated by the first object, the plurality of objects being objects used for controlling a plurality of devices, respectively;

identifying a first position in a specific space identified through a camera after the first object is selected, wherein the first position corresponds to one of a wall, a floor, a specific object, or a specific device in the specific space;

identifying coordinate information corresponding to the first position with respect to the specific space to display an augmented reality object for controlling the first device, wherein the augmented reality object is obtained based on the first information;

displaying, at the first position of a display including a transparent lens included in the wearable electronic device, the augmented reality object corresponding to an augmented reality image of the first object, based on the coordinate information;

based on a user input, identifying an attribute of the augmented reality object; and

displaying, through the display, the augmented reality object having the attribute.

13. The operating method of claim 12 , further comprising:

based on the user input, adjusting at least one of a size, a form, a shape, a level of transparency, a slope, a direction, a color, or a position of the augmented reality object.

14. The operating method of claim 13 , further comprising:

based on a gesture of a user identified through a sensor of the wearable electronic device, identifying the user input.

15. The operating method of claim 13 , further comprising:

based on an input applied to the external electronic device, determining the user input.

16. The operating method of claim 12 , further comprising:

identifying whether a user gazes at the first object for a specified period of time through the camera; and

in response to identifying that the user gazes at the first object for the specified period of time, selecting the first object.

17. The operating method of claim 16 , wherein the identifying whether the user gazes at the first object comprises:

identifying an identifier of the first object through the camera, the identifier being comprised in the first object displayed on the external electronic device; and

identifying whether the user gazes at the first object using the identified identifier.

18. The operating method of claim 12 , wherein the identifying of the first position comprises:

obtaining three-dimensional (3D) space coordinate information about an area indicated by a gaze direction of a user; and

identifying the first position based on a first coordinate corresponding to the gaze direction among coordinates of a closed space comprised in the 3D space coordinate information.

19. The operating method of claim 12 , further comprising:

displaying an image showing a path between a position indicated by a first coordinate and the wearable electronic device through the display.

20. The operating method of claim 12 , wherein the displaying of the augmented reality object at the first position comprises displaying the augmented reality object fixed at the first position through the display.

21. A non-transitory recording medium storing a program, the program comprising instructions which, when executed by at least one processor of a wearable electronic device, cause the wearable electronic device to:

in response to a first object being selected from among a plurality of objects being displayed by an external electronic device, obtain, through a communication circuit of the wearable electronic device, first information about a first device indicated by the first object, the plurality of objects being objects used for controlling a plurality of devices, respectively;

identify a first position pointed by a gaze of a user in a specific space identified through a camera after the first object is selected, wherein the first position corresponds to one of a wall, a floor, a specific object, or a specific device in the specific space;

identify coordinate information corresponding to the first position with respect to the specific space to display an augmented reality object for controlling the first device, wherein the augmented reality object is obtained based on the first information;

display, at the first position, the augmented reality object corresponding to an augmented reality image of the first object, based on the coordinate information;

based on a user input, identify an attribute of the augmented reality object; and

display, through the display, the augmented reality object having the attribute.

22. The non-transitory recording medium of claim 21 , wherein a gaze direction is based on coordinates of the first position in real space.

23. The non-transitory recording medium of claim 21 , wherein the user input includes at least one of a touch input, a gesture input, or a voice input.

24. The non-transitory recording medium of claim 21 , wherein the camera includes a first camera for tracking a hand and a second camera for tracking an eye.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2022
From: JEONG, JINHONG; JEONG, EUNSU; CHO, MINJI; KIM, SEOKHYUN; SONG, GAJIN; LEE, SUNKEY; LIM, CHAIGIL
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 060870/0834 →
Priority Claims (2)
KR 10-2021-0111154 · Aug 23, 2021 · national
KR 10-2021-0157721 · Nov 16, 2021 · national
Continuity (2)
Continuation PCTKR2022012532 · Aug 22, 2022
Related Publication 20230058340A1 · Feb 23, 2023
Cited By (2)
US 12,518,307 US 12,626,470