IP Library › Granted Patent US 11,829,527
Granted Patent B2
US 11,829,527 · App. 17/538,144 · Granted Nov 28, 2023

Augmented reality device, electronic device interacting with augmented reality device, and controlling method thereof

Inventors: Kyunghwa Kim (Gyeonggi-do, KR); Ilku Chang (Gyeonggi-do, KR); Yeojin Kim (Gyeonggi-do, KR); Giyoung Shin (Gyeonggi-do, KR)
G06F3/013G02B27/0093G02B27/0101G02B27/017G06F3/03547G06F3/04883G06V20/20G02B2027/014G02B2027/0138
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,829,527
App. No.
17/538,144
Granted
Nov 28, 2023
Kind
B2
Abstract

Methods and devices are provided in which a first screen is displayed on a first display of the first electronic device. An image of an eye of a user of the first electronic device is obtained. A first object in the first screen corresponding to a gaze direction of the user is identified based on the image of the eye of the user. An input interface is displayed on a second display of the second electronic device based on a type of the first object. A second screen that is different from the first screen is displayed on the first display of the first electronic device, based on a first input obtained through the input interface.

Claims (52)

1. An electronic device comprising:

a display;

at least one camera;

a communication circuit; and

at least one processor,

wherein the at least one processor is configured to:

display a first screen on the display;

obtain an image of an eye of a user of the electronic device through the at least one camera;

identify a first object in the first screen corresponding to a gaze direction of the user, based on the image of the eye of the user;

transmit, to an external electronic device through the communication circuitry, first information triggering display of an input interface for obtaining an input related to the first object at the external electronic device;

receive, from the external electronic device, through the communication circuit, second information about the input obtained through the input interface; and

display, on the display, a second screen that is different from the first screen, based on the second information about the input.

2. The electronic device of claim 1 , wherein the first information related to the first object indicates coordinates of the first object on the first screen.

3. The electronic device of claim 1 , wherein the first information related to the first object indicates a type of the input interface corresponding to the first object.

4. The electronic device of claim 1 , wherein the input interface comprises a trackpad input interface.

5. The electronic device of claim 1 , wherein the input interface comprises a setting change interface for changing a setting related to an operation of the electronic device.

6. The electronic device of claim 1 , wherein the first object is related to a text input, and the input interface comprises a keypad.

7. The electronic device of claim 1 , wherein the input interface comprises an option input interface displaying a plurality of options related to the first object.

8. The electronic device of claim 1 , wherein the at least one processor is further configured to:

receive sensor information related to at least one sensor of the external electronic device from the external electronic device through the communication circuit; and

identify the second screen based on the sensor information.

9. The electronic device of claim 1 , wherein the at least one processor is further configured to:

obtain a field of view (FOV) image of the user through the at least one camera; and

display a message inquiring whether to couple the electronic device to the external electronic device on the display in response to the external electronic device being included in the FOV image.

10. The electronic device of claim 1 , wherein the at least one processor is further configured to:

identify, through the communication circuit, that a distance between the external electronic device and the electronic device is less than a predetermined distance, and

display, on the display, a message inquiring whether to control the display by using the external electronic device, in response to the distance between the external electronic device and the electronic device being identified as less than the predetermined distance.

11. An electronic device comprising:

a display;

a communication circuit; and

at least one processor,

wherein the at least one processor is configured to:

receive first information related to a first object corresponding to a gaze direction of a user of an external electronic device, in a first screen displayed on the external electronic device, from the external electronic device through the communication circuit;

display, on the display, an input interface for obtaining an input related to the first object based on the information related to the first object;

obtain the input related to the first object through the input interface; and

transmit second information about the input to the external electronic device.

12. The electronic device of claim 11 , wherein the first information related to the first object indicates coordinates of the first object on the first screen.

13. The electronic device of claim 11 , wherein the first information related to the first object indicates a type of the input interface corresponding to the first object.

14. The electronic device of claim 11 , wherein the input interface comprises a trackpad input interface.

15. The electronic device of claim 11 , wherein the input interface comprises a setting change interface for changing a setting related to an operation of the external electronic device.

16. The electronic device of claim 11 , wherein the first object is related to a text input, and the input interface comprises a keypad.

17. The electronic device of claim 11 , wherein the input interface comprises an option input interface displaying a plurality of options related to the first object.

18. The electronic device of claim 11 , wherein the input interface comprises an indicator indicating at least one of a plurality of numbers or amounts related to the first object.

19. A method of controlling a first electronic device and a second electronic device, the method comprising:

displaying a first screen on a first display of the first electronic device;

obtaining an image of an eye of a user of the first electronic device;

identifying a first object in the first screen corresponding to a gaze direction of the user, based on the image of the eye of the user;

displaying an input interface for obtaining a first input related to the first object based on a type of the first object on a second display of the second electronic device; and

displaying, on the first display, a second screen that is different from the first screen, based on the first input obtained through the input interface.

20. The method of claim 19 , further comprising:

displaying, on the first display, a message inquiring whether to establish a connection between the first electronic device and the second electronic device; and

identifying a second input instructing to establish the connection between the first electronic device and the second electronic device and to control the first electronic device by using the second electronic device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2021
From: KIM, KYUNGHWA; CHANG, ILKU; KIM, YEOJIN; SHIN, GIYOUNG
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 058266/0426 →
Priority Claims (1)
KR 10-2020-0164402 · Nov 30, 2020 · national
Continuity (2)
Continuation PCTKR2021017866 · Nov 30, 2021
Related Publication 20220171455A1 · Jun 2, 2022
Cited By (2)
US 12,518,307 US 12,626,470