IP Library › Granted Patent US 11,714,485
Granted Patent B2
US 11,714,485 · App. 17/247,930 · Granted Aug 1, 2023

Electronic device and method for providing position of user

Inventors: Junwhon Uhm (Suwon-si, KR); Jaewon Kim (Suwon-si, KR); Minhyuk Nam (Suwon-si, KR); Yunguk Lee (Suwon-si, KR)
Assignee: Samsung Electronics Co., Ltd.
G06F3/012G06T7/73H04N13/117H04N13/344H04N23/57H04W4/029G06T2207/30244H01Q21/0025
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,714,485
App. No.
17/247,930
Granted
Aug 1, 2023
Kind
B2
Abstract

Various embodiments disclose an electronic device including a camera, at least one mmWave antenna module, and at least one processor, wherein the at least one processor is configured to: acquire image information of a surrounding environment via the camera; acquire signal information resulting from a signal emitted from the at least one mmWave antenna module by the surrounding environment; and track a position of the user, based on at least one of the image information acquired via the camera and the signal information acquired via the at least one mmWave antenna module. Various other embodiments derived from the specification are possible.

Claims (79)

1. An electronic device wearable by a user, comprising:

a camera;

an illuminance sensor;

at least one mmWave antenna module configured to perform wireless communication by using an mmWave signal; and

at least one processor,

wherein the at least one processor is configured to:

sense an illuminance by using the illuminance sensor;

identify a state of the wireless communication with an external device;

acquire image information of a surrounding environment via the camera;

acquire signal information resulting from reflection of a signal emitted from the at least one mmWave antenna module by the surrounding environment; and

track a position of the electronic device of the user, based on at least one piece of information among the image information acquired via the camera and the signal information acquired via the at least one mmWave antenna module, according to the state of the wireless communication and the illuminance,

wherein the at least one mmWave antenna module comprises a first antenna module and a second antenna module,

wherein the at least one processor is configured to:

in response to the state of the wireless communication with the external device corresponding to a first state, acquire the signal information via the second antenna module to track the position; and

in response to the state of the wireless communication with the external device corresponding to a second state that does not correspond to the first state, communicate with the external device via the first antenna module and the second antenna module, and

wherein the at least one processor is further configured to:

in response to the state of the wireless communication with the external device corresponding to the second state and the illuminance sensed by the illuminance sensor exceeding a threshold value, acquire the image information via the camera to track the position; and

in response to the illuminance being equal to or less than the threshold value, acquire the signal information via the second antenna module to track the position.

2. The electronic device of claim 1 ,

wherein the at least one processor is configured to:

in response to the illuminance sensed by the illuminance sensor exceeding a threshold value, acquire the image information via the camera to track the position; and

in response to the illuminance being equal to or less than the threshold value, acquire the signal information via the at least one mmWave antenna module to track the position.

3. The electronic device of claim 1 , wherein the at least one processor is further configured to:

track the position by using the camera in response to a number of feature points detected by the camera based on the image information exceeding a designated number; and

in response to the number of the feature points being equal to or less than the designated number, acquire the signal information via the at least one mmWave antenna module to track the position.

4. The electronic device of claim 1 , wherein the at least one processor is configured to, in response to tracking the position by using the signal information acquired via the second antenna module, correct the position by using the image information acquired via the camera.

5. The electronic device of claim 1 , wherein the at least one processor is further configured to:

in response to the state of the wireless communication with the external device corresponding to the second state and a number of feature points detected by the camera based on the image information exceeding a designated number, track the position via the camera; and

in response to the number of the feature points being less than the designated number, acquire the signal information via the at least one antenna module to track the position.

6. The electronic device of claim 1 , wherein the at least one processor is configured to acquire the signal information via the at least one mmWave antenna module to track the position.

7. The electronic device of claim 1 , further comprising a display, wherein the at least one processor is configured to display information about the tracking of the position of the user on the display.

8. An operation method of an electronic device wearable by a user, comprising:

sensing an illuminance by using an illuminance sensor;

identifying a state of a wireless communication with an external device;

acquiring image information of a surrounding environment via a camera;

acquiring, by at least one mmWave antenna module in the electronic device, signal information resulting from reflection of a signal emitted from the at least one mmWave antenna module by the surrounding environment; and

tracking a position of the electronic device of the user, based on at least one piece of information among the image information acquired via the camera and the signal information acquired via the at least one mmWave antenna module, according to the state of the wireless communication and the illuminance,

wherein the at least one mmWave antenna module comprises a first antenna module and a second antenna module,

wherein the operation method of the electronic device is further comprising:

in response to the state of the wireless communication with the external device corresponding to a first state, acquiring the signal information via the second antenna module to track the position; and

in response to the state of the wireless communication with the external device corresponding to a second state that does not correspond to the first state, communicating with the external device via the first antenna module and the second antenna module, and

wherein the operation method of the electronic device is further comprising:

in response to the state of the wireless communication with the external device corresponding to the second state and the illuminance sensed by the illuminance sensor exceeding a threshold value, acquiring the image information via the camera to track the position; and

in response to the illuminance being equal to or less than the threshold value, acquiring the signal information via the second antenna module to track the position.

9. The method of claim 8 , further comprising:

in response to the illuminance sensed by the illuminance sensor exceeding a threshold value, acquiring the image information via the camera to track the position; and

in response to the illuminance being equal to or less than the threshold value, acquiring the signal information via the at least one mmWave antenna module to track the position.

10. The method of claim 8 , further comprising:

tracking the position by using the camera in response to a number of feature points detected by the camera based on the image information exceeding a designated number; and

in response to the number of the feature points being equal to or less than the designated number, acquiring the signal information via the at least one mmWave antenna module to track the position.

11. The method of claim 8 , further comprising:

displaying information about the tracking of the position of the electronic device of the user on the display.

12. An electronic device wearable by a user, comprising:

a camera;

an illuminance sensor;

at least one mmWave antenna module configured to perform wireless communication by using an mmWave signal; and

a lens part;

a projector configured to output data acquired via the at least one mmWave antenna module to the lens part, the output data being refracted through a prism and displayed on the lens part; and

at least one processor, wherein the at least one processor is configured to:

sense an illuminance by using the illuminance sensor;

identify a state of the wireless communication with an external device;

acquire image information of a surrounding environment via the camera;

acquire signal information resulting from reflection of a signal emitted from the at least one mmWave antenna module by the surrounding environment; and

track a position of the electronic device of the user, based on at least one piece of information among the image information acquired via the camera and the signal information acquired via the at least one mmWave antenna module, and

the projector is configured to output an augmented image to the lens part, based on the position of the electronic device of the user and the acquired data,

wherein the at least one mmWave antenna module comprises a first antenna module and a second antenna module,

wherein the at least one processor is configured to:

in response to the state of the wireless communication with the external device corresponding to a first state, acquire the signal information via the second antenna module to track the position; and

in response to the state of the wireless communication with the external device corresponding to a second state that does not correspond to the first state, communicate with the external device via the first antenna module and the second antenna module, and

wherein the at least one processor is further configured to:

in response to the state of the wireless communication with the external device corresponding to the second state and the illuminance sensed by the illuminance sensor exceeding a threshold value, acquire the image information via the camera to track the position; and

in response to the illuminance being equal to or less than the threshold value, acquire the signal information via the second antenna module to track the position.

13. The electronic device of claim 12 , wherein the at least one processor is further configured to:

track the position by using the camera in response to a number of feature points detected by the camera based on the image information exceeding a designated number; and

in response to the number of the feature points being equal to or less than the designated number, acquire the signal information via the at least one mmWave antenna module to track the position.

14. The electronic device of claim 12 , further comprising:

a single mmWave antenna module as the at least one mmWave antenna module, wherein the at least one processor is further configured to:

in response to communication with the external device being performed via the single mmWave antenna module, acquire the image information via the camera to track the position; and

in response to the communication with the external device not being performed, acquire the signal information via the single mmWave antenna module to track the position.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2020
From: UHM, JUNWHON; KIM, JAEWON; NAM, MINHYUK; LEE, YUNGUK
To: SAMSUNG ELECTRONICS CO., LTD
Reel/Frame 054780/0924 →
Priority Claims (1)
KR 10-2019-0178116 · Dec 30, 2019 · national
Continuity (1)
Related Publication 20210200305A1 · Jul 1, 2021
Cited By (1)
US 12,431,978