IP Library › Granted Patent US 11,668,782
Granted Patent B2
US 11,668,782 · App. 16/737,219 · Granted Jun 6, 2023

Electronic apparatus, controlling method of electronic apparatus and computer readable medium

Inventors: Jongyoub Ryu (Suwon-si, KR); Geunwoo Kim (Suwon-si, KR); Changhyun Kim (Suwon-si, KR); Soohoon Lee (Suwon-si, KR); Byoungkab Choi (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G01S3/801G01S3/8083H04M1/03
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,668,782
App. No.
16/737,219
Granted
Jun 6, 2023
Kind
B2
Abstract

Provided is an electronic apparatus. The electronic apparatus includes an audio receiver configured to obtain an audio signal of sound output by an external object; a sensor configured to sense a posture of the electronic apparatus; a display; and a processor configured to, based on the audio signal that is obtained by the audio receiver, determine a direction in which the external object is located with respect to the electronic apparatus, and control the display to display a graphical object that corresponds to the external object based on the posture of the electronic apparatus sensed by the sensor and the direction in which the external object is located.

Claims (67)

1. An electronic apparatus comprising:

an audio receiver configured to obtain an audio signal of sound output by an external object;

a sensor;

a display; and

a processor configured to:

based on the audio signal that is obtained by the audio receiver, determine a type of the external object by inputting the audio signal to a pre-trained artificial intelligence to determine a type of a sound source of the audio signal and a direction in which the external object is located with respect to the electronic apparatus,

determine a posture of the electronic apparatus based on the sensor,

determine an area of the display that corresponds to the direction, based on the posture of the electronic apparatus,

determine a shape of the external object being obtained at a viewpoint corresponding to the posture of the electronic apparatus based on the posture of the electronic apparatus and the direction

determine a type of a graphical object corresponding to the type of the external object, and determine a shape of the graphical object corresponding to the shape of the external object; and

control the display to display the graphical object having the shape corresponding to the shape of the external object in the determined area of the display,

wherein the processor is further configured to:

based on the posture being within a first predetermined range of a first posture in which the display faces up, determine the shape of the external object as a vertical view,

based on the posture being within a second predetermined range of a second posture that is perpendicular to a direction of the display from the first posture, determine the shape of the external object as a horizontal view, and

identify a distance between the electronic apparatus and the external object based on the audio signal and, change a size of the area of the display and a size of the graphical object having the shape corresponding to the shape of the external object based on the distance identified, and

wherein the area of the display is differently identified for the external object in the same direction according to the posture of the electronic apparatus.

2. The electronic apparatus of claim 1 , wherein the processor is further configured to:

based on the posture of the electronic apparatus being within the first predetermined range from the first posture, control the display to display the graphical object having the shape corresponding to a top view of the external object, and

based on the posture of the electronic apparatus being within the second predetermined range from the second posture, control the display to display the graphical object having the shape corresponding to a side view of the external object.

3. The electronic apparatus of claim 2 , wherein the processor is further configured to, based on the posture of the electronic apparatus being within the first predetermined range from the first posture, control the display to display the graphical object having the shape corresponding to the top view at an angle corresponding to the determined area.

4. The electronic apparatus of claim 2 , wherein the processor is further configured to, based on the posture of the electronic apparatus being within the second predetermined range from the second posture, control the display to display the graphical object having the shape corresponding to the side view at a viewpoint corresponding to the determined area.

5. The electronic apparatus of claim 1 , wherein the processor is further configured to:

identify a speed at which the external object approaches the electronic apparatus by comparing the audio signal obtained by the audio receiver with a prestored audio signal, and

provide a notification based on the identified speed.

6. The electronic apparatus of claim 1 , wherein the audio receiver comprises:

a first microphone configured to obtain an audio signal of the sound output by the external object; and

a second microphone configured to obtain an audio signal of the sound output by the external object, and

wherein the processor is further configured to determine the direction in which the external object is located based on a first audio signal obtained by the first microphone and a second audio signal obtained by the second microphone when the electronic apparatus is in a first position, and at least one of a third audio signal obtained by the first microphone and a fourth audio signal obtained by the second microphone when the electronic apparatus is in a second position.

7. A method of controlling an electronic apparatus, the method comprising:

based on obtaining through an audio receiver of the electronic apparatus an audio signal of sound output from an external object, determining a type of the external object by inputting the audio signal to a pre-trained artificial intelligence to determine a type of a sound source of the audio signal and a direction in which the external object is located with respect to the electronic apparatus based on the audio signal;

determining a posture of the electronic apparatus based on a sensor of the electronic apparatus;

determine an area of the display that corresponds to the direction, based on the posture of the electronic apparatus;

determining a shape of the external object being obtained at a viewpoint corresponding to the posture of the electronic apparatus based on the posture of the electronic apparatus and the direction;

determining a type of a graphical object corresponding to the type of the external object, and determining a shape of the graphical object corresponding to the shape of the external object; and

displaying the graphical object having the shape corresponding to the view of the external object,

wherein the determining the shape of the external object comprises:

based on the posture being within a first predetermined range of a first posture in which the display faces up, determining the shape of the external object as a vertical view, and

based on the posture being within a second predetermined range of a second posture that is perpendicular to a direction of the display from the first posture, determining the shape of the external object as a horizontal view, and

wherein the displaying the graphical object comprises:

identifying a distance between the electronic apparatus and the external object based on the audio signal and,

changing a size of the graphical object based on the distance identified, and

wherein the area of the display is differently identified for the external object in the same direction according to the posture of the electronic apparatus.

8. The method of claim 7 , wherein the displaying the graphical object comprises:

based on the posture of the electronic apparatus being within the first predetermined range from the first posture, displaying the graphical object having the shape corresponding to a top view of the external object, and

based on the posture of the electronic apparatus being within the second predetermined range from the second posture, displaying the graphical object having the shape corresponding to a side view of the external object.

9. The method of claim 8 , wherein the displaying the graphical object comprises, based on the posture of the electronic apparatus being within the first predetermined range from the first posture, displaying the graphical object having the shape corresponding to the top view at an angle corresponding to the determined area.

10. The method of claim 8 , wherein the displaying the graphical object comprises, based on the posture of the electronic apparatus being within the second predetermined range from the second posture, displaying the graphical object having the shape corresponding to the side view at a viewpoint corresponding to the determined area.

11. The method of claim 7 , further comprising:

identifying a speed at which the external object approaches the electronic apparatus by comparing the audio signal obtained through the audio receiver with a prestored audio signal; and

providing a notification for a dangerous situation based on the identified speed.

12. The method of claim 7 , further comprising identifying the external object based on the audio signal obtained through the audio receiver,

wherein the displaying the graphical object comprises displaying the graphical object corresponding to the identified external object.

13. The method of claim 7 , wherein the determining the direction in which the external object is located comprises determining the direction in which the external object is located based a first audio signal of the sound output by the external object obtained through a first microphone and a second audio signal of the sound output by the external object obtained through a second microphone when the electronic apparatus is in a first position, and at least one of a third audio signal of the sound output by the external object obtained through the first microphone and a fourth audio signal of the sound output by the external object obtained through the second microphone when the electronic apparatus is in a second position.

14. A non-transitory computer readable medium having stored therein a computer instruction which is executed by a processor of an electronic apparatus to perform the method comprising:

based on obtaining through an audio receiver of the electronic apparatus an audio signal of sound output from an external object, determining a type of the external object by inputting the audio signal to a pre-trained artificial intelligence to determine a type of a sound source of the audio signal and a direction in which the external object is located with respect to the electronic apparatus based on the audio signal;

determining a posture of the electronic apparatus based on a sensor of the electronic apparatus;

determine an area of the display that corresponds to the direction, based on the posture of the electronic apparatus;

determining a shape of the external object being obtained at a viewpoint corresponding to the posture of the electronic apparatus based on the posture of the electronic apparatus and the direction;

determining a type of a graphical object corresponding to the type of the external object, and determining a shape of the graphical object corresponding to the shape of the external object; and

displaying the graphical object having the shape corresponding to the view of the external object,

wherein the determining the shape of the external object comprises:

based on the posture being within a first predetermined range of a first posture in which the display faces up, determining the shape of the external object as a vertical view, and

based on the posture being within a second predetermined range of a second posture that is perpendicular to a direction of the display from the first posture, determining the shape of the external object as a horizontal view, and

wherein the displaying the graphical object comprises:

identifying a distance between the electronic apparatus and the external object based on the audio signal, and

changing a size of the graphical object based on the distance identified, and

wherein the area of the display is differently identified for the external object in the same direction according to the posture of the electronic apparatus.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2020
From: RYU, JONGYOUB; KIM, GEUNWOO; KIM, CHANGHYUN; LEE, SOOHOON; CHOI, BYOUNGKAB
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 051656/0321 →
Priority Claims (1)
KR 10-2019-0002346 · Jan 8, 2019 · national
Continuity (1)
Related Publication 20200217917A1 · Jul 9, 2020