IP Library › Granted Patent US 12,198,732
Granted Patent B2
US 12,198,732 · App. 17/310,460 · Granted Jan 14, 2025

Electronic device for processing audio data and method for operating same

Inventors: Gupil Cheong (Suwon-si, KR); Hyunwook Kim (Suwon-si, KR); Hangil Moon (Suwon-si, KR); Byoungchul Lee (Suwon-si, KR); Juyeon Jin (Suwon-si, KR); Doosuk Kang (Suwon-si, KR); Bokun Choi (Suwon-si, KR)
Assignee: Samsung Electronics Co., Ltd.
G11B27/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,198,732
App. No.
17/310,460
Granted
Jan 14, 2025
Kind
B2
Abstract

An electronic device according to various embodiments includes a camera; a display; a communication module supporting Bluetooth communication; and a processor configured to: establish a communication link with a plurality of external electronic devices through the communication module; transmit a first signal indicating an occurrence of an event using the camera to at least one of the plurality of external electronic devices through the communication link; receive audio data corresponding to sound acquired by each of the plurality of external electronic devices, from each of the plurality of external electronic devices in predetermined time periods through the communication link in a state in which the plurality of external electronic devices are worn; and synchronize the audio data with video acquired using the camera and store the synchronized data, based on time and an order at which each of the plurality of external electronic devices acquires the sound.

Claims (43)

1. An electronic device comprising:

a camera;

a display;

a communication circuit configured to support Bluetooth communication; and

a processor, wherein the processor is configured to:

establish at least one communication link with a plurality of external electronic devices through the communication circuit;

perform an event for capturing video data using the camera in a binaural mode;

transmit a first signal indicating an occurrence of the event to at least one of the plurality of external electronic devices through the at least one communication link;

while capturing the video data using the camera, receive first audio data corresponding to a first sound acquired by a first external electronic device among the plurality of external electronic devices, in a first time period, and second audio data corresponding to a second sound acquired by a second external electronic device among the plurality of external electronic devices, in a second time period different from the first time period, through the at least one communication link in a state in which the plurality of external electronic devices are worn on a user's body;

acquire stereo audio data using the first audio data and the second audio data, based on time and sequence at which the first external electronic device and the second external electronic device acquire the first sound and the second sound, respectively;

identify a latency between the stereo audio data and the video data based on a first delay time for receiving the first audio data and the second audio data and a second delay time for acquiring the stereo audio data; and

synchronize and store the stereo audio data with the video data captured using the camera, based on the latency between the stereo audio data and the video data.

2. The electronic device of claim 1 , wherein the processor is configured to:

transmit a first acknowledgement (ACK) signal to the first external electronic device in the first time period when the first audio data is received; and

transmit a second ACK signal to the second external electronic device in the second time period when the second audio data is received.

3. The electronic device of claim 1 , wherein:

the first external electronic device includes an earphone in a first direction, and

the second external electronic device includes an earphone in a second direction.

4. The electronic device of claim 3 , wherein the processor is configured to:

check a wearing state of each of the first external electronic device and the second external electronic device; and

display information on the wearing state through the display.

5. The electronic device of claim 1 , wherein the processor is configured to:

receive the first audio data through a first communication link established with the first external electronic device; and

receive the second audio data through a second communication link established with the second external electronic device.

6. The electronic device of claim 1 , wherein the processor is configured to:

receive the first audio data from the first external electronic device through a first communication link established with the first external electronic device; and

receive the second audio data from the second external electronic device using the first communication link.

7. The electronic device of claim 1 , wherein the processor is configured to:

receive the first audio data and the second audio data from of the first external electronic device and the second external electronic device, respectively, using at least one of an extended synchronous connections (eSCO) communication channel, an asynchronous connection-less (ACL) communication channel, or a BLE communication channel.

8. A method of operating an electronic device, comprising:

establishing at least one communication link with a plurality of external electronic devices;

performing an event for capturing video data using a camera included in the electronic device in a binaural mode;

transmitting a first signal indicating an occurrence of the event to at least one of the plurality of external electronic devices through the at least one communication link;

while capturing the video data using the camera, receiving first audio data corresponding to a first sound acquired by a first external electronic device among the plurality of external electronic devices, in a first time period, and second audio data corresponding to a second sound acquired by a second external electronic device among the plurality of external electronic devices, in a second time period different from the first time period, through the at least one communication link in a state in which the plurality of external electronic devices are worn;

acquiring stereo audio data using the first audio data and the second audio data, based on time and sequence at which the first external electronic device and the second external electronic device acquire the first sound and the second sound, respectively;

identifying a latency between the stereo audio data and the video data based on a first delay time for receiving the first audio data and the second audio data and a second delay time for acquiring the stereo audio data; and

synchronizing and storing the stereo audio data with video data captured using the camera, based on the latency between the stereo audio data and the video data.

9. The method of claim 8 , further comprising:

transmitting a first ACK signal to the first external electronic device in the first time period when the first audio data is received; and

transmitting a second ACK signal to the second external electronic device in the second time period when the second audio data is received.

10. The method of claim 8 , further comprising:

identifying a wearing state of each of the first external electronic device and the second external electronic device; and

displaying information on the wearing state through the display.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 4, 2021
From: CHEONG, GUPIL; KIM, HYUNWOOK; MOON, HANGIL; LEE, BYOUNGCHUL; JIN, JUYEON; KANG, DOOSUK; CHOI, BOKUN
To: SAMSUNG ELECTRONICS CO., LTD
Reel/Frame 057080/0954 →
Priority Claims (1)
KR 10-2020-0097681 · Aug 4, 2020 · national
Continuity (1)
Related Publication 20220319553A1 · Oct 6, 2022
References Cited (32)
US 8922713B1 · Chakrovorthy · 2014 [cited by examiner]
US 9787862B1 · Newman · 2017 [cited by examiner]
US 10419853B2 · Hariharan et al. · 2019 [cited by applicant]
US 20120058727A1 · Cook et al. · 2012 [cited by applicant]
US 20150364143A1 · Abildgren et al. · 2015 [cited by applicant]
US 20160001193A1 · Eck · 2016 [cited by examiner]
US 20160357510A1 · Watson · 2016 [cited by examiner]
US 20170150266A1 · Mattana · 2017 [cited by applicant]
US 20170192743A1 · Chun et al. · 2017 [cited by applicant]
US 20170264792A1 · Lee et al. · 2017 [cited by applicant]
US 20180084215A1 · Jing et al. · 2018 [cited by applicant]
US 20190158722A1 · Jung et al. · 2019 [cited by applicant]
US 20190261089A1 · Hariharan · 2019 [cited by examiner]
US 20200053460A1 · Pedersen · 2020 [cited by examiner]
CN 101278596A · 2008 [cited by applicant]
CN 108702559A · 2018 [cited by applicant]
KR 1020050046296A · 2005 [cited by applicant]
KR 1020140060003A · 2014 [cited by applicant]
KR 101467822B1 · 2014 [cited by applicant]
KR 1020150142566A · 2015 [cited by applicant]
KR 1020170022272A · 2017 [cited by applicant]
KR 1020170106862A · 2017 [cited by applicant]
KR 101843800B1 · 2018 [cited by examiner]
KR 1020180138017 · 2018 [cited by applicant]
KR 1020190056696 · 2019 [cited by applicant]
KR 1020190090069 · 2019 [cited by applicant]
WO 20202102156A1 · 2020 [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority in connection with International Application No. PCT/KR2021/009906 issued Nov. 4, 2021, 9 pages. [cited by applicant]
The First Office Action dated Oct. 19, 2023, in connection with Chinese Application No. 202180058285.2, 17 pages. [cited by applicant]
Supplementary European Search Report dated Aug. 18, 2023, in connection with European Application No. 21852943.6, 10 pages. [cited by applicant]
Notification of the Second Office Action dated Mar. 8, 2024, in connection with Chinese Application No. 202180058285.2, 16 pages. [cited by applicant]
Notification to Grant Patent Right for Invention dated May 30, 2024, in connection with Chinese Application No. 202180058285.2, 8 pages. [cited by applicant]
Cited By (1)
US 12,328,564