IP Library Granted Patent US 12,148,430
Granted Patent B2
US 12,148,430 · App. 17/448,616 · Granted Nov 19, 2024

Method, system, and computer-readable recording medium for managing text transcript and memo for audio file

Inventors: Jieun Shin (Seongnam-si, KR); Gil Hwan Hwang (Seongnam-si, KR); Ye Lim Jeong (Seongnam-si, KR); Dai-Hyun Lim (Seongnam-si, KR); SuMee Lee (Seongnam-si, KR); Hyangnam Oh (Seongnam-si, KR); Junghoon Jang (Seongnam-si, KR); Namkyu Jung (Seongnam-si, KR); Geonmin Kim (Seongnam-si, KR); Deok Su Kim (Seongnam-si, KR); Hyunjin Kim (Seongnam-si, KR); Daigeun Sohn (Seongnam-si, KR); Yeon Soo Song (Seongnam-si, KR); Dongyeol Lee (Seongnam-si, KR); Jin-myoung Lee (Seongnam-si, KR); Yujin Jeon (Seongnam-si, KR); Jongmin Choi (Seongnam-si, KR)
Assignees: NAVER CORPORATION; LINE WORKS CORP
G10L15/26G06F40/106G10L15/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,148,430
App. No.
17/448,616
Granted
Nov 19, 2024
Kind
B2
Abstract

An audio recording management method includes creating a text transcript of an audio recording by converting speech to text, matching and managing the text transcript and a memo written during recording or playback of the audio, and providing the text transcript in connection with the memo.

Claims (34)

1. An audio recording management method executed by a computer device having at least one processor configured to execute computer-readable instructions contained in a memory, the method comprising the steps of:

creating a text transcript of an audio recording by converting speech to text;

matching and managing the text transcript and a memo generated during an utterance spoken at a specific time in recording of the audio recording by a user, wherein the memo is matched to audio of the audio recording corresponding to the specific time of the utterance spoken;

providing the text transcript in connection with the memo; and

searching in the text transcript for a word with a similar pronunciation as a search word and whose similarity to the search word is a certain percent or more by comparing the word and the search word depending on what type of smaller units the word and the search word are broken down into, wherein the type of smaller units is one of phonemes, stems, and terms.

2. The audio recording management method of claim 1 , wherein the providing of the text transcript comprises displaying the text transcript and the memo together in a dual view on a screen.

3. The audio recording management method of claim 1 , wherein the providing of the text transcript comprises matching and displaying the text transcript and the memo in terms of time based on a timestamp.

4. The audio recording management method of claim 1 , wherein the providing of the text transcript comprises:

displaying a timestamp for the text transcript of the utterance in the audio recording to indicate when the utterance occurs; and

displaying a timestamp for the memo to indicate a text transcript time or audio playback time at the time of creation of the memo.

5. The audio recording management method of claim 1 , wherein the providing of the text transcript comprises:

displaying a memo tag for the text of the utterance included in the text transcript depending on whether there is a memo written in the utterance; and

displaying the full memo written in the utterance of the text when the memo tag is selected.

6. The audio recording management method of claim 1 , wherein the creating of the text transcript comprises receiving an audio file recorded from a client or an audio file uploaded from the client and converting speech data corresponding to the utterance into text.

7. The audio recording management method of claim 1 , wherein the managing of the text transcript comprises matching and managing the memo generated during recording of the audio based on a timestamp of the audio recording.

8. The audio recording management method of claim 1 , wherein the creating of the text transcript comprises:

connecting an artificial intelligent device providing an audio-based interface to a user account; and

creating a text transcript by converting a speech received from the artificial intelligent device into text.

9. The audio recording management method of claim 8 , wherein the connecting of the artificial intelligent device comprises:

issuing a connect key in response to a request from the artificial intelligent device; and

connecting the user account and the artificial intelligent device upon receiving the connect key through the user account.

10. The audio recording management method of claim 8 , wherein the providing of the text transcript comprises sharing the text transcript of the audio recording with at least one user specified through the user account.

11. The audio recording management method of claim 1 , wherein the creating of the text transcript comprises performing speech recognition by assigning a weight to a custom keyword the user enters in relation to the speech.

12. The audio recording management method of claim 11 , wherein the speech recognition is performed by applying boosting to the custom keyword.

13. The audio recording management method of claim 11 , wherein the performing of the speech recognition comprises extracting the custom keyword from the memo.

14. The audio recording management method of claim 11 , wherein the performing of the speech recognition comprises extracting, as the custom keyword, at least either a word entered by the user to give information on the text transcript or a keyword registered as a frequently used word.

15. The audio recording management method of claim 1 , further comprising correcting at least one word contained in a search result for the text transcript and replacing the same with another word.

16. A non-transitory computer-readable recording medium storing a program for executing the audio recording management method of claim 1 .

17. A computer device comprising at least one processor configured to execute computer-readable instructions stored in a memory, the at least one processor comprising:

a text transcript creation part for creating a text transcript of an audio recording by converting speech to text;

a memo matching and management part for matching and managing the text transcript and a memo generated during an utterance spoken at a specific time in recording of the audio recording by a user, wherein the memo is matched to audio of the audio recording corresponding to the specific time of the utterance spoken; and

a text transcript provision part for:

providing the text transcript in connection with the memo; and

searching in the transcript for a word with a similar pronunciation as the search word and whose similarity to the search word is a certain percent or more by comparing the word and the search word depending on what type of smaller units the word and the search word are broken down into, wherein the type of smaller units is one of phonemes, stems, and terms.

Assignments (7)
CHANGE OF NAME Recorded Mar 7, 2024
From: WORKS MOBILE JAPAN CORPORATION
To: LINE WORKS CORP.
Reel/Frame 066684/0098 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2023
From: LINE CORPORATION
To: WORKS MOBILE JAPAN CORPORATION
Reel/Frame 064807/0656 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SPELLING OF THE ASSIGNEES CITY IN THE ADDRESS SHOULD BE TOKYO, JAPAN PREVIOUSLY RECORDED AT REEL: 058597 FRAME: 0303. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 17, 2023
From: A HOLDINGS CORPORATION
To: LINE CORPORATION
Reel/Frame 062401/0490 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THE CITY SHOULD BE SPELLED AS TOKYO PREVIOUSLY RECORDED AT REEL: 058597 FRAME: 0141. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 17, 2023
From: LINE CORPORATION
To: A HOLDINGS CORPORATION
Reel/Frame 062401/0328 →
CHANGE OF NAME Recorded Dec 28, 2021
From: LINE CORPORATION
To: A HOLDINGS CORPORATION
Reel/Frame 058597/0141 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2021
From: A HOLDINGS CORPORATION
To: LINE CORPORATION
Reel/Frame 058597/0303 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2021
From: SHIN, JIEUN; HWANG, GIL HWAN; JEONG, YE LIM; LIM, DAI-HYUN; LEE, SUMEE; OH, HYANGNAM; JANG, JUNGHOON; JUNG, NAMKYU; KIM, GEONMIN; KIM, DEOK SU; KIM, HYUNJIN; SOHN, DAIGEUN; SONG, YEON SOO; LEE, DONGYEOL; LEE, JIN-MYOUNG; JEON, YUJIN; CHOI, JONGMIN
To: NAVER CORPORATION; LINE CORPORATION
Reel/Frame 057705/0616 →
Priority Claims (4)
KR 10-2020-0122847 · Sep 23, 2020 · national
KR 10-2020-0127519 · Sep 29, 2020 · national
KR 10-2020-0133224 · Oct 15, 2020 · national
KR 10-2020-0137324 · Oct 22, 2020 · national
Continuity (1)
Related Publication 20220093103A1 · Mar 24, 2022