IP Library Granted Patent US 8,433,431
Granted Patent B1
US 8,433,431 · App. 13/372,399 · Granted Apr 30, 2013

Displaying text to end users in coordination with audio playback

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,433,431
App. No.
13/372,399
Granted
Apr 30, 2013
Kind
B1
Abstract

Technologies described relate to coordination between audio and tracking of the corresponding text in an audio recognition mode and an audio playback mode. Optionally, audio recognition includes receiving a signal corresponding to an audio track; determining an identity of the audio track and a current audio position within the audio track; displaying on a display a portion of a text that is linked and synchronized to the audio track identified and automatically scrolling the portion of the text displayed in pace with the audio track playing; and displaying a current text indicator that emphasizes current text, wherein the current text indicator is visually synchronized on the display to current audio playing from the audio track. Optionally include redetermining the current audio position in the audio track from the signal received and updating synchronization of the current text indicator with the current audio playing.

Claims (39)

1. A method of providing text responsive to detection of audio playing, including:

receiving a signal corresponding to an audio track;

determining from the signal an identity of the audio track and a current audio position within the audio track;

displaying on a display a portion of a text that is linked and synchronized to the audio track identified and automatically scrolling the portion of the text displayed in pace with the audio track playing; and

displaying a current text indicator that emphasizes current text, wherein the current text indicator is visually synchronized on the display to current audio playing from the audio track;

wherein the display is a touchscreen, further including, responsive to user interaction with the display that scrolls the current text indicator beyond an area visible on the display, displaying an updated portion of the text as the audio track continues to play and at least temporarily suspending the automatic scrolling of the portion of text displayed until the audio track catches up with the portion of the text displayed.

2. The method of claim 1 , wherein the signal corresponds to audio captured from an external source as the audio track is playing.

3. The method of claim 1 , wherein the text is lyrics and the audio track is musical.

4. The method of claim 1 , wherein the current text indicator emphasizes a stanza or paragraph of text.

5. The method of claim 1 , further including receiving a validity interval and redetermining the current audio position in the audio track upon expiration of the validity interval from an additional portion of the signal received and updating synchronization of the current text indicator with the current audio playing.

6. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving a signal corresponding to an audio track;

determining from the signal an identity of the audio track and a current audio position within the audio track;

displaying on a display a portion of a text that is linked and synchronized to the audio track identified and automatically scrolling the portion of the text displayed in pace with the audio track playing; and

displaying a current text indicator that emphasizes current text, wherein the current text indicator is visually synchronized on the display to current audio playing from the audio track

wherein the display is a touchscreen, further including, responsive to user interaction with the display that scrolls the current text indicator beyond an area visible on the display, displaying an updated portion of the text as the audio track continues to play and at least temporarily suspending the automatic scrolling of the portion of text displayed until the audio track catches up with the portion of the text displayed.

7. The non-transitory computer storage medium of claim 6 , wherein the signal corresponds to audio captured from an external source as the audio track is playing.

8. The non-transitory computer storage medium of claim 6 , wherein the text is lyrics and the audio track is musical.

9. The non-transitory computer storage medium of claim 6 , wherein the current text indicator emphasizes a stanza or paragraph of text.

10. The non-transitory storage medium of claim 6 , wherein the computer program instructions, when executed, further cause the data processing apparatus to perform operations comprising receiving a validity interval and redetermining the current audio position in the audio track upon expiration of the validity interval from an additional portion of the signal received and updating synchronization of the current text indicator with the current audio playing.

11. A system comprising:

one or more processors configured to perform operations comprising:

receiving a signal corresponding to an audio track;

determining from the signal an identity of the audio track and a current audio position within the audio track;

displaying on a display a portion of a text that is linked and synchronized to the audio track identified and automatically scrolling the portion of the text displayed in pace with the audio track playing; and

displaying a current text indicator that emphasizes current text, wherein the current text indicator is visually synchronized on the display to current audio playing from the audio track

wherein the display is a touchscreen, further including, responsive to user interaction with the display that scrolls the current text indicator beyond an area visible on the display, displaying an updated portion of the text as the audio track continues to play and at least temporarily suspending the automatic scrolling of the portion of text displayed until the audio track catches up with the portion of the text displayed.

12. The system of claim 11 , wherein the signal corresponds to audio captured from an external source as the audio track is playing.

13. The system of claim 11 , wherein the text is lyrics and the audio track is musical.

14. The system of claim 11 , wherein the current text indicator emphasizes a stanza or paragraph of text.

15. The system of claim 11 , wherein the processors are further configured to perform operations comprising receiving a validity interval and redetermining the current audio track upon expiration of the validity interval from an additional portion of the signal received and updating synchronization of the current text indicator with the current audio playing.

16. A method of providing text responsive to detection of audio playing, including:

receiving a signal corresponding to an audio track;

determining from the signal an identity of the audio track and a current audio position within the audio track, including a validity interval;

displaying on a display a portion of a text that is synchronized to the audio track identified and automatically scrolling the portion of the text displayed in pace with the audio track playing;

displaying a current text indicator that emphasizes current text; and

upon expiration of the validity interval, automatically redetermining the current audio position in the audio track from an additional portion of the signal received and updating synchronization of the current text indicator with the current audio playing.

17. The method of claim 16 , wherein the text is lyrics and the audio track is musical.

18. The method of claim 16 , wherein the current text indicator emphasizes a stanza or paragraph of text.

Assignments (12)
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Dec 3, 2024
From: MONROE CAPITAL MANAGEMENT ADVISORS, LLC, AS COLLATERAL AGENT
To: SOUNDHOUND, INC.
Reel/Frame 069480/0312 →
SECURITY INTEREST Recorded Aug 9, 2024
From: SOUNDHOUND, INC.
To: MONROE CAPITAL MANAGEMENT ADVISORS, LLC, AS COLLATERAL AGENT
Reel/Frame 068526/0413 →
RELEASE OF SECURITY INTEREST Recorded Jun 11, 2024
From: ACP POST OAK CREDIT II LLC, AS COLLATERAL AGENT
To: SOUNDHOUND, INC.; SOUNDHOUND AI IP, LLC
Reel/Frame 067698/0845 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2023
From: SOUNDHOUND AI IP HOLDING, LLC
To: SOUNDHOUND AI IP, LLC
Reel/Frame 064205/0676 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2023
From: SOUNDHOUND, INC.
To: SOUNDHOUND AI IP HOLDING, LLC
Reel/Frame 064083/0484 →
RELEASE OF SECURITY INTEREST Recorded Apr 21, 2023
From: FIRST-CITIZENS BANK & TRUST COMPANY, AS AGENT
To: SOUNDHOUND, INC.
Reel/Frame 063411/0396 →
RELEASE OF SECURITY INTEREST Recorded Apr 19, 2023
From: OCEAN II PLO LLC, AS ADMINISTRATIVE AGENT AND COLLATERAL AGENT
To: SOUNDHOUND, INC.
Reel/Frame 063380/0625 →
SECURITY INTEREST Recorded Apr 17, 2023
From: SOUNDHOUND, INC.; SOUNDHOUND AI IP, LLC
To: ACP POST OAK CREDIT II LLC
Reel/Frame 063349/0355 →
CORRECTIVE ASSIGNMENT TO CORRECT THE COVER SHEET PREVIOUSLY RECORDED AT REEL: 056627 FRAME: 0772. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Apr 12, 2023
From: SOUNDHOUND, INC.
To: OCEAN II PLO LLC, AS ADMINISTRATIVE AGENT AND COLLATERAL AGENT
Reel/Frame 063336/0146 →
SECURITY INTEREST Recorded Jun 18, 2021
From: OCEAN II PLO LLC, AS ADMINISTRATIVE AGENT AND COLLATERAL AGENT
To: SOUNDHOUND, INC.
Reel/Frame 056627/0772 →
SECURITY INTEREST Recorded Apr 1, 2021
From: SOUNDHOUND, INC.
To: SILICON VALLEY BANK
Reel/Frame 055807/0539 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2013
From: MASTER, AARON, MR.; MOHAJER, KAMYAR, MR.; MOHAJER, KEYVAN, MR.; STONEHOCKER, TIMOTHY P., MR.; MONT-REYNAUD, BERNARD, MR.
To: SOUNDHOUND, INC.
Reel/Frame 030053/0557 →