IP Library › Granted Patent US 10,002,596
Granted Patent B2
US 10,002,596 · App. 15/198,499 · Granted Jun 19, 2018

Intelligent crossfade with separated instrument tracks

Inventors: Miikka Tapani Vilermo (Siuro, FI); Arto Juhani Lehtiniemi (Lempaala, FI); Lasse Juhani Laaksonen (Nokia, FI); Mikko Tapio Tammi (Tampere, FI)
Assignee: Nokia Technologies Oy
G10H1/0008H04H60/04G10H2210/056G10H2210/076G10H2210/125
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,002,596
App. No.
15/198,499
Granted
Jun 19, 2018
Kind
B2
Abstract

A method is provided including separating a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each instrument track of each of the first plurality and second plurality corresponds to a type of instrument; selecting a first instrument track from the first plurality of instrument tracks and a second instrument track from the second plurality of instrument tracks based at least on the type of instrument corresponding to the first instrument track and the second instrument track; fading out other instrument tracks from the first plurality of instrument tracks; performing a crossfade between the first instrument track and the second instrument track; and fading in other instrument tracks from the second plurality of instrument tracks.

Claims (60)

1. A method comprising:

separating a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each instrument track of each of the first plurality and second plurality corresponds to a type of instrument;

selecting a first instrument track from the first plurality of instrument tracks and a second instrument track from the second plurality of instrument tracks based at least on a similarity of the first instrument track and the second instrument track;

fading out other instrument tracks from the first plurality of instrument tracks;

performing a crossfade between the first instrument track and the second instrument track; and

fading in other instrument tracks from the second plurality of instrument tracks.

2. The method of claim 1 , further comprising:

determining a dominant instrument in the first plurality of instrument tracks and a corresponding instrument track in the second plurality of instrument tracks comprising the dominant instrument, wherein the selecting comprises: selecting the dominant instrument track as the first instrument track and the corresponding instrument track as the second instrument track.

3. The method of claim 2 , wherein, if none of the instrument tracks in the second plurality of instrument tracks comprises the dominant instrument, the method further comprises at least one of:

determining a different dominant instrument in the first plurality of instrument tracks, wherein each of the selected first instrument track and the selected second instrument track comprises the different dominant instrument; and

determining a similar instrument in the second plurality of instrument tracks as the dominant instrument, wherein the selected first instrument track comprises the dominant instrument, and the selected second instrument track comprises the similar instrument.

4. The method of claim 1 , wherein the method further comprises:

creating a first group of instrument tracks by finding at least one further instrument track in the first plurality that is similar to the selected first instrument track;

creating a second group of instrument tracks by finding at least one further instrument track in the second plurality that is similar to the selected second instrument track; and

performing the crossfade between the first group of instrument tracks and the second group of instrument tracks.

5. The method of claim 4 , wherein finding similar instrument tracks is based on comparing at least one of: loudness, timbre, direction, and zero-crossing rate.

6. The method of claim 1 , further comprising:

determining a difference in tempo between the first instrument track and the second instrument track; and

adjusting, during the crossfade, the tempo of at least one of the first track and the second track based on the type of instrument.

7. The method of claim 1 , wherein at least one of:

the fading out comprises fading out each track in the first plurality of instrument tracks other than the selected first instrument track;

the fading in comprises fading in each track in the second plurality of instrument tracks other than the selected second instrument track.

8. The method of claim 1 , wherein, during at least a portion of the crossfade, at least one of:

one or more instrument tracks from the first plurality of instrument tracks that are different from the selected first instrument track are silenced; and

one or more instrument tracks from the second plurality of instrument tracks that are different from the selected second instrument track are silenced.

9. The method of claim 1 , wherein the separation is based on at least one of: MPEG Spatial Audio Object Coding (SAOC) and blind single sound source separation (BSS).

10. An apparatus, comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured, with the at least one processor, to cause the apparatus to perform at least the following:

separate a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each instrument track of each of the first plurality and second plurality corresponds to a type of instrument;

select a first instrument track from the first plurality of instrument tracks and a second instrument track from the second plurality of instrument tracks based at least on a similarity of the first instrument track and the second instrument track;

fade out other instrument tracks from the first plurality of instrument tracks;

perform a crossfade between the first instrument track and the second instrument track; and

fade in other instrument tracks from the second plurality of instrument tracks.

11. The apparatus of claim 10 , wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to:

determine a dominant instrument in the first plurality of instrument tracks and a corresponding instrument track in the second plurality of instrument tracks comprising the dominant instrument, wherein the selection comprises: selection of the dominant instrument track as the first instrument track and the corresponding instrument track as the second instrument track.

12. The apparatus of claim 11 , wherein, if none of the instrument tracks in the second plurality of instrument tracks comprises the dominant instrument, the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least one of:

determine a different dominant instrument in the first plurality of instrument tracks, wherein each of the selected first instrument track and the selected second instrument track comprises the different dominant instrument; and

determine a similar instrument in the second plurality of instrument tracks as the dominant instrument, wherein the selected first instrument track comprises the dominant instrument, and the selected second instrument track comprises the similar instrument.

13. The apparatus of claim 10 , wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least the following:

create a first group of instrument tracks by finding at least one further instrument track in the first plurality that is similar to the selected first instrument track;

create a second group of instrument tracks by finding at least one further instrument track in the second plurality that is similar to the selected second instrument track; and

perform the crossfade between the first group of instrument tracks and the second group of instrument tracks.

14. The apparatus of claim 13 , wherein finding similar instrument tracks is based on comparing at least one of: loudness, timbre, direction, and zero-crossing rate.

15. The apparatus of claim 10 , wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least the following:

determine a difference in tempo between the first instrument track and the second instrument track; and

adjust, during the crossfade, the tempo of at least one of the first instrument track and the second instrument track.

16. The apparatus of claim 15 , wherein adjustment of the tempo comprises selecting a tempo manipulation algorithm based on the type of instrument.

17. The apparatus of claim 10 , wherein at least one of:

the fade out comprises fading out each instrument track in the first plurality of instrument tracks other than the selected first instrument track; and

the fade in comprises fading in each instrument track in the second plurality of instrument tracks other than the selected second instrument track.

18. The apparatus of claim 10 , wherein the separation is based on at least one of: MPEG Spatial Audio Object Coding (SAOC) and blind single sound source separation (BSS).

19. The apparatus of claim 10 , wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least the following:

create a third file comprising the crossfade, and store the third file in the memory for audio playback.

20. A computer program product comprising a non-transitory computer-readable storage medium having computer program code embodied thereon which when executed by an apparatus causes the apparatus to perform:

separating a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each of the instruments track of each of the first plurality and the second plurality correspond to a type of instrument;

selecting a first instrument track from the first plurality of instrument tracks and a second instrument corresponding to the first instrument track and the second instrument track based at least on a similarity between the first instrument track and the second instrument track;

fading out other instrument tracks from the first plurality of instrument tracks;

performing a crossfade between the first instrument track and the second instrument track; and

fading in other instrument tracks from the second plurality of instrument tracks.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 4, 2016
From: VILERMO, MIIKKA TAPANI; LEHTINIEMI, ARTO JUHANI; LAAKSONEN, LASSE JUHANI; TAMMI, MIKKO TAPIO
To: NOKIA TECHNOLOGIES OY
Reel/Frame 039341/0191 →
Continuity (1)
Related Publication 20180005614A1 · Jan 4, 2018