IP Library › Granted Patent US 11,651,758
Granted Patent B2
US 11,651,758 · App. 17/063,347 · Granted May 16, 2023

Automatic orchestration of a MIDI file

Inventors: François Pachet (Paris, FR); Pierre Roy (Paris, FR); Benoit Jean Carré (Paris, FR)
Assignee: Spotify AB
G10H1/0066G10H1/0025G10H2240/021G10H2240/056G10H2240/305
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,651,758
App. No.
17/063,347
Granted
May 16, 2023
Kind
B2
Abstract

An electronic device segments a first and second MIDI files into pluralities of source segments and target segments. For each of a plurality of consecutive pairs of first and second target segments, the electronic device identifies a first source segment corresponding to the first target segment of the consecutive pair and identifies a second source segment corresponding to the second target segment of the consecutive pair, where the first and second source segments are identified by determining that the first and second source segments are harmonically conformant to the corresponding first and second target segments, and determining that a transition between the first and second source segments is graphically conformant to a transition between a consecutive pair of source segments. The electronic device generates a third MIDI file using the identified first and second source segments for each of the plurality of consecutive pairs of first and second target segments.

Claims (78)

1. A method of generating a Musical Instrument Digital Interface (MIDI) file, comprising:

at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors:

segmenting a first MIDI file into a plurality of source segments;

segmenting a second MIDI file into a plurality of target segments;

for each of a plurality of consecutive pairs of first and second target segments:

identifying a first source segment corresponding to the first target segment of the consecutive pair; and

identifying a second source segment corresponding to the second target segment of the consecutive pair;

wherein identifying the first and second source segments comprises:

determining that the first and second source segments are harmonically conformant to the corresponding first and second target segments; and

determining that a transition between the first and second source segments is graphically conformant to a transition between a consecutive pair of source segments; and

generating a third MIDI file using the identified first and second source segments for each of the plurality of consecutive pairs of first and second target segments.

2. The method of claim 1 , wherein:

the first MIDI file comprises a particular style or genre of a musical piece used as a source for orchestration;

the second MIDI file comprises a particular melody or chord progression of a musical piece targeted for orchestration; and

the third MIDI file comprises an orchestration of the particular melody or chord progression of the second MIDI file in the style or genre of the first MIDI file.

3. The method of claim 1 , wherein determining that the first and second source segments are harmonically conformant to the corresponding first and second target segments comprises:

comparing pitch profiles of the first and second source segments with pitch profiles of the corresponding first and second target segments; and

determining harmonic conformance of the first and second source segments to the corresponding first and second target segments based on the comparing.

4. The method of claim 3 , wherein comparing pitch profiles of the first and second source segments with pitch profiles of the corresponding first and second target segments comprises:

comparing Boolean matrices representing piano rolls of the first and second source segments and piano rolls of the corresponding first and second target segments.

5. The method of claim 1 , wherein determining that a transition between the first and second source segments is graphically conformant to a transition between a consecutive pair of source segments comprises:

comparing a rhythm and/or pitch transition between the first and second source segments with a rhythm and/or pitch transition between a plurality of consecutive pairs of source segments; and

determining graphical conformance of the transition between the first and second source segments to the transition between the consecutive pair of source segments based on the comparing.

6. The method of claim 5 , wherein comparing the rhythm and/or pitch transition between the first and second source segments with the rhythm and/or pitch transition between the plurality of consecutive pairs of source segments comprises:

determining a Hamming distance between merged piano rolls of the first and second source segments and merged piano rolls of the consecutive pairs of source segments.

7. The method of claim 1 , wherein generating the third MIDI file comprises:

reordering at least some of the source segments based on their correspondence to respective target segments; and

concatenating the reordered source segments.

8. An electronic device, comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to:

segment a first MIDI file into a plurality of source segments;

segment a second MIDI file into a plurality of target segments;

for each of a plurality of consecutive pairs of first and second target segments:

identify a first source segment corresponding to the first target segment of the consecutive pair; and

identify a second source segment corresponding to the second target segment of the consecutive pair;

wherein identifying the first and second source segments comprises:

determining that the first and second source segments are harmonically conformant to the corresponding first and second target segments; and

determining that a transition between the first and second source segments is graphically conformant to a transition between a consecutive pair of source segments; and

generate a third MIDI file using the identified first and second source segments for each of the plurality of consecutive pairs of first and second target segments.

9. The electronic device of claim 8 , wherein:

the first MIDI file comprises a particular style or genre of a musical piece used as a source for orchestration;

the second MIDI file comprises a particular melody or chord progression of a musical piece targeted for orchestration; and

the third MIDI file comprises an orchestration of the particular melody or chord progression of the second MIDI file in the style or genre of the first MIDI file.

10. The electronic device of claim 8 , wherein determining that the first and second source segments are harmonically conformant to the corresponding first and second target segments comprises:

comparing pitch profiles of the first and second source segments with pitch profiles of the corresponding first and second source segments; and

determining harmonic conformance of the first and second source segments to the corresponding first and second target segments based on the comparing.

11. The electronic device of claim 10 , wherein comparing pitch profiles of the first and second source segments with pitch profiles of the corresponding first and second source segments comprises:

comparing Boolean matrices representing piano rolls of the first and second source segments and piano rolls of the corresponding first and second target segments.

12. The electronic device of claim 8 , wherein determining that a transition between the first and second source segments is graphically conformant to a transition between a consecutive pair of source segments comprises:

comparing a rhythm and/or pitch transition between the first and second source segments with a rhythm and/or pitch transition between a plurality of consecutive pairs of source segments; and

determining graphical conformance of the transition between the first and second source segments to the transition between the consecutive pair of source segments based on the comparing.

13. The electronic device of claim 12 , wherein comparing the rhythm and/or pitch transition between the first and second source segments with the rhythm and/or pitch transition between the plurality of consecutive pairs of source segments comprises:

determining a Hamming distance between merged piano rolls of the first and second source segments and merged piano rolls of the consecutive pairs of source segments.

14. The electronic device of claim 8 , wherein generating the third MIDI file comprises:

reordering at least some of the source segments based on their correspondence to respective target segments; and

concatenating the reordered source segments.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by an electronic device with one or more processors, cause the one or more processors to:

segment a first MIDI file into a plurality of source segments;

segment a second MIDI file into a plurality of target segments;

for each of a plurality of consecutive pairs of first and second target segments:

identify a first source segment corresponding to the first target segment of the consecutive pair; and

identify a second source segment corresponding to the second target segment of the consecutive pair;

wherein identifying the first and second source segments comprises:

determining that the first and second source segments are harmonically conformant to the corresponding first and second target segments; and

determining that a transition between the first and second source segments is graphically conformant to a transition between a consecutive pair of source segments; and

generate a third MIDI file using the identified first and second source segments for each of the plurality of consecutive pairs of first and second target segments.

16. The non-transitory computer-readable storage medium of claim 15 , wherein determining that the first and second source segments are harmonically conformant to the corresponding first and second target segments comprises:

comparing pitch profiles of the first and second source segments with pitch profiles of the corresponding first and second source segments; and

determining harmonic conformance of the first and second source segments to the corresponding first and second target segments based on the comparing.

17. The non-transitory computer-readable storage medium of claim 16 , wherein comparing pitch profiles of the first and second source segments with pitch profiles of the corresponding first and second source segments comprises:

comparing Boolean matrices representing piano rolls of the first and second source segments and piano rolls of the corresponding first and second target segments.

18. The non-transitory computer-readable storage medium of claim 15 , wherein determining that a transition between the first and second source segments is graphically conformant to a transition between a consecutive pair of source segments comprises:

comparing a rhythm and/or pitch transition between the first and second source segments with a rhythm and/or pitch transition between a plurality of consecutive pairs of source segments; and

determining graphical conformance of the transition between the first and second source segments to the transition between the consecutive pair of source segments based on the comparing.

19. The non-transitory computer-readable storage medium of claim 18 , wherein comparing the rhythm and/or pitch transition between the first and second source segments with the rhythm and/or pitch transition between the plurality of consecutive pairs of source segments comprises:

determining a Hamming distance between merged piano rolls of the first and second source segments and merged piano rolls of the consecutive pairs of source segments.

20. The non-transitory computer-readable storage medium of claim 15 , wherein generating the third MIDI file comprises:

reordering at least some of the source segments based on their correspondence to respective target segments; and

concatenating the reordered source segments.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 19, 2023
From: SPOTIFY AB
To: SOUNDTRAP AB
Reel/Frame 064315/0727 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 3, 2021
From: PACHET, FRANÇOIS; ROY, PIERRE; CARRÉ, BENOIT JEAN
To: SPOTIFY AB
Reel/Frame 055484/0775 →
Priority Claims (1)
EP 19205553 · Oct 28, 2019 · regional
Continuity (1)
Related Publication 20210125593A1 · Apr 29, 2021