IP Library Granted Patent US 9,786,327
Granted Patent B2
US 9,786,327 · App. 14/841,262 · Granted Oct 10, 2017

Utilizing audio digital impact to create digital media presentations

Inventor: Peter Merrill (Sunnyvale, CA)
Assignee: Adobe Systems Incorporated
G11B27/30G11B27/034H04N21/4302H04N21/4307H04N21/85H04N21/8547
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,786,327
App. No.
14/841,262
Granted
Oct 10, 2017
Kind
B2
Abstract

Systems and methods disclosed create one or more digital media presentations based on impact values. In particular, in one or more embodiments, systems and methods generate impact values based on a change in a measure of energy associated with digital audio content over time. For instance, systems and methods generate impact values by calculating a difference in a measure of energy over time in relation to the amount of energy at a particular time period. Based on the generated impact values, systems and methods identify transition points in the digital audio content. Specifically, systems and methods utilize a decaying masking threshold to identify transition points from generated impact values. Moreover, systems and method utilize identified transition points to modify digital visual content displayed in conjunction with the digital audio content.

Claims (59)

1. In a digital medium environment for creating digital media presentations, a method of syncing visual media to audio media by setting transitions based on audio impact, comprising:

calculating impact values for a plurality of time periods associated with digital audio content by:

dividing the plurality of time periods associated with the digital audio content into a plurality of ranges, each range of the plurality of ranges in a first time period corresponding to a range of the plurality of ranges in a second time period; and

for each range in the second time period, calculating the difference between a measure of energy associated with the range from the digital audio content in the second time period and a measure of energy associated with the corresponding range from the digital audio content in the first time period;

selecting transition points based on the calculated impact values in the digital audio content using a decaying masking threshold that decays until a first impact value of the calculated impact values is intersected, increases in response to the intersection of the first impact value, and decays again until a second impact value of the calculated impact values is intersected, wherein the intersected impact values are utilized to select transition points; and

generating a presentation of digital visual content by syncing the digital visual content with the digital audio content based on the selected transition points.

2. The method of claim 1 , wherein the plurality of ranges are a plurality of frequency ranges and each range in the first time period is a frequency range corresponding to a frequency range in the second time period calculating.

3. The method of claim 1 , wherein calculating impact values further comprises:

for each range in the second time period, calculating a range impact value by multiplying the calculated difference between the measure of energy associated with the range from the digital audio content in the second time period and the measure of energy associated with the corresponding range from the digital audio content in the first time period by the calculated measure of energy associated with the range from the digital audio content in the second time period.

4. The method of claim 3 , wherein calculating impact values further comprises:

calculating an impact value associated with the second time period by combining the range impact values.

5. The method of claim 4 , wherein calculating impact values further comprises:

calculating a centroid of the measure of energy with regard to the digital audio content;

normalizing the measure of energy based on the centroid; and

applying a curve to the normalized measure of energy.

6. The method of claim 4 , wherein the measure of energy comprises one or more of: amplitude, power, intensity, volume or loudness.

7. The method of claim 1 wherein selecting transition points further comprises:

calculating audio interest values by comparing the decaying masking threshold and the impact values; and

applying a second decaying masking threshold that decays until a first audio interest value of the calculated audio interest values is intersected, expands in response to the intersection of the first audio interest value, and decays again until a second audio interest value of the calculated audio interest values is intersected, wherein the intersected audio interest values are utilized to select transition points.

8. The method of claim 7 , wherein:

the decaying masking threshold decays at a first rate; and

the second decaying masking threshold decays at a second rate different than the first rate.

9. The method of claim 1 , wherein the decaying masking threshold expands by a magnitude in proportion to a magnitude of the first impact value.

10. The method of claim 1 further comprising:

detecting one or more characteristics of the digital visual content; and

modifying at least one of the following based on the detected one or more characteristics of the visual content: a magnitude that the decaying masking threshold expands; a rate that the decaying masking threshold expands; a magnitude that the decaying asking threshold decays; or a rate at which the decaying masking threshold decays.

11. A system, comprising:

at least one processor;

at least one non-transitory computer readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the system to:

calculate impact values for a plurality of time periods associated with digital audio content by calculating a change in a measure of energy associated with the time periods by:

dividing the plurality of time periods associated with the digital audio content into a plurality of ranges, each range in a first time period corresponding to a range in a second time period;

for each range in the second time period, calculating a difference between a measure of energy associated with the range from the digital audio content in the second time period and a measure of energy associated with the corresponding range from the digital audio content in the first time period; and

calculating an impact value associated with the second time period by combining the calculated difference for each range;

select transition points based on the calculated impact values in the digital audio content using a decaying masking threshold that decays until a first impact value of the calculated impact values is intersected, increases in response to the intersection of the first impact value, and decays again until a second impact value of the calculated impact values is intersected, wherein the intersected impact values are utilized to select transition points; and

generate a presentation of digital visual content by syncing the digital visual content with the digital audio content based on the selected transition points.

12. The system of claim 11 , wherein the plurality of ranges are a plurality of frequency ranges and each range in the first time period is a frequency range corresponding to a frequency range in the second time period.

13. The system of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the system to:

detect one or more characteristics of the visual content; and

modify at least one of the following based on the detected one or more characteristics of the visual content: a magnitude that the decaying masking threshold expands; a rate that the decaying masking threshold expands; a magnitude that the decaying asking threshold decays; or a rate at which the decaying masking threshold decays.

14. The system of claim 11 , wherein the decaying masking threshold expands by a magnitude in proportion to a magnitude of the first impact value.

15. The system of claim 13 , wherein the one or more characteristics of the digital visual content comprises at least one of the following: a user interaction with the digital visual content, an object portrayed in the digital visual content; an individual portrayed in the digital visual content, and a type of the digital visual content.

16. In a digital medium environment for creating digital media presentations, a method of combining and presenting visual and audio media over time, comprising:

calculating impact values for a plurality of time periods associated with digital audio content by calculating a change in a measure of energy associated with the time periods;

generating audio interest values by applying a first decaying masking threshold to the calculated impact values, the first decaying masking threshold decaying at a first rate;

detecting one or more characteristics of the digital visual content;

modifying at least one of the following based on the detected one or more characteristics of the visual content: a magnitude that a second decaying masking threshold expands; a rate that the second decaying masking threshold expands; a magnitude that the second decaying masking threshold decays; or a rate at which the second decaying masking threshold decays;

selecting transition points by applying the second decaying masking threshold to the audio interest values, the second decaying masking threshold decaying at a second rate different than the first rate; and

based on the selected transition points, generating a presentation of digital visual content in conjunction with the digital audio content.

17. The method of 16 , wherein:

the first decaying mask threshold decays until a first impact value is intersected, expands in response to the intersection of the first impact value, and decays again until a second impact value is intersected;

the method further comprises generating audio interest values based on a difference between the impact values and the first decaying mask threshold;

the second decaying mask threshold decays until a first audio interest value is intersected, expands in response to the intersection of the first audio interest value, and decays again until a second audio interest value is intersected; and

the method further comprises selecting transition points based on the intersected audio interest values.

18. The method of claim 16 , wherein calculating impact values further comprises calculating a change in the measure of energy between a first time period in the plurality of time periods and a second time period in the plurality of time periods in relation to an amount of the measure of energy associated with the digital audio content at the second time period.

19. The method of claim 16 , wherein calculating impact values further comprises:

dividing the plurality of time periods associated with the digital audio content into a plurality of ranges, each range in a first time period corresponding to a range in a second time period;

for each range in the second time period, calculating a difference between a measure of energy associated with the range from the digital audio content in the second time period and a measure of energy associated with the corresponding range from the digital audio content in the first time period; and

calculating an impact value associated with the second time period by combining the calculated difference for each range.

20. The method of claim 19 wherein the one or more characteristics of the digital visual content comprises at least one of the following: a user interaction with the digital visual content, an object portrayed in the digital visual content; an individual portrayed in the digital visual content, and a type of the digital visual content.

Assignments (2)
CHANGE OF NAME Recorded Apr 8, 2019
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 048867/0882 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2015
From: MERRILL, PETER
To: ADOBE SYSTEMS INCORPORATED
Reel/Frame 036462/0049 →
Continuity (1)
Related Publication 20170062011A1 · Mar 2, 2017