IP Library Granted Patent US 11,600,370
Granted Patent B2
US 11,600,370 · App. 16/511,491 · Granted Mar 7, 2023

Utilizing athletic activities to augment audible compositions

Inventors: Justin Fraga (Portland, OR); Harold L. Lindstrom, Jr. (Portland, OR); Willoughby H. Walling (Portland, OR); Christopher Andon (Portland, OR); Kristopher J. Schultz (Columbus, OH); Eric S. Mcgary (Columbus, OH)
Assignee: NIKE, Inc.
G16H20/30A61B5/024G06F3/04842G06F3/165G10L25/21G16H20/40G10H1/0025G10H2240/085G10H2250/311H04S3/008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,600,370
App. No.
16/511,491
Granted
Mar 7, 2023
Kind
B2
Abstract

Example embodiments relate to methods and systems for playback of adaptive music corresponding to an athletic activity. A user input is received from a user selecting an existing song for audible playback to the user, the song comprising a plurality of audio layers including at least a first layer, a second layer, and a third layer. Augmented playback of the existing song to the user is initiated by audibly providing the first layer but not the second layer. Physical activity information derived from a sensor corresponding to a real-time physical activity level of a user is received. If the physical activity level of the user is above a first activity level threshold, the augmented playback of the existing song is continued by audibly providing the first layer and the second layer to the user.

Claims (64)

1. A computer-implemented method comprising:

determining, based on performing a sonic analysis of each segment of a plurality of segments of a song, values for a plurality of sonic characteristics associated with each segment;

initiating playback of one or more layers of a plurality of layers of the song;

determining, based on receiving physical activity information derived from sensor data, an activity level of a user;

determining, based on the activity level of the user and the plurality of sonic characteristics of a current segment of the song, a next segment of the song to transition to from the current segment;

determining a plurality of pathways between the current segment and the determined next segment, wherein each pathway of the plurality of pathways comprises one or more segments of the song;

for each pathway of the plurality of pathways, determining, based on a duration of the one or more segments of the song associated with the pathway, a pathway score for the pathway;

determining, based on selecting a pathway having a pathway score indicating a shortest pathway duration, a first pathway of the plurality of pathways to reach the determined next segment; and

initiating playback of the one or more segments of the song associated with the first pathway.

2. The method of claim 1 , wherein each of the plurality of layers corresponds to audio of a different musical instrument or vocals.

3. The method of claim 1 , further comprising determining, based on the activity level, one or more additional layers of the song to play.

4. The method of claim 1 , wherein the activity level is determined based on a heart rate of the user during a physical activity, when the sensor data is collected from a heart rate measuring device.

5. The method of claim 1 , wherein the activity level is determined based on at least one of a pace or a speed of the user during a physical activity, when the sensor data is collected from an accelerometer.

6. The method of claim 1 , wherein for each pathway of the plurality of pathways, the one or more segments of the song associated with the pathway have a sonic similarity, and

wherein initiating playback of the one or more segments of the song associated with the first pathway comprises:

playing, in sequential order, the one or more segments of the song associated with the first pathway, and

playing the determined next segment.

7. The method of claim 1 , wherein determining the next segment of the song to transition to comprises:

selecting, from the plurality of segments of the song, a segment having one or more sonic characteristics having a threshold level of similarity to one or more of the plurality of sonic characteristics of the current segment.

8. The method of claim 1 , wherein performing the sonic analysis further comprises:

determining, for each segment of the plurality of segments of the song, a confidence value indicating a degree of confidence in an accuracy of the determined values of the plurality of sonic characteristics associated with the segment.

9. The method of claim 1 , wherein performing the sonic analysis further comprises:

for each segment of the plurality of segments of the song:

determining an energy level of the segment; and

associating, based on the determined energy level of the segment, the segment with a physical activity exertion level, and

wherein determining the next segment of the song to transition to comprises:

determining, based on the activity level of the user, an exertion level of the user; and

determining as the next segment, a segment having an energy level associated with a physical activity exertion level corresponding to the exertion level of the user.

10. The method of claim 1 , further comprising:

outputting, to a display device, a visualization of a determined exertion level of the user while performing a physical activity and an energy level of the current segment of the plurality of segments.

11. An apparatus comprising:

a sensor;

a processor; and

a computer-readable medium storing instructions that, when executed by the processor, cause the apparatus to:

determine, based on performing a sonic analysis of each segment of a plurality of segments of a song, values for a plurality of sonic characteristics associated with each segment;

initiate playback of one or more layers of a plurality of layers of the song;

determine, based on receiving physical activity information derived from sensor data collected from the sensor, an activity level of a user;

determine, based on the activity level of the user and the plurality of sonic characteristics of a current segment of the song, a next segment of the song to transition to from the current segment;

determine a plurality of pathways between the current segment and the determined next segment, wherein each pathway of the plurality of pathways comprises one or more segments of the song;

for each pathway of the plurality of pathways, determine, based on a duration of the one or more segments of the song associated with the pathway, a pathway score for the pathway;

determine, based on selecting a pathway having a pathway score indicating a shortest pathway duration, a first pathway of the plurality of pathways to reach the determined next segment; and

initiate playback of the one or more segments of the song associated with the first pathway.

12. The apparatus of claim 11 , wherein the instructions, when executed by the processor, further cause the apparatus to:

determine a type of the sensor; and

determine, based on the determined type of the sensor, whether to determine the activity level based on a heart rate, pace, or speed of the user during a physical activity.

13. The apparatus of claim 11 , wherein for each pathway of the plurality of pathways, the one or more segments of the song associated with the pathway have a sonic similarity, and

wherein the instructions, when executed by the processor, further cause the apparatus to initiate playback of the one or more segments of the song associated with the first pathway by:

playing, in sequential order, the one or more segments of the song associated with the first pathway, and

playing the determined next segment.

14. A non-transitory, computer-readable medium storing instructions that, when executed by a processor, cause:

determining, based on performing a sonic analysis of each segment of a plurality of segments of a song, values for a plurality of sonic characteristics associated with each segment;

initiating playback of one or more layers of a plurality of layers of the song;

determining, based on receiving physical activity information derived from sensor data, an activity level of a user;

determining, based on the activity level of the user and the plurality of sonic characteristics of a current segment of the song, a next segment of the song to transition to from the current segment;

determining a plurality of pathways between the current segment and the determined next segment, wherein each pathway of the plurality of pathways comprises one or more segments of the song;

for each pathway of the plurality of pathways, determining, based on a duration of the one or more segments of the song associated with the pathway, a pathway score for the pathway;

determining, based on selecting a pathway having a pathway score indicating a shortest pathway duration, a first pathway of the plurality of pathways to reach the determined next segment; and

initiating playback of the one or more segments of the song associated with the first pathway.

15. The non-transitory, computer-readable medium of claim 14 , wherein the activity level is determined based on a heart rate of the user during a physical activity, when the sensor data is collected from a heart rate measuring device.

16. The non-transitory, computer-readable medium of claim 14 , wherein the activity level is determined based on at least one of a pace or a speed of the user during a physical activity, when the sensor data is collected from an accelerometer.

17. The non-transitory, computer-readable medium of claim 14 , wherein for each pathway of the plurality of pathways, the one or more segments of the song associated with the pathway have a sonic similarity, and

wherein the instructions, when executed by the processor, further cause initiating playback of the one or more segments of the song associated with the first pathway by:

playing, in sequential order, the one or more segments of the song associated with the first pathway, and

playing the determined next segment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2019
From: FRAGA, JUSTIN; LINDSTROM, HAROLD L., JR.; WALLING, WILLOUGHBY H.; ANDON, CHRISTOPHER; SCHULTZ, KRISTOPHER J.; MCGARY, ERIC S.
To: NIKE, INC.
Reel/Frame 049754/0450 →
Continuity (3)
Continuation 15221224 · Jul 27, 2016
Provisional Application 62199655 · Jul 31, 2015
Related Publication 20190339931A1 · Nov 7, 2019