Augmented performance synchronization
Various embodiments of the invention pertain to augmented performance synchronization systems and methods. According to some embodiments of the invention, an audio waveform may be used to generate one or more haptic waveforms for one or more electronic devices. The haptic waveforms may be generated based on any of a number of factors, including features of the audio waveform, capabilities of the haptic actuators performing the haptic waveforms, the number, type and location of devices having haptic actuators, and the like. The haptic waveforms may be synchronized with performance of the audio waveform to provide an augmented listening experience to a user.
1. A method comprising:
receiving an audio waveform at an electronic device that includes one or more speakers and a haptic actuator;
generating an actuator control signal from the audio waveform, the actuator control signal comprising discrete haptic actuation taps corresponding to peak intensities or the onset of peak intensities of the audio waveform;
generating an audio output using the audio waveform at the one or more speakers;
synchronizing transmission of the actuator control signal to the haptic actuator with generation of the audio output at the one or more speakers; and
actuating the haptic actuator with the actuator control signal while performing the audio output by the one or more speakers.
2. The method of claim 1 , further comprising modifying the actuator control signal based on metadata associated with the audio waveform.
3. The method of claim 2 , wherein the metadata associated with the audio waveform is selected from a group consisting of an artist, a genre and an album.
4. The method of claim 3 , wherein generating the actuator control signal from the audio waveform comprises filtering the audio waveform using a bandpass filter.
5. The method of claim 1 , wherein the electronic device further includes an input element, and wherein the method further comprises:
receiving a user input from the input element; and
adjusting the actuator control signal based on the user input.
6. The method of claim 1 , further comprising:
determining a contact location of the electronic device with a user of the electronic device; and
modifying the actuator control signal based on the contact location.
7. The method of claim 1 , further comprising:
identifying one or more different types of instruments contributing to audio waves making up the audio waveform; and
modifying the actuator control signal based on the identified type of music.
8. The method of claim 1 , wherein the electronic device includes a plurality of haptic actuators.
9. The method of claim 1 , wherein generating the actuator control signal includes:
extracting a feature from the audio waveform selected from a group consisting of treble, bass, beat, tempo, time signature, rhythmic patterns, loudness range, change of loudness over time, accents, melodic properties, complexity of harmony, prominent pitch classes, melody, chorus, time, verse, number of instruments, types of instruments;
applying a haptic element to the actuator control signal based on the extracted feature.
10. An electronic device comprising:
one or more speakers;
a haptic actuator;
one or more processors; and
a non-transitory computer readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:
receiving an audio waveform, wherein the audio waveform is stereophonic;
generating an actuator control signal from the audio waveform, the actuator control signal comprising discrete haptic actuation taps corresponding to peak intensities of the audio waveform; and
synchronizing transmission of the actuator control signal to the haptic actuator with transmission of the audio waveform to the one or more speakers; and
actuating the haptic actuator with the actuator control signal while performing the audio waveform by the one or more speakers.
11. The electronic device of claim 10 , wherein the haptic actuator is a linear actuator.
12. The electronic device of claim 10 , wherein the electronic device further comprises an input element, and wherein the operations further include:
receiving a user input from the input element; and
adjusting actuation of the haptic actuator based on the user input.
13. The electronic device of claim 10 , wherein the operations further include:
determining a contact location of the electronic device with a user of the electronic device; and
modifying the actuator control signal based on the contact location.
14. The electronic device of claim 10 , wherein the operations further include:
identifying one or more different types of instruments contributing to audio waves making up the audio waveform; and
modifying the actuator control signal based on the identified type of music.
15. The electronic device of claim 8 , wherein generating the actuator control signal includes:
extracting a feature from the audio waveform; and
applying a haptic element to the feature.
16. The electronic device of claim 10 , wherein the electronic device includes a plurality of haptic actuators.