IP Library Granted Patent US 9,619,980
Granted Patent B2
US 9,619,980 · App. 14/078,445 · Granted Apr 11, 2017

Systems and methods for generating haptic effects associated with audio signals

Inventors: Juan Manuel Cruz-Hernandez (Montreal, CA); Jamal Saboune (Montreal, CA); Vincent Levesque (Montreal, CA); Ali Modarres (Montreal, CA)
G08B6/00G06F3/016G06F3/165G06F3/167G10L15/26
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,619,980
App. No.
14/078,445
Granted
Apr 11, 2017
Kind
B2
Abstract

Systems and methods for generating haptic effects associated with audio signals are disclosed. One disclosed system for outputting haptic effects includes a processor configured to: receive an audio signal; determine a haptic effect based in part on the audio signal by: identifying one or more components in the audio signal; and determining a haptic effect associated with the one or more components; and output a haptic signal associated with the haptic effect.

Claims (75)

1. A system for outputting haptic effects comprising:

a processor configured to:

receive an audio signal;

determine a haptic effect based in part on the audio signal by:

identifying one or more components in the audio signal by isolating a source of an audio effect in the audio signal and isolating one or more background noises in the audio signal; and

determining a haptic effect associated with the source of the audio effect; and

store the haptic effect in a haptic track on a data store; and

output a haptic signal associated with the haptic effect.

2. The system of claim 1 , wherein identifying one or more components in the audio signal comprises:

receiving a signal associated with one or more components;

dividing the audio signal into a plurality of segments;

analyzing one of the plurality of segments to determine if one or more of the components is present; and

classifying the segment based on the presence of one or more of the components.

3. The system of claim 2 , wherein analyzing one of the plurality of audio segments comprises implementing an acoustic event detection technique.

4. The system of claim 3 , wherein the acoustic event detection technique comprises automatic speech recognition.

5. The system of claim 2 , wherein classifying one of the plurality of audio segments comprises comparing the audio segment to a database of data associated with the one or more components.

6. The system of claim 2 , wherein determining a haptic effect comprises assigning a haptic effect based on the classification of the segment.

7. The system of claim 1 , wherein determining a haptic effect associated with the one or more components comprises assigning a haptic effect to the source.

8. The system of claim 1 , wherein isolating the source of an audio effect in the audio signal comprises isolating speech in the audio signal.

9. The system of claim 8 , wherein determining a haptic effect associated with the one or more components comprises:

determining one or more emotions associated with the speech; and

determining a haptic effect associated with the one or more emotions.

10. The system of claim 9 , wherein determining one or more emotions associated with the speech comprises one or more of: a frequency based analysis, a time based analysis, a voice quality analysis, or comparing features of the speech to a database of data associated with emotions in speech.

11. The system of claim 1 , further comprising:

an audio output device configured to receive the audio signal and output an audible effect; and

a haptic output device in communication with the processor, the haptic output device configured to receive the haptic signal and output the haptic effect.

12. The system of claim 1 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

13. A method for outputting haptic effects comprising:

receiving an audio signal;

determining a haptic effect based in part on the audio signal by:

identifying one or more components in the audio signal by isolating a source of an audio effect in the audio signal and isolating one or more background noises in the audio signal; and

determining a haptic effect associated with the source of the audio effect;

storing the haptic effect in a haptic track on a data store; and

outputting a haptic signal associated with the haptic effect.

14. The method of claim 13 , wherein identifying one or more components in the audio signal comprises:

receiving a signal associated with one or more components;

dividing the audio signal into a plurality of segments;

analyzing one of the plurality of segments to determine if one or more of the components is present; and

classifying the segment based on the presence of one or more of the components.

15. The method of claim 14 , wherein analyzing one of the plurality of audio segments comprises implementing an acoustic event detection technique.

16. The method of claim 15 , wherein the acoustic event detection technique comprises automatic speech recognition.

17. The method of claim 14 , wherein classifying one of the plurality of audio segments comprises comparing the audio segment to a database of data associated with the one or more components.

18. The method of claim 14 , wherein determining a haptic effect comprises assigning a haptic effect based on the classification of the segment.

19. The method of claim 13 , wherein determining a haptic effect associated with the one or more components comprises assigning a haptic effect to the source.

20. The method of claim 13 , wherein isolating the source of an audio effect in the audio signal comprises isolating speech in the audio signal.

21. The method of claim 20 , wherein determining a haptic effect associated with the one or more components comprises:

determining one or more emotions associated with the speech; and

determining a haptic effect associated with the one or more emotions.

22. The method of claim 21 , wherein determining one or more emotions associated with the speech comprises one or more of: a frequency based analysis, a time based analysis, a voice quality analysis, or comparing features of the speech to a database of data associated with emotions in speech.

23. The method of claim 13 , further comprising:

outputting an audible effect; and

outputting the haptic effect.

24. The method of claim 13 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

25. A non-transitory computer readable medium comprising program code, which when executed by a processor, is configured to cause the processor to:

receive an audio signal;

determine a haptic effect based in part on the audio signal by:

identifying one or more components in the audio signal by isolating a source of an audio effect in the audio signal and isolating one or more background noises in the audio signal; and

determining a haptic effect associated with the source of the audio effect; and

store the haptic effect in a haptic track on a data store; and

output a haptic signal associated with the haptic effect.

26. The system of claim 25 , wherein identifying one or more components in the audio signal comprises:

receiving a signal associated with one or more components;

dividing the audio signal into a plurality of segments;

analyzing one of the plurality of segments to determine if one or more of the components is present; and

classifying the segment based on the presence of one or more of the components.

27. The non-transitory computer readable medium of claim 26 , wherein analyzing one of the plurality of audio segments comprises implementing an acoustic event detection technique.

28. The non-transitory computer readable medium of claim 27 , wherein the acoustic event detection technique comprises automatic speech recognition.

29. The non-transitory computer readable medium of claim 27 , wherein classifying one of the plurality of audio segments comprises comparing the audio segment to a database of data associated with the one or more components.

30. The non-transitory computer readable medium of claim 26 , wherein determining a haptic effect comprises assigning a haptic effect based on the classification of the segment.

31. The non-transitory computer readable medium of claim 25 , wherein determining a haptic effect associated with the one or more components comprises assigning a haptic effect to the source.

32. The non-transitory computer readable medium of claim 25 , wherein isolating the source of an audio effect in the audio signal comprises isolating speech in the audio signal.

33. The non-transitory computer readable medium of claim 32 , wherein determining a haptic effect associated with the one or more components comprises:

determining one or more emotions associated with the speech; and

determining a haptic effect associated with the one or more emotions.

34. The non-transitory computer readable medium of claim 33 , wherein determining one or more emotions associated with the speech comprises one or more of: a frequency based analysis, a time based analysis, a voice quality analysis, or comparing features of the speech to a database of data associated with emotions in speech.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 13, 2013
From: CRUZ-HERNANDEZ, JUAN MANUEL; SABOUNE, JAMAL; LEVESQUE, VINCENT; MODARRES, ALI
To: IMMERSION CORPORATION
Reel/Frame 031589/0783 →
Continuity (2)
Provisional Application 61874933 · Sep 6, 2013
Related Publication 20150070148A1 · Mar 12, 2015