IP Library Granted Patent US 9,576,445
Granted Patent B2
US 9,576,445 · App. 14/078,442 · Granted Feb 21, 2017

Systems and methods for generating haptic effects associated with an envelope in audio signals

Inventors: Juan Manuel Cruz-Hernandez (Montreal, CA); Jamal Saboune (Montreal, CA)
Assignee: Immersion Corp.
G08B6/00G06F3/016G10L21/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,576,445
App. No.
14/078,442
Filed
Nov 12, 2013
Granted
Feb 21, 2017
Kind
B2
Art Unit
2689
USPC
340/407.1
Abstract

Systems and methods for generating haptic effects associated with envelopes in audio signals are disclosed. One disclosed system for outputting haptic effects includes a processor configured to: receive an audio signal; determine an envelope associated with the audio signal; determine a haptic effect based in part on the envelope; and output a haptic signal associated with the haptic effect.

Claims (71)

1. A system for outputting haptic effects comprising:

a processor configured to:

receive an audio signal;

determine an envelope signal associated with the audio signal, wherein determining the envelope signal comprises:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determine a haptic effect based in part on the envelope signal;

output a haptic signal associated with the haptic effect;

an audio output device configured to receive the audio signal and output an audible effect;

a data store configured to receive the haptic signal and store the haptic effect in a haptic track on the data store; and

a haptic output device in communication with the processor, the haptic output device configured to receive the haptic signal and output the haptic effect.

2. The system of claim 1 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

3. The system of claim 1 , wherein determining an envelope signal associated with the audio signal further comprises:

determining a normalized signal by normalizing the magnitudes; and

determining the positive values from the normalized signal.

4. The system of claim 3 , wherein the time windows are overlapping.

5. The system of claim 4 , wherein the envelope signal is the positive values of the normalized signal.

6. The system of claim 1 , wherein determining an envelope signal associated with the audio signal comprises:

determining a power spectral density of the audio signal within two or more time windows;

determining a sum signal by summing the power spectral density values for each time window;

sampling the sum signal to determine a sample signal; and

interpolating and normalizing the sample signal.

7. The system of claim 6 , wherein the envelope signal is the interpolated and normalized sample signal.

8. The system of claim 1 , wherein the processor is further configured to filter the envelope signal.

9. The system of claim 8 , wherein filtering the envelope signal comprises one or more of: filtering the envelope signal using a threshold and filtering the envelope signal using a level of haptic silence.

10. A method for outputting haptic effects comprising:

receiving, by a processor, an audio signal;

determining, by the processor, an envelope signal associated with the audio signal, wherein determining the envelope signal comprises:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determining a haptic effect based in part on the envelope signal;

storing a haptic signal associated with the haptic effect in a haptic track on a data store;

outputting the haptic signal to a haptic output device configured to receive the haptic signal and output the haptic effect; and

receiving the audio signal by an audio output device configured to receive the audio signal and output an audible effect.

11. The method of claim 10 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

12. The method of claim 10 , wherein determining an envelope signal associated with the audio signal further comprises:

determining a normalized signal by normalizing the magnitudes; and

determining the positive values from the normalized signal.

13. The method of claim 12 , wherein the time windows are overlapping.

14. The method of claim 12 , wherein the envelope signal is the positive values of the normalized signal.

15. The method of claim 10 , wherein determining an envelope signal associated with the audio signal comprises:

determining a power spectral density of the audio signal within two or more time windows;

determining a sum signal by summing the power spectral density values for each time window;

sampling the sum signal to determine a sample signal; and

interpolating and normalizing the sample signal.

16. The method of claim 15 , wherein the envelope signal is the interpolated and normalized sample signal.

17. The method of claim 10 , wherein the processor is further configured to filter the envelope signal.

18. The method of claim 17 , wherein filtering the envelope signal comprises one or more of: filtering the envelope signal using a threshold and filtering the envelope signal using a level of haptic silence.

19. A non-transitory computer readable medium comprising program code, which when executed by a processor, is configured to cause the processor to:

receive an audio signal;

determine an envelope signal associated with the audio signal, wherein determining the envelope signal comprises:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determine a haptic effect based in part on the envelope signal;

storing a haptic signal associated with the haptic effect in a haptic track on a data store;

output the haptic signal to a haptic output device configured to receive the haptic signal and output the haptic effect; and

output the audio signal to an audio output device configured to output an audible effect.

20. The non-transitory computer readable medium of claim 19 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

21. The non-transitory computer readable medium of claim 19 , wherein determining an envelope signal associated with the audio signal further comprises:

determining a normalized signal by normalizing the magnitudes; and

determining the positive values from the normalized signal.

22. The non-transitory computer readable medium of claim 21 , wherein the time windows are overlapping.

23. The non-transitory computer readable medium of claim 21 , wherein the envelope signal is the positive values of the normalized signal.

24. The non-transitory computer readable medium of claim 19 , wherein determining an envelope signal associated with the audio signal comprises:

determining a power spectral density of the audio signal within two or more time windows;

determining a sum signal by summing the power spectral density values for each time window;

sampling the sum signal to determine a sample signal; and

interpolating and normalizing the sample signal.

25. The non-transitory computer readable medium of claim 24 , wherein the envelope signal is the interpolated and normalized sample signal.

26. The non-transitory computer readable medium of claim 19 , wherein the processor is further configured to filter the envelope signal.

27. The non-transitory computer readable medium of claim 26 , wherein filtering the envelope signal comprises one or more of: filtering the envelope signal using a threshold and filtering the envelope signal using a level of haptic silence.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 13, 2013
From: CRUZ-HERNANDEZ, JUAN MANUEL; SABOUNE, JAMAL
To: IMMERSION CORPORATION
Reel/Frame 031589/0685 →
Continuity (2)
Provisional Application 61874933 · Sep 6, 2013
Related Publication 20150070147A1 · Mar 12, 2015