IP Library Granted Patent US 9,934,660
Granted Patent B2
US 9,934,660 · App. 15/404,380 · Granted Apr 3, 2018

Systems and methods for generating haptic effects associated with an envelope in audio signals

Inventors: Juan Manuel Cruz-Hernandez (Montreal, CA); Jamal Saboune (Montreal, CA)
Assignee: Immersion Corporation
G08B6/00G06F3/165H04L27/265
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,934,660
App. No.
15/404,380
Granted
Apr 3, 2018
Kind
B2
Abstract

Systems and methods for generating haptic effects associated with envelopes in audio signals are disclosed. One disclosed system for outputting haptic effects includes a processor configured to: receive an audio signal; determine an envelope associated with the audio signal; determine a haptic effect based in part on the envelope; and output a haptic signal associated with the haptic effect.

Claims (62)

1. A system for outputting haptic effects comprising:

a processor configured to:

receive an audio signal;

receive data associated with a haptic track;

synchronize the haptic track to the audio signal based in part on an envelope associated with the audio signal;

determine a haptic effect based in part on the audio signal and the haptic track;

store the haptic effect in the haptic track on a data store;

output a haptic signal associated with the haptic effect; and

output the audio signal to an audio output device configured to output an audible effect.

2. The system of claim 1 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

3. The system of claim 1 , wherein the processor is configured to determine the envelope by:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determining a normalized signal by normalizing the magnitudes; and

determining positive values from the normalized signal.

4. The system of claim 3 , wherein the time windows are overlapping.

5. The system of claim 4 , wherein the envelope is the positive values of the normalized signal.

6. The system of claim 1 , wherein the processor is configured to determine the envelope by:

determining a power spectral density of the audio signal within two or more time windows;

determining a sum signal by summing the power spectral density values for each time window;

sampling the sum signal to determine a sample signal; and

interpolating and normalizing the sample signal.

7. A method for outputting haptic effects comprising:

receiving an audio signal;

receiving data associated with a haptic track;

synchronizing the haptic track to the audio signal based in part on an envelope associated with the audio signal;

determining a haptic effect based in part on the audio signal and the haptic track;

storing the haptic effect in the haptic track on a data store;

outputting a haptic signal associated with the haptic effect; and

outputting the audio signal to an audio output device configured to output an audible effect.

8. The method of claim 7 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

9. The method of claim 7 , wherein the envelope associated with the audio signal comprises a signal determined by:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determining a normalized signal by normalizing the magnitudes; and

determining positive values from the normalized signal.

10. The method of claim 9 , wherein the time windows are overlapping.

11. The method of claim 9 , wherein the envelope is the positive values of the normalized.

12. The method of claim 7 , wherein the envelope associated with the audio signal comprises a signal determined by:

determining a power spectral density of the audio signal within two or more time windows;

determining a sum signal by summing the power spectral density values for each time window;

sampling the sum signal to determine a sample signal; and

interpolating and normalizing the sample signal.

13. The method of claim 12 , wherein the envelope is the interpolated and normalized sample signal.

14. The method of claim 7 , further comprising filtering the envelope.

15. The method of claim 14 , wherein filtering the envelope signal comprises one or more of: filtering the envelope signal using a threshold and filtering the envelope using a level of haptic silence.

16. A non-transitory computer readable medium comprising program code, which when executed by a processor, is configured to cause the processor to:

receive an audio signal;

receive data associated with a haptic track;

synchronize the haptic track to the audio signal based in part on an envelope associated with the audio signal;

determine a haptic effect based in part on the audio signal and the haptic track;

store the haptic effect in the haptic track on a data store;

output a haptic signal associated with the haptic effect; and

output the audio signal to an audio output device configured to output an audible effect.

17. The non-transitory computer readable medium of claim 16 , wherein the haptic effect comprises one or more of: a variation in coefficient of friction, a simulated texture, or a vibration.

18. The non-transitory computer readable medium of claim 16 , wherein the envelope associated with the audio signal comprises a signal determined by:

determining a Fast Fourier Transform of the audio signal within two or more time windows;

determining a magnitude for the Fast Fourier Transform for each time window;

determining a normalized signal by normalizing the magnitudes; and

determining positive values from the normalized signal.

19. The non-transitory computer readable medium of claim 18 , wherein the time windows are overlapping.

20. The non-transitory computer readable medium of claim 18 , wherein the envelope is the positive values of the normalized signal.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 5, 2017
From: CRUZ-HERNANDEZ, JUAN MANUEL; SABOUNE, JAMAL
To: IMMERSION CORPORATION
Reel/Frame 042905/0284 →
Continuity (3)
Continuation 14078442 · Nov 12, 2013
Provisional Application 61874933 · Sep 6, 2013
Related Publication 20170162009A1 · Jun 8, 2017