IP Library Granted Patent US 10,055,950
Granted Patent B2
US 10,055,950 · App. 15/433,488 · Granted Aug 21, 2018

Sound to haptic effect conversion system using waveform

Inventors: Satvir Singh Bhatia (San Jose, CA); Kanav Gandhi (Sunnyvale, CA); Christopher J. Ullrich (Ventura, CA); Juan Manuel Cruz-Hernandez (Montreal, CA); Herve Thu Timone (Montreal, CA); Jason Hoi Fun Lau (Foster City, CA)
Assignee: IMMERSION CORPORATION
G08B6/00G06F3/016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,055,950
App. No.
15/433,488
Granted
Aug 21, 2018
Kind
B2
Abstract

A haptic conversion system is provided that intercepts frames of audio data, such as a digital audio signal, converts the frames into a haptic signal, and plays the created haptic signal through an actuator to produce haptic effects. The haptic signal is based on a maximum value of each audio data frame, which defines a magnitude of the haptic signal. The haptic signal is applied to the actuator configured to receive the haptic signal, where the actuator utilizes the haptic signal to generate the one or more haptic effects.

Claims (44)

1. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to convert an audio signal into a haptic effect, the converting comprising:

receiving an audio signal comprising a first data frame and a second data frame;

determining a maximum amplitude value of the first data frame;

generating a haptic signal based on the maximum amplitude value of the first data frame; and

sending the haptic signal to an actuator to generate the haptic effect.

2. The non-transitory computer-readable medium of claim 1 , the converting further comprising:

decoding the audio signal into the first data frame and the second data frame.

3. The non-transitory computer-readable medium of claim 2 , the converting further comprising determining a maximum amplitude value of the second data frame.

4. The non-transitory computer-readable medium of claim 1 , the converting further comprising transforming the first data frame to generate a transformed first data frame.

5. The non-transitory computer-readable medium of claim 4 , the converting further comprising modulating the transformed first data frame.

6. The non-transitory computer-readable medium of claim 1 , further comprising transforming the first data frame by band-pass filtering the first data frame.

7. The non-transitory computer-readable medium of claim 1 , further comprising transforming the first data frame by low-pass filtering the first data frame.

8. The non-transitory computer-readable medium of claim 1 , further comprising boosting the audio signal contained within the first data frame after a transforming of the first data frame.

9. The non-transitory computer-readable medium of claim 1 , wherein the haptic signal is generated only when the maximum amplitude value of the first data frame is greater than a specified threshold.

10. A computer-implemented method for converting an audio signal into a haptic effect, the computer-implemented method comprising:

receiving an audio signal;

decoding the audio signal into a first data frame and second data frame;

determining a maximum amplitude value of the first data frame;

generating a haptic signal based on the maximum amplitude value of the first data frame; and

sending the haptic signal to an actuator to generate the haptic effect.

11. The computer-implemented method of claim 10 , further comprising: determining a maximum amplitude value of the second data frame and generating a haptic signal based on the maximum amplitude value of the second data frame.

12. The computer-implemented method of claim 10 , further comprising transforming the first data frame to generate a transformed first data frame.

13. The computer-implemented method of claim 12 , further comprising modulating the transformed first data frame.

14. A haptic conversion system comprising:

a device configured to receive an audio signal comprising a set of samples; and

an actuator configured to output a haptic effect,

wherein the device, upon receiving the set of samples of the audio signal, is configured to

resample the audio signal to generate a resampled audio signal,

transform a set of samples of the resampled audio signal to generate a transformed set of samples, wherein the transforming comprises boosting the set of samples of the resampled audio signal, and

generate a haptic signal based on the transformed set of samples, and

send the haptic signal to the actuator to output the haptic effect.

15. The haptic conversion system of claim 14 , wherein the device is further configured to modulate the transformed set of samples to generate a modulated set of samples, wherein the haptic signal is generated based on the modulated set of samples.

16. The haptic conversion system of claim 14 , wherein the device is further configured to determine a maximum amplitude value of the set of samples of the audio signal, wherein the haptic signal is generated only when the maximum amplitude value of the set of samples is greater than a specified threshold.

17. The haptic conversion system of claim 14 , wherein the set of samples of the audio signal is part of a data frame, wherein the device includes at least a communication device configured to receive the audio signal and a processor configured to generate the haptic signal, and wherein the haptic conversion system is part of a mobile device.

18. A computer-implemented method for converting an audio signal into one or more haptic effects, the computer-implemented method comprising:

receiving a set of samples of the audio signal;

transforming the set of samples to generate a transformed set of samples;

generating a haptic signal by mixing the transformed set of samples with a periodic waveform; and

sending the haptic signal to an actuator to generate the one or more haptic effects.

19. The computer-implemented method of claim 18 ,

wherein transforming the set of samples comprises filtering the set of samples to generate a filtered set of samples,

wherein the periodic waveform has an amplitude value that is based on a maximum amplitude value of the filtered set of samples, and

wherein the mixing comprises generating a weighted sum of the transformed set of samples and the periodic waveform.

20. The computer-implemented method of claim 19 , wherein the weighted sum is generated as a normalized value that is a sum of a first multiple and a second multiple, wherein the first multiple is a first value multiplied by the transformed set of samples, and the second multiple is a second value multiplied by the periodic waveform, wherein the second value is equal to 1 minus the first value, the first value being a decimal value greater than 0 and less than 1.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2017
From: BHATIA, SATVIR SINGH; GANDHI, KANAV; ULLRICH, CHRISTOPHER J.; CRUZ-HERNANDEZ, JUAN MANUEL; TIMONE, HERVE THU; LAU, JASON HOI FUN
To: IMMERSION CORPORATION
Reel/Frame 042651/0848 →
Continuity (5)
Continuation 14709672 · May 12, 2015
Continuation 14175163 · Feb 7, 2014
Continuation 13366010 · Feb 3, 2012
Provisional Application 61441792 · Feb 11, 2011
Related Publication 20170213430A1 · Jul 27, 2017
Cited By (5)
US 12,190,716 US 12,244,253 US 12,276,687 US 12,314,558 US 12,482,335