IP Library › Granted Patent US 10,951,951
Granted Patent B2
US 10,951,951 · App. 16/525,988 · Granted Mar 16, 2021

Haptics metadata in a spectating stream

Inventor: Steven Osman (San Mateo, CA)
Assignee: Sony Interactive Entertainment Inc.
H04N21/4722A63F13/285A63F13/355H04N21/4325
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,951,951
App. No.
16/525,988
Granted
Mar 16, 2021
Kind
B2
Abstract

Metadata indicating demanded haptic events during playback of an audio video (AV) stream is embedded with the AV stream, such that the metadata can be extracted during AV playback and rendered on the player's side as demanded. In this way, previously recorded video can be spectated during playback without losing whatever haptic experiences may have been generated for original playback without having to rely on merely deriving haptic events from audio.

Claims (43)

1. A device comprising:

at least one computer memory that is not a transitory signal and that comprises instructions executable by at least one processor to:

associate metadata with non-audio, non-video sensory generation information;

provide the metadata and provide packets of an audio video (AV) stream such that the AV stream can be recorded along with the metadata and played back to cause a play back apparatus to use the metadata to retrieve the associated non-audio, non-video sensory generation information and generate signals as demanded by the non-audio, non-video sensory generation information during play back of the AV stream;

receive accessibility information associated with the playback device; and

responsive to the accessibility information, remap first non-audio, non-video sensory generation information to second sensory generation information for playback of sensory signals as demanded by the second sensory generation information, the metadata being associated with the second sensory generation information.

2. The device of claim 1 , wherein the device is implemented by at least one network server.

3. The device of claim 1 , wherein the device is implemented by a device in a local area network with the playback device and/or by a storage device.

4. The device of claim 1 , wherein the non-audio, non-video sensory generation information comprises haptic information.

5. The device of claim 1 , wherein the non-audio, non-video sensory generation information comprises scent information.

6. The device of claim 1 , wherein the non-audio, non-video sensory generation information comprises taste information.

7. The device of claim 1 , wherein the instructions are executable to:

identify the non-audio, non-video sensory generation information based at least in part on capability information received from the playback device.

8. A device comprising:

at least one computer memory that is not a transitory signal and that comprises instructions executable by at least one processor to:

associate metadata with non-audio, non-video sensory generation information;

provide the metadata and provide packets of an audio video (AV) stream such that the AV stream can be recorded along with the metadata and played back to cause a play back apparatus to use the metadata to retrieve the associated non-audio, non-video sensory generation information and generate signals as demanded by the non-audio, non-video sensory generation information during play back of the AV stream;

provide first metadata corresponding to first non-audio, non-video sensory generation information in the AV stream for playback of non-audio, non-video sensory signals demanded by the first non-audio, non-video sensory generation information on a first device associated with a first player of a computer simulation comprising the AV stream; and

provide second metadata corresponding to second non-audio, non-video sensory generation information in the AV stream for playback of non-audio, non-video sensory signals demanded by the second non-audio, non-video sensory generation information on a second device associated with a second player of the computer simulation.

9. A device comprising:

at least one processor configured with instructions to:

receive at least one AV stream;

identify metadata in the AV stream;

correlate the metadata to demanded non-audio, non-video sensory playback;

present the AV stream along with non-audio, non-video sensory playback;

prior to playing back non-audio, non-video sensory playback associated with the metadata, transform the non-audio, non-video sensory playback to transformed playback based at least in part on at least one setting of the AV playback device; and

play back the transformed playback in lieu of the non-audio, non-video sensory playback.

10. The device of claim 9 , wherein the non-audio, non-video sensory playback comprises haptic playback.

11. The device of claim 9 , wherein the non-audio, non-video sensory playback comprises scent playback.

12. The device of claim 9 , wherein the non-audio, non-video sensory playback comprises taste playback.

13. The device of claim 9 , wherein the at least one setting comprises an accessibility setting.

14. The device of claim 9 , wherein the instructions are executable to:

load into memory at least one file associated with the non-audio, non-video sensory playback based at least in part on identifying a start of a chapter in the AV stream.

15. The device of claim 9 , wherein the instructions are executable to:

load into memory at least one file associated with the non-audio, non-video sensory playback based at least in part on identifying a player of a computer simulation arriving within a space or time distance of a haptic event.

16. A device comprising:

at least one processor configured with instructions to:

receive at least one AV stream;

identify metadata in the AV stream;

correlate the metadata to demanded non-audio, non-video sensory playback;

present the AV stream along with non-audio, non-video sensory playback;

present on the display a first code that can be imaged by a first device associated with a first player of a computer simulation for correlation of the first code to first demanded non-video, non-audio sensory signals for presentation of first demanded non-video, non-audio sensory playback on the first device in concert with the AV stream; and

present on the display a second code that can be imaged by a second device associated with a second player of a computer simulation for correlation of the second code to second demanded non-video, non-audio sensory signals for presentation of second demanded non-video, non-audio sensory playback on the second device in concert with the AV stream.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2019
From: OSMAN, STEVEN
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 050010/0597 →
Continuity (1)
Related Publication 20210037288A1 · Feb 4, 2021
Cited By (1)
US 12,254,134