System for synchronization of captioning to smart devices
View Patent ↗An automatic personal enhancement system that is tailored to each user's special access needs. The system having a multimedia presentation device, an onsite node having a user protocol time code message protocol, and the system sending a real-time time-code to the system, over a private LAN, a transmission device to transmit information to the onsite node, a fleet management subsystem, a content database server, a smart device, and a web server. The web server having instructions for: identifying a user entering a venue; connecting to the show control subsystem; selecting a language for multimedia pretensions from the show control subsystem; tracking the user's smart device with the personal enhancement system for automatically playing multimedia associated with the location in the venue in proximity to the user's smart device; and stopping the tracking and disconnecting the personal enhancement system from the user's smart device after the user exits the venue.
1 . An automatic personal enhancement system that is tailored to each user's special access needs comprising:
a. at least one presentation device, wherein the at least one presentation device is a multimedia device;
b. at least one onsite node operably connected to the at least one presentation device, wherein the at least one onsite node comprises a user protocol time code message protocol, and wherein the at least one onsite node sends a real-time time-code to the system, over a private local area network;
c. at least one transmission device operably connected to the at least one onsite node, wherein the at least one transmission device is configured to transmit information to the onsite node wired, wirelessly, or both wired and wirelessly;
d. a fleet management subsystem that is operably connected to the at least one transmission device;
e. at least one content database server that is operably connected to the fleet management subsystem;
f. at least one smart device operably connected to at least one web server; and
g. the at least one web server operably connected to the fleet management subsystem and the at least one content database server, wherein the at least one web server comprises instructions operable on one or more than one processor for:
1) identifying a user entering a venue equipped with the system;
2) connecting to the show control subsystem;
3) selecting a language for multimedia pretensions from the show control subsystem;
4) tracking the user's smart device with the personal enhancement system for automatically playing multimedia associated with the location in the venue in proximity to the user's smart device; and
5) stopping the tracking and disconnecting the personal enhancement system from the user's smart device after the user exits the venue;
wherein the system further comprises instructions for:
interpreting timing based solely on listening to an incoming audio feed;
synchronizing a media source with subtitles or other associated content without the need for onsite node programming or networking, where programming isn't feasible due to complexity, cost, where security is a concern due to legacy media, and where digital time codes are not an available;
ingesting target media in advance;
dividing the target media into snippets;
creating metadata from the snippets;
uploading the metadata to a data server; and
listening to incoming audio over an analog input port, and the at least one onsite node downloading and comparing the incoming audio to the snippets and metadata; wherein the system regularly samples an incoming audio stream and compares the sample to a library of snippets to develop a confidence rating using a cross-correlation algorithm, and wherein the system outputs a time code of the moment of highest correlation across all of the snippets.
2 . The system of claim 1 , wherein the at least one onsite node comprises instructions operable on one or more than one processor to send a time code to a specific local internet protocol address for the user to access on the one or more than one smart device.
3 . The system of claim 2 , wherein the at least one onsite node further comprises instructions to:
a. rate-limit and standardize messages and transmits the messages to an application program interface; and
b. transmit an industry time code from preexisting onsite nodes to cloud-based application program interface hosted on a least one at least one presentation device, to synchronize the system across multiple physical locations.
4 . The system of claim 1 , wherein the fleet management subsystem comprises instructions operable on one or more than one processor for a secure container-based technology stack that enables a subtext manager to deploy, manage, and scale fleets of internet of things devices.
5 . The system of claim 1 , wherein the system further comprises an application program interface for receiving time codes, interfacing with the at least one content database and outputting multimedia to the user's smart device.
6 . The system of claim 1 , wherein the system further comprises instructions operable on one or more than one processor for at least one presentation device-less application program interface to connect users to the system without additional software added to the at least one smart device, and wherein the users connect to the at least one web server by scanning a QR code, a bar code or a RFID tag or an NFC device, and select a language that opens a streaming connection to the application program interface and the show control subsystem.
7 . The system of claim 1 , wherein an at least one onsite node rate-limits and standardizes messages and transmits them to the application program interface on the at least one web server.
8 . The system of claim 1 , wherein when the application program interface receives a new time-code, a multimedia file stored in the at least one content database server is propagated to the users connected smart device, wherein the multimedia file is coincident with the venue location.
9 . The system of claim 1 , wherein the system is automatic and operates without an operator and delivers multimedia content enhancements for both impaired and non-impaired users, along with translation services directly to the user's smart device without the need of downloading an extra application to the smart device.
10 . The system of claim 1 , wherein connecting to the show control subsystem is made by the user scanning a code provided by the venue with a user's smart device.
11 . The system of claim 1 , wherein the connection to the show control subsystem is wireless.
12 . The system of claim 1 , wherein other user assistance options are made available so that any user is able to maximize the personal experience at the venue, where the other user assistance options are selected from the group consisting of closed captioning, volume, and vibration.
13 . The system of claim 1 , wherein a direct connection between the user's smart device and the user's hearing aids, or visual aids.
14 . The system of claim 1 , wherein the user can manually activate or deactivate the automatic personalized enhancements.
15 . The system of claim 1 , wherein prior to exiting the venue, gifts, tickets, discounts and other offers can be displayed to the user on the smart device for immediate purchase or redemption at a later date.
16 . The system of claim 1 , wherein impaired users that are deaf, hard of hearing, or blind can receive emergency alerts using the show control subsystem presenting auditory, vibratory and visual alerts.
17 . The system of claim 1 , wherein both impaired and non-impaired users are informed of any emergency situation and where to go or how to proceed as an additional communication method for the venue.
18 . A method for using the automatic personal enhancement system that is tailored to each user's special access needs, the method comprising the steps of:
a. entering a venue equipped with the system;
b. connecting to the show control subsystem, wherein the connection can be made by scanning a code provided by the venue with a user's smart device;
c. selecting a language for multimedia presentations from the show control subsystem;
d. tracking the user's smart device the personal enhancement system for automatically playing multimedia associated with the location in the venue in proximity to the user's smart device; and
e. exiting the venue, wherein tracking is stopped and the personal enhancement system disconnects from the user's smart device;
f. interpreting timing based solely on listening to an incoming audio feed;
g. synchronizing a media source with subtitles or other associated content without the need for onsite node programming or networking, where programming isn't feasible due to complexity, cost, where security is a concern due to legacy media, and where digital time codes are not an available;
h. ingesting target media in advance;
i. dividing the target media into snippets;
j. creating metadata from the snippets;
k. uploading the metadata to a data server; and
l. listening to incoming audio over an analog input port, and the at least one onsite node downloading and comparing the incoming audio to the snippets and metadata; wherein the system regularly samples an incoming audio stream and compares the sample to a library of snippets to develop a confidence rating using a cross-correlation algorithm, and wherein the system outputs a time code of the moment of highest correlation across all of the snippets.