Method and device for generating and detecting a fingerprint functioning as a trigger marker in a multimedia signal
View Patent ↗A device is configured to perform a method that detects a trigger marker for an action corresponding to a segment of a multimedia signal. A fingerprint is generated based on the segment of the multimedia signal at a trigger time point. The generated fingerprint is stored in a database and communicated to the device. During playback of the multimedia signal, fingerprints of segments of the multimedia signal are generated and matched against fingerprints in the database. When a match is found, one or more associated actions for the segment are retrieved by the device. The trigger time point may be determined as a time point near or at the segment of the multimedia signal with the matched fingerprint. In this way, trigger markers for actions may be enabled without modifying the multimedia signal.
1. A system comprising:
a memory that stores instructions; and
one or more processors configured by the instructions to perform operations comprising:
playing multimedia content by a multimedia playback device;
while playing the multimedia content by the multimedia playback device, deriving fingerprints from respective segments of the multimedia content;
comparing the derived fingerprints to one or more reference fingerprints representing features of the multimedia content, wherein at least one reference fingerprint of the one or more reference fingerprints is associated with an action;
determining that one of the derived fingerprints matches the at least one reference fingerprint; and
in response to determining that the one of the derived fingerprints matches the at least one reference fingerprint, causing execution of the action associated with the at least one reference fingerprint,
wherein the action to be executed is associated with a time point indicating when, in the multimedia content, the action is to be executed.
2. The system of claim 1 , wherein the multimedia playback device comprises at least one of a display or a speaker, and
wherein playing the multimedia content on the multimedia playback device comprises at least one of (i) displaying at least some of the multimedia content on the display or (ii) playing at least some of the multimedia content via the speaker.
3. The system of claim 2 , wherein the action to be executed comprises at least one action in the group consisting of:
retrieving and displaying additional information on the display; or
retrieving and playing additional information via the speaker.
4. The system of claim 1 , wherein the action to be executed comprises playing another multimedia content instead of the multimedia content for a period of time.
5. The system of claim 1 , wherein the action to be executed comprises presenting data overlayed on the multimedia content.
6. The system of claim 5 , wherein the data overlayed on the multimedia content comprises biography data for an actor appearing in the multimedia content.
7. The system of claim 1 , wherein the action to be executed comprises presenting a link to a website.
8. The system of claim 1 , wherein the action to be executed comprises replacing national content in the multimedia content with local content.
9. The system of claim 1 , wherein the action to be executed comprises executing a control command.
10. The system of claim 1 , wherein the action to be executed comprises preparing the multimedia playback device to receive a user input.
11. The system of claim 1 , wherein the operations further comprise:
receiving, by the multimedia playback device, the one or more reference fingerprints;
responsive to receiving the one or more reference fingerprints, storing the one or more reference fingerprints in the memory for comparison with the derived fingerprints;
receiving, by the multimedia playback device, for each reference fingerprint of the at least one reference fingerprint, a representation of the action with which the reference fingerprint is associated; and
responsive to receiving each representation of the action, storing the representation in association with the reference fingerprint associated with the representation.
12. A multimedia playback device comprising:
a multimedia signal receiver configured to receive a media stream;
at least one of a display or a speaker configured to playback the media stream;
a memory that stores instructions; and
one or more processors configured by the instructions to perform operations comprising:
determining a trigger fingerprint from a segment of the media stream, wherein the trigger fingerprint is a representation of features of the segment;
matching the trigger fingerprint to a reference fingerprint, wherein the reference fingerprint is stored in a database of reference fingerprints and is associated with a reference action, and wherein the reference action is associated with a time point indicating when, in the media stream, the reference action is to be performed; and
responsive to matching the trigger fingerprint to the reference fingerprint, causing execution of the reference action at the time point.
13. The multimedia playback device of claim 12 , wherein causing the execution of the reference action at the time point comprises playing, using the at least one of the display or the speaker, a second media stream in place of the media stream at the time point.
14. The multimedia playback device of claim 12 , wherein causing the execution of the reference action at the time point comprises presenting, using the display, data overlayed on multimedia content of the media stream.
15. The multimedia playback device of claim 14 , wherein the data overlayed on the multimedia content comprises biography data for an actor appearing in the multimedia content.
16. The multimedia playback device of claim 12 , wherein the operations further comprise:
before determining the trigger fingerprint, receiving a plurality of reference fingerprints that include the reference fingerprint matched with the trigger fingerprint;
storing, in the memory, the received plurality of reference fingerprints for comparison with the trigger fingerprint;
receiving for the reference fingerprint matched with the trigger fingerprint, a representation of one or more actions with which the reference fingerprint is associated; and
responsive to receiving the representation of the one or more actions, storing the representation in association with the reference fingerprint.
17. A system comprising:
a non-transitory computer readable media that stores instructions; and
one or more processors configured by the instructions to perform operations comprising:
accessing, by the processor, a plurality of reference fingerprints generated from respective segments of multimedia content, at least one reference fingerprint of the plurality of reference fingerprints being associated with an action;
determining that a trigger fingerprint generated by a multimedia playback device matches the at least one reference fingerprint, wherein the at lease least one reference fingerprint is associated with a particular action; and
transmitting, by the processor, a representation of the particular action to a recipient device to cause the recipient device to perform the particular action,
wherein the particular action is associated with a time point indicating when, in the multimedia content, the action is to be performed.
18. The system of claim 17 , wherein the particular action comprises playing, by the recipient device, another multimedia content instead of the multimedia content for a period of time.
19. The system of claim 17 , wherein the particular action comprises presenting, by the recipient device, a link to a website.
20. The system of claim 17 , wherein the particular action comprises presenting, by the recipient device, data overlayed on the multimedia content.
21. The system of claim 20 , wherein the data overlayed on the multimedia content comprises biography data for an actor appearing in the multimedia content.
22. The system of claim 17 , wherein the particular action comprises replacing, by the recipient device, national content in the multimedia content with local content.
23. The system of claim 17 , wherein the particular action comprises executing, by the recipient device, a control command.
24. The system of claim 17 , wherein the particular action comprises preparing the recipient device to receive a user input.
25. The system of claim 17 , wherein the operations further comprise:
generating, by the one or more processors, the plurality of reference fingerprints from the respective segments of the multimedia content; and
storing, in the non-transitory computer readable media, the at least one reference fingerprint and the particular action in association with each other.
26. The system of claim 17 , wherein the particular action comprises displaying information related to a segment of multimedia content from which the at least one reference fingerprint was generated.
27. The system of claim 17 , wherein performance of the particular action occurs during playback, by the multimedia playback device, of a segment of multimedia content associated with the at least one reference fingerprint.
28. The system of claim 17 , wherein the recipient device is the multimedia playback device.
29. The system of claim 17 , wherein the operations further comprise sending, to the recipient device, the plurality of reference fingerprints.