Identification of concurrently broadcast time-based media
A real time messaging platform identifies an audio snippet of a time-based media (TBM) event. The messaging platform maintains a real time repository of concurrently broadcasting TBM events as well as a historical repository of previously broadcast TBM events. These repositories contain acoustic fingerprints of their respective TBM events. The messaging platform matches an acoustic fingerprint of the audio snippet with one of the stored acoustic fingerprints to identify the TBM event in the recorded snippet. To identify the TBM event, the messaging platform matches multiple overlapping reference audio segments of the reference audio stream with multiple test audio segments of the audio snippet. This allows the platform to account for time delays between the test and reference audio segments that would otherwise hinder the matching process.
1. A method performed by a messaging platform comprising one or more computers, wherein the messaging platform enables users subscribing to the platform to create, publish, and view content in a message stream visible to themselves and other subscribing users, the method comprising:
receiving a reference audio stream of a time-based media event;
identifying reference audio segments of the reference audio stream;
receiving, from a client device, a client audio snippet, wherein the client device is associated with a subscribing user of the platform;
identifying test audio segments of the client audio snippet;
determining that a first reference audio segment matches a first test audio segment and that the client audio snippet matches the event; and
providing to the client device, in a message stream, in response to determining that the user audio snippet matches the event, content related to the event, the content comprising content generated by other account holders or content uploaded by an account officially associated with the event.
2. The method of claim 1 , wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on audio features identified in each of the first test audio segment and the first reference audio segment.
3. The method of claim 2 , wherein determining that a first reference audio segment matches a first test audio segment comprises:
determining that a second reference audio segment matches the first test audio segment;
disambiguating between the reference audio segments including:
rescoring the first reference audio segment and the second reference audio segment using one or more additional features; and
determining that the first reference audio segment matches the first test audio segment based on the rescoring.
4. The method of claim 1 , wherein the subscribing user is associated with one or more interests;
wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on the one or more interests of the subscribing user.
5. The method of claim 4 , wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on the one or more interests comprises comparing the one or more interests associated with the subscribing user with one or more interests of an account associated with the reference audio segment.
6. The method of claim 1 , wherein providing content related to the event to the client device comprises:
determining whether the time-based media event is ongoing;
in response to determining that the time-based even is ongoing, designating the subscribing user as synchronized to the time-based media event; and
providing promotional content related to the event while the subscribing user is synchronized to the time-based media event.
7. The method of claim 1 , wherein the client audio snippet is received with a request to identify a time-based media event; and
wherein providing the content related to the event comprises providing an identification of the matching event to the client device in response to the request.
8. A messaging platform comprising one or more computers and one or more storage devices on which are stored instructions, wherein the messaging platform enables users subscribing to the platform to create, publish, and view content in a message stream visible to themselves and other subscribing users, and wherein the instructions are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a reference audio stream of a time-based media event;
identifying reference audio segments of the reference audio stream;
receiving, from a client device, a client audio snippet, wherein the client device is associated with a subscribing user of the platform;
identifying test audio segments of the client audio snippet;
determining that a first reference audio segment matches a first test audio segment and that the client audio snippet matches the event; and
providing to the client device, in a message stream, in response to determining that the user audio snippet matches the event, content related to the event, the content comprising content generated by other account holders or content uploaded by an account officially associated with the event.
9. The messaging platform of claim 8 , wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on audio features identified in each of the first test audio segment and the first reference audio segment.
10. The messaging platform of claim 9 , wherein determining that a first reference audio segment matches a first test audio segment comprises:
determining that a second reference audio segment matches the first test audio segment;
disambiguating between the reference audio segments including:
rescoring the first reference audio segment and the second reference audio segment using one or more additional features; and
determining that the first reference audio segment matches the first test audio segment based on the rescoring.
11. The messaging platform of claim 8 , wherein the subscribing user is associated with one or more interests;
wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on the one or more interests of the subscribing user.
12. The messaging platform of claim 11 , wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on the one or more interests comprises comparing the one or more interests associated with the subscribing user with one or more interests of an account associated with the reference audio segment.
13. The messaging platform of claim 8 , wherein providing content related to the event to the client device comprises:
determining whether the time-based media event is ongoing;
in response to determining that the time-based even is ongoing, designating the subscribing user as synchronized to the time-based media event; and
providing promotional content related to the event while the subscribing user is synchronized to the time-based media event.
14. The messaging platform of claim 8 , wherein the client audio snippet is received with a request to identify a time-based media event; and
wherein providing the content related to the event comprises providing an identification of the matching event to the client device in response to the request.
15. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
receiving a reference audio stream of a time-based media event at a messaging platform, wherein the messaging platform enables users subscribing to the platform to create, publish, and view content in a message stream visible to themselves and other subscribing users;
identifying reference audio segments of the reference audio stream;
receiving, from a client device, a client audio snippet, wherein the client device is associated with a subscribing user of the platform;
identifying test audio segments of the client audio snippet;
determining that a first reference audio segment matches a first test audio segment and that the client audio snippet matches the event; and
providing to the client device, in a message stream, in response to determining that the user audio snippet matches the event, content related to the event, the content comprising content generated by other account holders or content uploaded by an account officially associated with the event.
16. The non-transitory computer-readable storage media of claim 15 , wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on audio features identified in each of the first test audio segment and the first reference audio segment.
17. The non-transitory computer-readable storage media of claim 16 , wherein determining that a first reference audio segment matches a first test audio segment comprises:
determining that a second reference audio segment matches the first test audio segment;
disambiguating between the reference audio segments including:
rescoring the first reference audio segment and the second reference audio segment using one or more additional features; and
determining that the first reference audio segment matches the first test audio segment based on the rescoring.
18. The non-transitory computer-readable storage media of claim 15 , wherein the subscribing user is associated with one or more interests;
wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on the one or more interests of the subscribing user.
19. The non-transitory computer-readable storage media of claim 18 , wherein determining that the first reference audio segment matches the first test audio segment includes scoring the first reference audio segment based on the one or more interests comprises comparing the one or more interests associated with the subscribing user with one or more interests of an account associated with the reference audio segment.
20. The non-transitory computer-readable storage media of claim 15 , wherein providing content related to the event to the client device comprises:
determining whether the time-based media event is ongoing;
in response to determining that the time-based even is ongoing, designating the subscribing user as synchronized to the time-based media event; and
providing promotional content related to the event while the subscribing user is synchronized to the time-based media event.
21. The non-transitory computer-readable storage media of claim 15 , wherein the client audio snippet is received with a request to identify a time-based media event; and
wherein providing the content related to the event comprises providing an identification of the matching event to the client device in response to the request.