Gathering and contributing content across diverse sources
A content unification system is described herein that aligns related content from various sources into one unified stream. The system leverages tagging (e.g., hashtags or other added metadata associated with content) to filter and connect related content across sites, formats, and sources. One component of the system is a user-friendly and customizable “dash board” view of the various topics, called a “tagboard”. Tagboards can be quickly and easily created by users and can be embedded on any website. Users can interact with various content sources such as blogs, forums, or services without leaving the tagboard they are viewing. The content unification system provides users the tools to make the web efficient, increase user interaction, and increase the signal to noise ratio. The system also allows site owners and publishers to monetize their traffic better by directing advertisements to their content in real time.
1. A computer-implemented method in a content unification system to gather data from various sources in response to a retrieval request, the method comprising:
receiving a request to retrieve data related to a particular data classification;
identifying one or more data sources from which to retrieve data to satisfy the request to retrieve data;
retrieving data from each of the identified data sources, wherein the data includes content potentially related to the particular classification identified in the request;
identifying one or more classifications associated with the retrieved data;
converting the retrieved data to a normalized format by removing information from the data that affects redisplay of the data;
associating metadata with each retrieved data item that describes which data source each data item originated from and one or more data classifications identified as associated with each data item; and
storing the normalized data and associated metadata in a data store for subsequent retrieval,
wherein the preceding steps are performed by at least one processor.
2. The method of claim 1 wherein receiving the request comprises determining that a user visited a web page associated with a particular hashtag occurring within data from multiple websites.
3. The method of claim 1 wherein receiving the request comprises the system initiating its own automatic request to retrieve data for a popular data classification.
4. The method of claim 1 wherein identifying data sources comprises accessing configuration information that specifies one or more specific web sites containing data to retrieve.
5. The method of claim 1 wherein identifying data sources comprises determining a uniform resource locator (URL) for each data source that indicates how the data source can be accessed to retrieve data.
6. The method of claim 1 wherein identifying data sources comprises identifying a data provider software library associated with each data source that the system will invoke to retrieve the data and perform source-specific handling for each data source.
7. The method of claim 1 wherein retrieving data comprises invoking an application programming interface (API) provided by each data source to identify and gather data from the identified data sources.
8. The method of claim 1 wherein retrieving data comprises retrieving associated information, including at least one of data classifications, timestamps associated with the data, geographical data, an author associated with the data, and information for interacting with the data.
9. The method of claim 1 wherein retrieving data comprises intercepting a retrieval request before retrieving data from the data sources, and accessing locally stored, previously cached data instead.
10. The method of claim 1 wherein identifying one or more classifications comprises performing a text search of the retrieved data to identify one or more hashtags embedded within the data.
11. The method of claim 1 wherein converting the retrieved data comprises removing select markup instructions within the data.
12. The method of claim 1 wherein converting the retrieved data produces a uniform set of data that can be redisplayed in the same way regardless of the original data source from which each data item is retrieved.
13. The method of claim 1 wherein associating metadata comprises associating one or more of a time a data item was posted, a location the data item was posted, and who authored the data item.
14. The method of claim 1 wherein associating metadata comprises adding metadata that contains interaction information that indicates what actions are valid for interacting with a particular data item.
15. The method of claim 1 wherein storing the normalized data comprises storing retrieved data that originated in a variety of formats and from multiple data sources and storing the data in the data store in a uniform format that allows for efficient redisplay of any stored data item regardless of the format and data source from which the data item originated.