INTERACTIVE VIEWING EXPERIENCES BY DETECTING ON-SCREEN TEXT

1 . A method comprising:

receiving, by a computer system, video data comprising a plurality of frames arranged in an order;

for a frame in the plurality of frames:

determining, by the computer system, a region containing text;

performing, by the computer system, a character recognition operation on the region to generate recognized characters;

generating, by the computer system, textual data based on the recognized characters; and

generating, by the computer system, a graphical user interface element definition corresponding to the region based on the textual data.

2 . The method of claim 1 wherein performing the character recognition operation on the region comprises performing the character recognition operation on corresponding regions containing the text in one or more other frames in the plurality of frames.

3 . The method of claim 1 further comprising accessing, by the computer system, a dictionary comprising expected textual data, and wherein generating the textual data comprises comparing the recognized characters with the expected textual data.

4 . The method of claim 1 further comprising transmitting the video data and the graphical user interface element definition from the computer system to a remote client computing device for display on the client computing device.

5 . The method of claim 1 further comprising storing, by the computer system, the video data and the graphical user interface element definition in one or more data stores accessible to a plurality of client computing devices.

6 . The method of claim 1 further comprising associating, by the computer system, the graphical user interface element definition with the frame and one or more other frames in the plurality of frames contiguous with the frame according to the order.

7 . The method of claim 1 further comprising:

generating, by the computer system, a graphical user interface element based on the graphical user interface element definition; and

associating, by the computer system, an operation to be performed in response to a user input received through the user interface element.

8 . The method of claim 7 wherein the user interface element comprises a visual representation of at least a portion of the region or the text.

9 . The method of claim 7 further comprising generating, by the computer system, a graphical user interface comprising the graphical user interface element, wherein the graphical user interface is superimposed on the frame and one or more other frames in the plurality of frames.

10 . The method of claim 7 further comprising executing, by the computer system, the operation, wherein the operation uses the textual data as input.

11 . The method of claim 10 , wherein the operation comprises generating a request for data comprising the textual data, the method further comprising:

sending the request for data from the computer system to an external data source;

receiving, in response to the request for data, additional data related to the textual data; and

generating, by the computer system, another graphical user interface comprising information based on the additional data.

12 . The method of claim 1 further comprising determining, by the computer system, metadata associated with the video data and comprising information about the content of the video data, and wherein generating the textual data is further based on the metadata.

13 . The method of claim 12 wherein determining the metadata comprises receiving electronic program guide data comprising descriptions of content of the video data.

14 . The method of claim 12 wherein determining the metadata comprises analyzing the video data to detect one or more segments of the video data.

15 . The method of claim 14 wherein the segments of the video data are defined by continuity of audio data.

16 . The method of claim 14 wherein the segments of the video data are defined by continuity of visual data.

17 . The method of claim 12 wherein determining the metadata comprises receiving a custom dictionary of expected textual data associated with the metadata or a user, and wherein generating the textual data comprises comparing the recognized characters with the custom dictionary.

18 . The method of claim 12 wherein the metadata further comprises predetermined coordinates for the region in the frame and an area, and wherein determining the region is based on the metadata.

19 . A method comprising:

receiving, by a computer system, video data comprising a plurality of frames arranged in an order;

for a frame in the plurality of frames:

determining, by the computer system, contextual data associated with the video data;

determining, by the computer system, a region containing text based on the contextual data;

performing, by the computer system, a character recognition operation on the region to generate recognized characters;

generating, by the computer system, textual data based on the recognized characters; and

generating, by the computer system, a graphical user interface element definition corresponding to the region based on the textual data

20 . A computing system comprising:

one or more processors; and

a memory comprising instructions that, when executed by the processors, configure the one or more processors to be configured to:

receive video data comprising a plurality of frames arranged in an order;

for a frame in the plurality of frames:

determine a region containing text;

perform a character recognition operation on the region to generate recognized characters;

generate textual data based on the recognized characters; and

generate a graphical user interface element definition corresponding to the region based on the textual data.