Resolution upscaling for event detection
A game-agnostic event detector can be used to automatically identify game events. Game-specific configuration data can be used to specify types of pre-processing to be performed on media for a game session, as well as types of detectors to be used to detect events for the game. Event data for detected events can be written to an event log in a form that is both human- and process-readable. The event data can be used for various purposes, such as to generate highlight videos or provide player performance feedback. The event data may be determined based upon output from detectors such as optical character recognition (OCR) engines, and the regions may be upscaled and binarized before OCR processing.
1. A computer-implemented method, comprising:
receiving video data corresponding to a session;
determining an event region in a frame of the video data, the event region including one or more objects associated with a type of event;
upscaling image data for the event region to obtain a higher graphical resolution for the event region;
processing the upscaled image data to recognize the one or more objects present in the event region;
determining, using a neural network, an inference of an occurrence of an event, the neural network determining the inference based on two or more correlated event cues and the one or more objects recognized from the event region in the higher graphical resolution; and
providing event data corresponding to the determined event.
2. The computer-implemented method of claim 1 , further comprising:
performing pre-processing of the image data before processing using optical character recognition (OCR), the pre-processing including binarization of the upscaled image data to remove background values from pixels of the event region.
3. The computer-implemented method of claim 1 , further comprising:
determining a size of the event region, the size including at least a minimum amount of pixel padding around the one or more objects.
4. The computer-implemented method of claim 1 , further comprising:
determining an upscaling algorithm to use for upscaling the image data, the upscaling algorithm including at least one of a bi-cubic, bilinear, edge-based, or fractal upscaling algorithm.
5. The computer-implemented method of claim 1 , further comprising:
determining the event based at least in part upon a current state, or change in state, of the one or more objects in the region.
6. The computer-implemented method of claim 1 , further comprising:
detecting the region using a pattern recognition algorithm.
7. The computer-implemented method of claim 1 , wherein the event region corresponds to a heads-up display (HUD) presented over rendered content.
8. The computer-implemented method of claim 1 , further comprising:
analyzing a periodic subset of frames of a video sequence to determine a plurality of events in the session.
9. The computer-implemented method of claim 1 , further comprising:
performing pre-processing of the image data, the pre-processing including at least one of filtering, stretching, warping, perspective correction, noise removal, color space transform, color isolation, or value thresholding.
10. The computer-implemented method of claim 1 , further comprising:
receiving, from an OCR engine, at least one of the two or more correlated event cues; and
utilizing at least one cue-to-event translation algorithm to determine the event data based, at least in part, upon the two or more correlated event cues.
11. The computer-implemented method of claim 1 , further comprising:
receiving the video data in the form of a file or stream during, or after, the session.
12. A computer-implemented method, comprising:
determining, in a frame of video data, an object region including one or more objects;
upscaling image data for the object region to obtain a higher graphical resolution for the object region than for the frame of video data;
performing pre-processing of the upscaled image data, the pre-processing including binarization of the upscaled image data to remove background values from pixels of the object region;
processing the upscaled and pre-processed image data to recognize the one or more objects present in the object region; and
providing content relating to the recognized one or more objects based at least in part on a neural network determination of an event, the neural network to compute an inference of an occurrence of the event, the neural network determining the inference based on two or more correlated event cues and the one or more objects recognized from the event region in the higher graphical resolution.
13. The computer-implemented method of claim 12 , further comprising:
determining a size of the object region, the size including at least a minimum amount of padding around the one or more objects.
14. The computer-implemented method of claim 12 , further comprising:
determining an upscaling algorithm to use for upscaling the image data, the upscaling algorithm including at least one of a bi-cubic, bilinear, edge-based, or fractal upscaling algorithm.
15. The computer-implemented method of claim 12 , further comprising:
determining the content to be provided based at least in part upon a current state, or change in state, of the one or more objects in the object region.
16. The computer-implemented method of claim 12 , further comprising:
detecting the object region using a pattern recognition algorithm.
17. A system comprising:
one or more processors; and
memory including instructions that, when executed by the one or more processors, cause the system to:
receive video data corresponding to a session for a user;
determine an event region in a frame of the video data, the event region including one or more objects associated with a type of event;
upscale image data for the event region to obtain a higher graphical resolution for the event region;
process the upscaled image data to recognize the one or more objects present in the event region;
determine, using a neural network, an inference of an occurrence of an event the neural network determining the inference based on two or more correlated event cues and the one or more objects recognized from the event region in the higher graphical resolution; and
provide event data corresponding to the determined event.
18. The system of claim 17 , wherein the instructions when executed further cause the system to:
perform pre-processing of the image data before processing using optical character recognition (OCR), the pre-processing including binarization of the upscaled image data to remove background values from pixels of the event region.
19. The system of claim 17 , wherein the instructions when executed further cause the system to:
determine a size of the event region, the size including at least a minimum amount of padding around the one or more objects.
20. The system of claim 17 , wherein the instructions when executed further cause the system to:
determine an upscaling algorithm to use for upscaling the image data, the upscaling algorithm including at least one of a bi-cubic, bilinear, edge-based, or fractal upscaling algorithm.