IP Library › Granted Patent US 11,798,279
Granted Patent B2
US 11,798,279 · App. 17/301,539 · Granted Oct 24, 2023

Methods and systems for sport data extraction

Inventors: Jeffrey Scott (Oakland, CA); Markus Kurt Peter Cremer (Orinda, CA); Nishit Umesh Parekh (Woodland Hills, CA); Dewey Ho Lee (Berkeley, CA)
Assignee: Gracenote, Inc.
G06V20/42G06T7/13G06V20/62H04N21/2187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,798,279
App. No.
17/301,539
Granted
Oct 24, 2023
Kind
B2
Abstract

A computing system engages in digital image processing of received video frames to generate sport data that indicates a score and/or a time associated with a sport event. The digital image processing includes: (i) identifying a first frame region of the video frames based on the first frame region depicting a scoreboard; (ii) executing a first procedure that analyzes the identified first frame region to detect, within the identified first frame region, second frame region(s) based on the second frame region(s) depicting text of the scoreboard; (iii) in response to detecting the second frame region(s), executing a second procedure to recognize the text in at least one of the second frame region(s); and (iv) based at least on the recognizing of the text, generating the sport data. In response to completing the digital image processing, the computing system then carries out an action based on the generated sport data.

Claims (74)

1. A method comprising:

receiving, by a computing system, video frames that depict a sport event, wherein the video frames include a depiction of data associated with the sport event, and wherein the computing system comprises a processing unit;

engaging, by the computing system, in digital image processing of the received video frames to automatically generate sport data, wherein the digital image processing comprises:

(i) identifying a first frame region of the video frames based on the first frame region depicting a scoreboard,

(ii) executing a first machine-driven pattern recognition procedure that includes analyzing the identified first frame region to detect, within the identified first frame region, one or more second frame regions based on the second frame regions depicting text of the scoreboard,

(iii) in response to detecting the second frame regions, executing a second machine-driven pattern recognition procedure to recognize the text in at least one of the one or more second frame regions, and

(iv) based at least on the recognizing of the text, generating the sport data; and

in response to completing the digital image processing, carrying out, by the computing system, an action based on the generated sport data.

2. The method of claim 1 , wherein receiving the video frames comprises receiving a real-time broadcast feed that includes the video frames.

3. The method of claim 2 , wherein engaging in the digital image processing comprises engaging in the digital image processing while receiving the real-time broadcast feed.

4. The method of claim 1 , further comprising:

receiving, by the computing system, electronic program guide (EPG) data that indicates event types respectively associated with a plurality of broadcast feeds, wherein a particular broadcast feed of the plurality includes the video frames; and

based on the received EPG data, making a determination, by the computing system, that the particular broadcast feed is associated with the sport event,

wherein engaging in the digital image processing comprises triggering the digital image processing in response to making the determination.

5. The method of claim 4 , wherein the EPG data also indicates times respectively associated with scheduling of the plurality of broadcast feeds, the method further comprising:

based on the received EPG data, determining, by the computing system, a particular time associated with scheduling of the particular broadcast feed that includes the video frames, and

wherein engaging in the digital image processing comprises, based on the determining of the particular time, triggering the digital image processing at the determined particular time.

6. The method of claim 1 , further comprising:

receiving, by the computing system, a request to trigger the digital image processing,

wherein engaging in the digital image processing comprises triggering the digital image processing in response the received request.

7. The method of claim 1 , wherein generating the sport data comprises generating sport data that indicates one or more of the following: (i) a first team name, (ii) a second team name, (iii) a first score associated with the first team name, and (iv) a second score associated with the second team name.

8. The method of claim 1 , further comprising:

receiving, by the computing system, information indicating the first frame region that depicts the scoreboard,

wherein identifying the first frame region is based on the received information indicating the first frame region.

9. The method of claim 1 , wherein identifying the first frame region comprises executing a third machine-driven pattern recognition procedure that analyzes one or more of the video frames to identify the first frame region depicting the scoreboard.

10. The method of claim 9 ,

wherein the third machine-driven pattern recognition procedure analyzes at least a particular one of the video frames to identify the first frame region depicting the scoreboard, and

wherein the first machine-driven pattern recognition procedure analyzes the first frame region within the particular video frame to detect the one or more second frame regions that depict text of the scoreboard.

11. The method of claim 9 ,

wherein the first frame region appears in at least a first and a second of the video frames,

wherein the third machine-driven pattern recognition procedure analyzes at least the first video frame to identify the first frame region depicting the scoreboard, and

wherein the first machine-driven pattern recognition procedure analyzes the first frame region within the second video frame to detect the one or more second frame regions that depict text of the scoreboard.

12. The method of claim 1 , wherein the digital image processing further comprises recording one or more of (i) a location of the identified first frame region within the video frames, (ii) an area within the video frames that the identified first frame region encompasses, (iii) a location of the detected one or more second frame regions within the video frames, and (iv) an area within the video frames that the detected one or more second frame regions encompass.

13. The method of claim 1 ,

wherein the video frames also depict a game time associated with the sport event,

wherein engaging in the digital image processing is also to automatically generate sport data that indicates the game time,

wherein the digital image processing further comprises, based at least on the recognizing of the text, generating the sport data that indicates the game time, and

wherein the action is also based on the generated sport data that indicates the game time.

14. The method of claim 1 , wherein carrying out the action comprises one or more of the following:

(i) storing the generated sport data in a data storage device,

(ii) causing presentation of the generated sport data,

(iii) outputting a report related to the generated sport data,

(iv) updating information in a user-account based on the generated sport data, and

(v) causing presentation of supplemental content based on the generated sport data.

15. A non-transitory computer readable medium having stored thereon instructions executable by a processing unit to cause a computing system to perform operations comprising:

receiving video frames that depict a sport event, wherein the video frames include a depiction of data associated with the sport event, and wherein the computing system comprises a processing unit;

engaging in digital image processing of the received video frames to automatically generate sport data, wherein the digital image processing comprises:

(i) identifying a first frame region of the video frames based on the first frame region depicting a scoreboard,

(ii) executing a first machine-driven pattern recognition procedure that includes analyzing the identified first frame region to detect, within the identified first frame region, one or more second frame regions based on the second frame regions depicting text of the scoreboard,

(iii) in response to detecting the second frame regions, executing a second machine-driven pattern recognition procedure to recognize the text in at least one of the one or more second frame regions, and

(iv) based at least on the recognizing of the text, generating the sport data; and

in response to completing the digital image processing, carrying out an action based on the generated sport data.

16. The non-transitory computer readable medium of claim 15 , wherein receiving the video frames comprises receiving a real-time broadcast feed that includes the video frames, and wherein engaging in the digital image processing comprises engaging in the digital image processing while receiving the real-time broadcast feed.

17. The non-transitory computer readable medium of claim 15 , wherein the operations further comprise:

receiving information indicating the first frame region that depicts the scoreboard,

wherein identifying the first frame region is based on the received information indicating the first frame region.

18. The non-transitory computer readable medium of claim 15 , wherein identifying the first frame region comprises executing a third machine-driven pattern recognition procedure that analyzes one or more of the video frames to identify the first frame region depicting the scoreboard.

19. The non-transitory computer readable medium of claim 15 , wherein carrying out the action comprises one or more of the following:

(i) storing the generated sport data in a data storage device,

(ii) causing presentation of the game time indicated by the generated sport data,

(iii) outputting a report related to the generated sport data,

(iv) updating information in a user-account based on the generated sport data, and

(v) causing presentation of supplemental content based on the generated sport data.

20. A computing system comprising:

a processing unit;

non-transitory data storage; and

program instructions stored in the non-transitory data storage and executable by the processing unit to:

receiving video frames that depict a sport event, wherein the video frames include a depiction of data associated with the sport event, and wherein the computing system comprises a processing unit;

engaging in digital image processing of the received video frames to automatically generate sport data, wherein the digital image processing comprises:

(i) identifying a first frame region of the video frames based on the first frame region depicting a scoreboard,

(ii) executing a first machine-driven pattern recognition procedure that includes analyzing the identified first frame region to detect, within the identified first frame region, one or more second frame regions based on the second frame regions depicting text of the scoreboard,

(iii) in response to detecting the second frame regions, executing a second machine-driven pattern recognition procedure to recognize the text in at least one of the one or more second frame regions, and

(iv) based at least on the recognizing of the text, generating the sport data; and

in response to completing the digital image processing, carrying out an action based on the generated sport data.

Assignments (4)
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 7, 2021
From: SCOTT, JEFFREY; CREMER, MARKUS KURT PETER; PAREKH, NISHIT UMESH; LEE, DEWEY HO
To: GRACENOTE, INC.
Reel/Frame 055857/0423 →
Continuity (2)
Continuation 16257339 · Jan 25, 2019
Related Publication 20210224547A1 · Jul 22, 2021