IP Library Granted Patent US 11,113,535
Granted Patent B2
US 11,113,535 · App. 16/678,520 · Granted Sep 7, 2021

Determining tactical relevance and similarity of video sequences

Inventors: Vivek Jayaram (Los Gatos, CA); Nikhil Mitra (Los Angeles, CA); Kevin Squire (Pacific Palisades, CA)
Assignee: Second Spectrum, Inc.
G06K9/00724G06K9/00744G06K9/6215H04N21/44008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,113,535
App. No.
16/678,520
Granted
Sep 7, 2021
Kind
B2
Abstract

Methods and systems for finding and ranking video sequences that contain tactical sequences (e.g., an uninterrupted portion of a sporting event) that are similar to tactical event content in a target video sequence, such as those occurring in sports, include learning and labelling (e.g., through use of metadata and the like) tactically significant sequences in a plurality of video segments, including a plurality of such tactically significant sequences in a single video segment.

Claims (25)

1. A method comprising:

receiving a first video feed, capturing a first filmed occurrence, the first video feed comprising a sequence of video frames captured by a camera, wherein the first video feed is a video feed that is consumable by a client device;

detecting at least one game chance sequence from the sequence of video frames based on spatiotemporal analysis that identifies game chance boundary events;

determining at least one semantic label and associated location data of at least one tactically relevant event in the at least one game chance sequence;

concatenating the at least one semantic label and the associated location data as a first labeled tactical sequence; and

identifying from a library of stored game chance sequences at least one stored game chance sequence that is similar to the at least one game chance sequence from the first video feed based on a tactical similarity of a stored labeled tactical sequence of the at least one stored game chance sequence with the first labeled tactical sequence.

2. The method of claim 1 , wherein the tactical similarity is further based on similarity of a semantic label for the stored labeled tactical sequence and the associated location data with the at least one semantic label and the associated location data of the first labeled tactical sequence.

3. The method of claim 1 , wherein a spatiotemporal pattern analysis of the at least one game chance sequence from the sequence of video frames and of the at least one stored game chance sequence indicates a substantive difference in spatiotemporal patterns therein.

4. The method of claim 1 , wherein identifying the game chance boundary events includes detecting actions that occur between labeled tactical sequences.

5. The method of claim 4 , wherein a game chance boundary event includes at least one of a game dock stoppage, a scoring attempt, a scoring deflection, an event occurring in a region of a field of play, or an event occurring outside of a field of play.

6. The method of claim 1 , wherein the game change boundary events facilitate partitioning the first video feed into one or more labeled tactical subsequences.

7. The method of claim 1 , wherein the tactical similarity of two sequences comprises each of the two sequences occurring proximal to a common location.

8. The method of claim 1 , wherein a location characteristic of a first sequence that is determined by a first level of tactical relevance and similarity computing informs a second level of tactical relevance and similarity computing performing tactical similarity comparison of the first sequence with at least one other sequence.

9. The method of claim 1 , wherein the tactical similarity of two sequences is responsive to outcomes of the two sequences.

10. The method of claim 9 , wherein the two sequences that produce different outcomes are not tactically similar.

11. The method of claim 1 , wherein the tactical similarity is determined by use of a Vantage Point (VP) tree algorithm that applies a distance metric based at least in part on one or more tactical characteristics of the first labeled tactical sequence and of the stored labeled tactical sequence.

12. The method of claim 1 , wherein the tactical similarity is determined by use of multi-dimensional distance metric that is based on a plurality of tactical characteristics of the first labeled tactical sequence and of the stored labeled tactical sequence.

13. The method of claim 1 , wherein the tactical similarity is determined by application of distance metric values for a plurality of tactical sequence characteristics via a Vantage Point (VP) tree algorithm.

14. The method of claim 13 , wherein the plurality of tactical sequence characteristics includes a player identity characteristic.

15. A method comprising:

receiving a first video feed capturing a first filmed occurrence, the first video feed comprising a sequence of video frames captured by a camera, wherein the first video feed is a video feed that is consumable by a client device;

processing the captured first video feed with a first tactical relevance and similarity computing circuit, wherein the first tactical relevance and similarity computing circuit determines, during processing, portions of the video feed that comprise location-event data for one or more semantic events;

accessing with a second tactical relevance and similarity computing circuit a tactical similarity support data structure; and

determining similarity of the one or more semantic events with tactical video sequences in a library of video sequences by determining for a plurality of tactical similarity characteristic types accessed in the tactical similarity support data structure a concurrence of instances of each of the plurality of tactical similarity characteristic types associated with at least one semantic event of the one or more semantic events and corresponding instances of tactical similarity characteristics associated with the tactical video sequences in the library of video sequences.

16. The method of claim 15 , wherein the location-event data indicates at least one of a play, a part of a chance, an event, or a chance demarcation.

Assignments (7)
SECURITY INTEREST Recorded May 1, 2026
From: GENIUS SPORTS SS, LLC
To: U.S. BANK NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 074544/0266 →
RELEASE OF SECURITY INTEREST Recorded May 1, 2026
From: CITIBANK, N.A.
To: GENIUS SPORTS SS, LLC
Reel/Frame 074544/0683 →
SECURITY INTEREST Recorded May 1, 2024
From: GENIUS SPORTS SS, LLC
To: CITIBANK, N.A.
Reel/Frame 067281/0470 →
MERGER Recorded Sep 17, 2021
From: SECOND SPECTRUM, INC.
To: GENIUS SPORTS SS, LLC
Reel/Frame 057509/0582 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2020
From: SECOND SPECTRUM SÀRL
To: SECOND SPECTRUM, INC.
Reel/Frame 051755/0043 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 5, 2019
From: MITRA, NIKHIL; SQUIRE, KEVIN
To: SECOND SPECTRUM, INC.
Reel/Frame 051194/0057 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 5, 2019
From: JAYARAM, VIVEK
To: SECOND SPECTRUM SÀRL
Reel/Frame 051194/0160 →
Continuity (1)
Related Publication 20210142066A1 · May 13, 2021