IP Library Patent Application 16310762
Patent Application
App. No. 16/310,762

IMAGE PROCESSING

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
16/310,762
Abstract

A image processing system and method are provided for receiving an image with a set of feature points characteristic of the image and selecting each of the feature points to be a selected feature point. Moreover, a number of neighboring feature points associated with the selected feature point are identified and a first hash is created that includes information associated with a first pair of neighboring feature points, with the information associated with the first and second neighboring feature points representative of the relative location of these neighboring feature points to the selected feature point. Moreover, a second hash is created that includes information associated with, a second pair of neighboring feature points, with the information associated with these neighboring feature points representative of the relative location of these to the selected feature point.

Claims (54)

1 - 38 . (canceled)

39 . An image processing system for identifying an image in at least one video frame, the system comprising:

a feature point selector configured to select at least one feature point in an image having a set of feature points characteristic of the image;

a neighboring feature point identifier configured to identify a plurality of neighboring feature points associated with the selected feature point;

a hash generator configured to:

create a first hash comprising information associated with a first pair of neighboring feature points that includes first and second neighboring feature points of the identified plurality of neighboring feature points, wherein the information associated with the first and second neighboring feature points represents a location of the first pair of neighboring feature points relative to the selected feature point, and

create a second comprising information associated with a second pair of neighboring feature points that includes third and fourth neighboring feature points of the identified plurality of neighboring feature points, wherein the information associated with the third and fourth neighboring feature points represents a location of the second pair of neighboring feature points relative to the selected feature point; and

an image identifier configured to match the image with at least one matching image in an image database by comparing the first and second hashes with known hash values associated with the at least one matching image in the image database.

40 . The image processing system of claim 39 , wherein one of neighboring feature points of the second pair is a same feature point as one of the neighboring feature points of the first pair.

41 . The image processing system of claim 39 , wherein the information associated with each neighboring feature point comprises a value from a set of possible values, with each of the set of possible values represents a coarse relative location defined by a range of relative angles between the selected feature point and a neighboring feature point.

42 . The image processing system of claim 39 , wherein each of the first and second hashes comprises information associated with the selected feature point, including at least one selected feature point flag that identifies a half of the image where the feature point is located.

43 . The image processing system of claim 39 , wherein the feature point selector is further configured to select the at least one feature point in the image by splitting the image into a number of tiles, and identifying maximum and minimum points on each tile of the split image.

44 . The image processing system of claim 43 , wherein the feature point selector is further configured to:

group the selected at least one feature point points into four groups, wherein each group corresponds to the feature points contained within one quadrant of the image,

sort the plurality of feature points in each group in order of prominence, wherein prominence is determined from an absolute difference between a maxima or a minima, and an average of the respective tile,

select a number of most prominent feature points in each group as the selected feature points, while retaining a list of which feature points correspond to the minima and which to the maxima, and

combine the lists to form a list of selected feature points.

45 . The image processing system of claim 39 , wherein the image identifier is further configured to match the image by comparing the first and second hashes with a plurality of pre-calculated tables, each associated with an entity, wherein each pre-calculated table comprises a row for every possible hash value and each row is populated by image identifiers of images in the respective entity associated with a respective hash.

46 . The image processing system of claim 45 , wherein the image identifier is further configured to:

generate a histogram that represents matches between the first and second hashes and images in the respective entity, wherein the histogram comprises a column for each image of the entity with at least one hash match, and the column has a value that is equal to the number of hash matches, and

score each entity image according to the number of matches.

47 . The image processing system of claim 46 , wherein the image identifier is further configured to select the image having a highest score as the at least one matching image.

48 . The image processing system of claim 46 , wherein the image identifier is further configured to:

rank images with a score above a pre-set value in a list in order of score,

select a highest ranking image and delete all images within a pre-selected temporal range from the list,

repeat a ranking and selecting with a next highest remaining image until a lowest remaining image is reached or no lower image is available, and

further analyze the remaining images to determine a best match as the at least one matching image.

49 . The image processing system of claim 46 , wherein the image identifier is further configured to score each entity image by a weighting value depending upon matching hashes.

50 . The image processing system of claim 46 , wherein the weighting value is dependent on at least one of a closeness of the selected at least one feature point to a center of the image, a prominence of the selected at least one feature point, and a number of entity images that are matched to the first and second hashes.

51 . An image processing system for identifying an image in at least one video frame, the system comprising:

an electronic memory configured to store a pre-calculated table associated with an entity that comprises a plurality of entity images, wherein the pre-calculated table includes a plurality of rows for hash values associated with the entity images and each row is populated by a respective entity image identifier for a respective entity image associated with a respective hash value, and wherein the hash values are associated with a series of feature points characteristic of each entity image;

a hash value comparator configured to compare a series of hashes representing one or more desired images with the pre-calculated table, with the series of hashes being associated with each of a series of feature points characteristic of the one or more desired images;

a histogram generator configured to generate a histogram that represents a number of matches between the series of hashes and each entity image, with the generated histogram comprising a column for each entity image with at least one hash match;

an entity image scorer configured to score each entity image based on the number of matches; and

an image identifier configured to identify candidate entity images based on the scored entity images.

52 . The image processing system of claim 51 , wherein the image identifier is further configured to select a matching image as the candidate entity image with a highest score.

53 . The image processing system of claim 51 , wherein the image identifier is further configured to:

rank entity images with a score above a pre-set value in a list in order of score,

select a highest ranking entity image and delete all images within a pre-selected temporal range from the list,

repeat a ranking and selecting with a next highest remaining image until a lowest remaining image is reached or no lower entity image is available, and

further analyze the remaining images to determine a best match as a matching image.

54 . The image processing system of claim 51 , further comprising:

an entity partitioner configured to partition the entity into two or more segments if the number of matches exceeds a pre-set limit,

wherein the histogram generator is configured to generate a histogram for each of the two or more segments.

55 . The image processing system of claim 51 , wherein the hash value comparator is further configured to sequentially compare the series of hashes with the pre-calculated table, such that all the hashes associated with a first desired image are compared before the hashes associated with a next desired image are compared.

56 . The image processing system of claim 55 , wherein the histogram generator is configured to add an temporal offset to the entity image matches of hashes associated with desired images that are not a most desired image, wherein the temporal offset is a same as the temporal offset between the matched desired image and the most desired image.

57 . The image processing system of claim 51 , further comprising a hash generator configured to generate the series of hashes representing the one or more desired images by:

identifying a number of feature points in a desired image that characterize the desired image;

identifying, for each feature point, three nearest neighboring feature points; and

generating three hashes for each of the identified feature points, wherein each created hash is comprised of features of two of three neighboring points.

58 . The image processing system of claim 57 , wherein the hash generator is configured to generate the three hashes for each feature point by:

creating a first hash comprising information associated with a first closest neighboring feature point and a second closest neighboring feature point, wherein the information associated with the first and second closest neighboring feature points represents a location of the first and second closes neighboring feature points relative to the selected feature point;

creating a second hash comprising information associated with the first closest neighboring feature point and a third closest neighboring feature point, wherein the information associated with the first and third closest neighboring feature points represents a location of the first and third closest neighboring feature points relative to the selected feature point;

creating a third hash comprising information associated with the second closest neighboring feature point and the third closest neighboring feature point, wherein the information associated with the second and third neighboring feature points represents the location of the second and third neighboring feature points relative to the selected feature point.

Assignments (4)
TERMINATION AND RELEASE OF PATENT SECURITY AGREEMENT Recorded Mar 21, 2024
From: MGG INVESTMENT GROUP LP
To: GRASS VALLEY USA, LLC; GRASS VALLEY CANADA; GRASS VALLEY LIMITED
Reel/Frame 066867/0336 →
GRANT OF SECURITY INTEREST - PATENTS Recorded Jul 2, 2020
From: GRASS VALLEY USA, LLC; GRASS VALLEY CANADA; GRASS VALLEY LIMITED
To: MGG INVESTMENT GROUP LP, AS COLLATERAL AGENT
Reel/Frame 053122/0666 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2020
From: DIGGINS, JONATHAN
To: GRASS VALLEY LIMITED
Reel/Frame 051675/0508 →
CHANGE OF NAME Recorded Jan 30, 2020
From: SNELL ADVANCED MEDIA LIMITED
To: GRASS VALLEY LIMITED
Reel/Frame 051759/0859 →