IP Library Granted Patent US 10,235,608
Granted Patent B2
US 10,235,608 · App. 15/368,132 · Granted Mar 19, 2019

Image quality assessment using adaptive non-overlapping mean estimation

Inventors: Setu Chokshi (Singapore, SG); Venkadachalam Ramalingam (Tamilnadu, IN)
Assignee: The Nielsen Company (US), LLC
G06K9/6284G06K9/036G06K9/4628G06T7/11G06T2207/20021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,235,608
App. No.
15/368,132
Granted
Mar 19, 2019
Kind
B2
Abstract

Methods, apparatus, systems and articles of manufacture (e.g., physical storage media) to assess image quality using adaptive non-overlapping mean estimation are disclosed. Example image quality assessment methods disclosed herein include replacing respective blocks of pixels of a first image with mean values of the respective blocks of pixels to determine a second image having a smaller size than the first image. Disclosed example image quality assessment methods also include determining a vector of features for the second image. Disclosed example image quality assessment methods further include applying the vector of features to a neural network, and classifying a quality of the first image based on an output of the neural network.

Claims (52)

1. An apparatus to perform image quality assessment, the apparatus comprising:

a training module to train a neural network, based on a set of training images, to classify quality of a first image;

an image determiner to replace respective blocks of pixels of the first image with mean values of the respective blocks of pixels to determine a second image having a smaller size than the first image;

a feature extractor to determine a vector of features including a blur feature value for the second image and to apply the vector of features to the neural network after the neural network has been trained, the feature extractor to determine the blur feature value based on an amount the second image differs from a version of the second image filtered with a blurring filter in horizontal and vertical directions of the second image; and

an image classifier to classify a quality of the first image based on an output of the neural network, the training module, the image determiner, the feature extractor, and the image classifier implemented by hardware circuitry or at least one processor.

2. The apparatus of claim 1 , wherein the respective blocks of pixels of the first image form adjacent blocks of pixels that are non-overlapping.

3. The apparatus of claim 1 , wherein to determine the second image, the image determiner is further to:

segment the first image into the respective blocks of pixels, the respective blocks of pixels having a first number of pixels in a horizontal direction and a second number of pixels in a vertical direction; and

for a first one of the respective blocks of pixels:

average values of the pixels in the first one of the respective blocks of pixels to determine a first mean value corresponding to the first one of the respective blocks of pixels; and

set a first pixel of the second image to be the first mean value, the first pixel at a first position in the second image that is to represent the first one of the respective blocks of pixels of the first image.

4. The apparatus of claim 3 , wherein the first number of pixels is two pixels, the second number of pixels is two pixels, the respective blocks of pixels of the first image each include four pixels, and the second image is one-fourth the size of the first image.

5. The apparatus of claim 1 , wherein to classify the quality of the first image, the image classifier is to perform a single classification of the quality of the first image by:

classifying the first image as acceptable for an image auditing application when the output of the neural network satisfies a threshold; and

classifying the first image as unacceptable for the image auditing application when the output of the neural network does not satisfy the threshold.

6. The apparatus of claim 1 , wherein to classify the quality of the first image, the image classifier is to compare a plurality of outputs of the neural network to a plurality of thresholds to perform multiple classifications of the quality of the first image.

7. An image quality assessment method comprising:

training, by executing an instruction with a processor, a neural network, based on a set of training images, to classify quality of a first image;

replacing, by executing an instruction with the processor, respective blocks of pixels of the first image with mean values of the respective blocks of pixels to determine a second image having a smaller size than the first image;

determining, by executing an instruction with the processor, a vector of features including a blur feature value for the second image, the blur feature value determined based on an amount the second image differs from a version of the second image filtered with a blurring filter in horizontal and vertical directions of the second image;

applying the vector of features to the neural network after the neural network has been trained; and

classifying, by executing an instruction with the processor, a quality of the first image based on an output of the neural network.

8. The method of claim 7 , wherein the respective blocks of pixels of the first image form adjacent blocks of pixels that are non-overlapping.

9. The method of claim 7 , wherein the replacing of the respective blocks of pixels of the first image with mean values of the respective blocks of pixels to determine the second image includes:

segmenting the first image into the respective blocks of pixels, the respective blocks of pixels having a first number of pixels in a horizontal direction and a second number of pixels in a vertical direction; and

for a first one of the respective blocks of pixels:

averaging values of the pixels in the first one of the respective blocks of pixels to determine a first mean value corresponding to the first one of the respective blocks of pixels; and

setting a first pixel of the second image to be the first mean value, the first pixel at a first position in the second image that is to represent the first one of the respective blocks of pixels of the first image.

10. The method of claim 9 , wherein the first number of pixels is two pixels, the second number of pixels is two pixels, the respective blocks of pixels of the first image each include four pixels, and the second image is one-fourth the size of the first image.

11. The method of claim 7 , further including converting the second image to a grayscale image before determining the vector of features for the second image.

12. The method of claim 7 , wherein the classifying of the quality of the first image includes performing a single classification of the quality of the first image by:

classifying the first image as acceptable for an image auditing application when the output of the neural network satisfies a threshold; and

classifying the first image as unacceptable for the image auditing application when the output of the neural network does not satisfy the threshold.

13. The method of claim 7 , wherein the classifying of the quality of the first image includes comparing a plurality of outputs of the neural network to a plurality of thresholds to perform multiple classifications of the quality of the first image.

14. A tangible computer readable storage medium comprising computer readable instructions that, when executed, cause a processor to at least:

train a neural network, based on a set of training images, to classify quality of a first image;

replace respective blocks of pixels of the first image with mean values of the respective blocks of pixels to determine a second image having a smaller size than the first image;

determine a vector of features including a blur feature value for the second image, the blur feature value determined based on an amount the second image differs from a version of the second image filtered with a blurring filter in horizontal and vertical directions of the second image;

apply the vector of features to the neural network after the neural network has been trained; and

classify a quality of the first image based on an output of the neural network.

15. The tangible computer readable storage medium of claim 14 , wherein the respective blocks of pixels of the first image form adjacent blocks of pixels that are non-overlapping.

16. The tangible computer readable storage medium of claim 14 , wherein to determine the second image, the instructions, when executed, further cause the processor to:

segment the first image into the respective blocks of pixels, the respective blocks of pixels having a first number of pixels in a horizontal direction and a second number of pixels in a vertical direction; and

for a first one of the respective blocks of pixels:

average values of the pixels in the first one of the respective blocks of pixels to determine a first mean value corresponding to the first one of the respective blocks of pixels; and

set a first pixel of the second image to be the first mean value, the first pixel at a first position in the second image that is to represent the first one of the respective blocks of pixels of the first image.

17. The tangible computer readable storage medium of claim 16 , wherein the first number of pixels is two pixels, the second number of pixels is two pixels, the respective blocks of pixels of the first image each include four pixels, and the second image is one-fourth the size of the first image.

18. The tangible computer readable storage medium of claim 14 , wherein the instructions, when executed, further cause the processor to convert the second image to a grayscale image before determining the vector of features for the second image.

19. The tangible computer readable storage medium of claim 14 , wherein to classify the quality of the first image, the instructions, when executed, cause the processor to perform a single classification of the quality of the first image by:

classifying the first image as acceptable for an image auditing application when the output of the neural network satisfies a threshold; and

classifying the first image as unacceptable for the image auditing application when the output of the neural network does not satisfy the threshold.

20. The tangible computer readable storage medium of claim 14 , wherein to classify the quality of the first image, the instructions, when executed, cause the processor to compare a plurality of outputs of the neural network to a plurality of thresholds to perform multiple classifications of the quality of the first image.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2017
From: CHOKSHI, SETU; RAMALINGAM, VENKADACHALAM
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 041980/0201 →
Priority Claims (1)
IN 4238/DEL/2015 · Dec 22, 2015 · national
Continuity (1)
Related Publication 20170177979A1 · Jun 22, 2017
Cited By (1)
US 12,602,748