IP Library › Granted Patent US 12,189,722
Granted Patent B2
US 12,189,722 · App. 17/505,105 · Granted Jan 7, 2025

Content based on-device image adjustment

Inventors: Geoffrey Pascal (Mirabel aux baronnies, FR); Christophe Menichetti (Montpellier, FR); Nicolas Tallet (Montpellier, FR); Pascal Vezolle (Villeneuve les Maguelone, FR)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06F18/2411G06F18/214G06F18/2431
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,189,722
App. No.
17/505,105
Granted
Jan 7, 2025
Kind
B2
Abstract

Using a trained image classification model, a first image captured using an image capture device is classified, the trained image classification model executing in the same system as the image capture device, the classifying producing an inappropriate content classification of the first image. Using a trained feature extraction model, a pattern within a portion of the first image is recognized from the first image, the trained feature extraction model executing in the same system as the image capture device, the pattern predefined as inappropriate content. Based on the classification and the pattern, the first image is adjusted.

Claims (36)

1. A computer-implemented method comprising:

classifying, using a trained image classification model, a first image captured using an image capture device, the trained image classification model executing in a same system as the image capture device, the classifying producing an inappropriate content classification of the first image;

recognizing, from the first image using a trained feature extraction model, a pattern within a portion of the first image, the trained feature extraction model executing in the same system as the image capture device, the pattern predefined as inappropriate content;

adjusting, based on the pattern and a classification resulting from the classifying, the portion of the first image, wherein the adjusting leaves a remainder of the first image outside the portion unchanged; and

communicating, based on the adjusting the portion of the first image, the first image from the first user device to an application executing on a second user device used by an administrator to confirm that the first image matches a desire of the administrator.

2. The computer-implemented method of claim 1 , wherein the classifying and the recognizing are performed concurrently.

3. The computer-implemented method of claim 1 , further comprising:

classifying, using the trained image classification model, a second image captured using the image capture device, the classifying producing an inappropriate content classification of the second image; and

adjusting, based on the classification of the second image, the second image.

4. The computer-implemented method of claim 1 , wherein the classifying further produces a classification confidence score corresponding to the inappropriate content classification of the first image.

5. The computer-implemented method of claim 4 , wherein the adjusting is performed responsive to determining that the classification confidence score is above a first confidence threshold.

6. The computer-implemented method of claim 1 , wherein the recognizing further produces a recognition confidence score corresponding to recognition of the pattern.

7. The computer-implemented method of claim 6 , wherein the adjusting is performed responsive to determining that the recognition confidence score is above a second confidence threshold.

8. The computer-implemented method of claim 1 , wherein the adjusting is performed responsive to determining that a combined confidence score is above a third confidence threshold, the combined confidence score comprising a product of a classification confidence score corresponding to the inappropriate content classification of the first image and a recognition confidence score corresponding to recognition of the pattern.

9. A computer program product for content based on-device image adjustment, the computer program product comprising:

one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the stored program instructions comprising:

program instructions to classify, using a trained image classification model, a first image captured using a first image capture device, the trained image classification model executing in a same system as the first image capture device, the program instruction to classify producing an inappropriate content classification of the first image;

program instructions to recognize, from the first image using a trained feature extraction model, a pattern within a portion of the first image, the trained feature extraction model executing in the same system as the first image capture device, the pattern predefined as inappropriate content;

program instructions to adjust, based on the pattern and a classification resulting from the program instructions to classify, the portion of the first image, wherein the adjusting leaves a remainder of the first image outside the portion unchanged; and

program instructions to communicate, based on the adjusting the portion of the first image, the first image from the first image capture device to an application executing on a second image capture device used by an administrator to confirm that the first image matches a desire of the administrator.

10. The computer program product of claim 9 , wherein the program instructions to classify and the program instructions to recognize are performed concurrently.

11. The computer program product of claim 9 , the stored program instructions further comprising:

program instructions to classify, using the trained image classification model, a second image captured using the image capture device, the classifying producing an inappropriate content classification of the second image; and

program instructions to adjust, based on the classification of the second image, the second image.

12. The computer program product of claim 9 , wherein the classifying further produces a classification confidence score corresponding to the inappropriate content classification of the first image.

13. The computer program product of claim 12 , wherein the adjusting is performed responsive to determining that the classification confidence score is above a first confidence threshold.

14. The computer program product of claim 9 , wherein the recognizing further produces a recognition confidence score corresponding to recognition of the pattern.

15. The computer program product of claim 14 , wherein the adjusting is performed responsive to determining that the recognition confidence score is above a second confidence threshold.

16. The computer program product of claim 9 , wherein the adjusting is performed responsive to determining that a combined confidence score is above a third confidence threshold, the combined confidence score comprising a product of a classification confidence score corresponding to the inappropriate content classification of the first image and a recognition confidence score corresponding to recognition of the pattern.

17. The computer program product of claim 9 , wherein the stored program instructions are stored in the at least one of the one or more storage media of a local data processing system, and wherein the stored program instructions are transferred over a network from a remote data processing system.

18. The computer program product of claim 9 , wherein the stored program instructions are stored in the at least one of the one or more storage media of a server data processing system, and wherein the stored program instructions are downloaded over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system.

19. A computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage media, and program instructions stored on at least one of the one or more storage media for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:

program instructions to classify, using a trained image classification model, a first image captured using a first image capture device, the trained image classification model executing in a same system as the first image capture device, the program instructions to classify producing an inappropriate content classification of the first image;

program instructions to recognize, from the first image using a trained feature extraction model, a pattern within a portion of the first image, the trained feature extraction model executing in the same system as the first image capture device, the pattern predefined as inappropriate content;

program instructions to adjust, based on the pattern and a classification resulting from the program instructions to classify, the portion of the first image, wherein the adjusting leaves a remainder of the first image outside the portion unchanged; and

program instructions to communicate, based on the adjusting the portion of the first image, the first image from the first image capture device to an application executing on a second image capture device used by an administrator to confirm that the first image matches a desire of the administrator.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 19, 2021
From: PASCAL, GEOFFREY; MENICHETTI, CHRISTOPHE; TALLET, NICOLAS; VEZOLLE, PASCAL
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 057836/0125 →
Continuity (1)
Related Publication 20230118072A1 · Apr 20, 2023
References Cited (33)
US 8385971B2 · Rhoads et al. · 2013 [cited by applicant]
US 10827023B1 · Yan et al. · 2020 [cited by applicant]
US 11314970B1 · Swaminathan · 2022 [cited by examiner]
US 11688159B2 · Peng · 2023 [cited by examiner]
US 11854264B2 · Marzorati · 2023 [cited by examiner]
US 20080012935A1 · Echtenkamp · 2008 [cited by applicant]
US 20090074271A1 · Nakamura et al. · 2009 [cited by applicant]
US 20130283388A1 · Ashok · 2013 [cited by examiner]
US 20150070516A1 · Shoemake et al. · 2015 [cited by applicant]
US 20160063386A1 · Xie · 2016 [cited by examiner]
US 20160104040A1 · Ragnet et al. · 2016 [cited by applicant]
US 20160328923A1 · Alewine · 2016 [cited by examiner]
US 20170289624A1 · Avila · 2017 [cited by examiner]
US 20180032845A1 · Polak · 2018 [cited by examiner]
US 20180089505A1 · El-Khamy et al. · 2018 [cited by applicant]
US 20180293908A1 · Wang · 2018 [cited by examiner]
US 20190156193A1 · Jaroch · 2019 [cited by examiner]
US 20200412744A1 · Spurlock · 2020 [cited by examiner]
US 20210105518A1 · Kannan · 2021 [cited by examiner]
US 20210329338A1 · Khov · 2021 [cited by examiner]
US 20210383394A1 · Zarakas · 2021 [cited by examiner]
US 20220094713A1 · Lee · 2022 [cited by examiner]
US 20220167051A1 · Hall · 2022 [cited by examiner]
US 20220405510A1 · Marzorati · 2022 [cited by examiner]
CN 101389261A · 2009 [cited by applicant]
CN 107871117A · 2018 [cited by applicant]
CN 113393471A · 2021 [cited by applicant]
International Searching Authority, PCT/CN2022/124364, Dec. 30, 2022. [cited by applicant]
International Search Report and Written Opinion dated Dec. 30, 2022, for International Application No. PCT/CN2022/124304, filed Oct. 10, 2022. [cited by applicant]
Velasco, Facebook Wants Artificial Intelligence to Filter Live Videos and Take Down Offensive Content, May 26, 2018, https://www.techtimes.com/articles/228653/20180526/facebook-wants-artificial-intelligence-to-filter-li… [cited by applicant]
ip.com, Real Time Visuals Restrictions in Smart Devices, Oct. 27, 2020. [cited by applicant]
Yuan et al., Violation Detection of Live Video Based on Deep Learning, Hindawi, Scientific Programming, vol. 2020, May 11, 2020. [cited by applicant]
Liu et al., New classification models for detecting Hate and Violence web content, Aug. 1, 2016, 2015 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K), Nov… [cited by applicant]