IP Library Granted Patent US 12,347,178
Granted Patent B2
US 12,347,178 · App. 18/102,969 · Granted Jul 1, 2025

Systems, methods, and techniques for training neural networks and utilizing the neural networks to detect non-compliant content

Inventors: Shreyansh Prakash Gandhi (Milpitas, CA); Alessandro Magnani (Menlo Park, CA); Abhinandan Krishnan (Sunnyvale, CA); Abon Chaudhuri (Sunnyvale, CA); Samrat Kokkula (Santa Clara, CA); Venkatesh Kandaswamy (San Ramon, CA)
Assignee: WALMART APOLLO, LLC
G06V10/82G06F18/2148G06V30/19147G06V30/194
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,347,178
App. No.
18/102,969
Granted
Jul 1, 2025
Kind
B2
Abstract

A system including one or more processors and one or more non-transitory computer-readable storage media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform: training a neural network detection model with a training dataset comprising synthetic training images by: using a transformation algorithm to create the synthetic training images by appending edge case training images to one or more compliant images; receiving, at the neural network detection model, as trained, at least one image; and determining, using the neural network detection model, as trained, whether the at least one image comprises non-compliant content. Other embodiments are disclosed herein.

Claims (90)

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable storage media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:

training a neural network detection model with a training dataset comprising synthetic training images by:

using a transformation algorithm to create the synthetic training images by appending edge case training images to one or more compliant images; and

utilizing the transformation algorithm by at least applying one or more random transformations on the edge case training images;

receiving, at the neural network detection model, as trained, at least one image; and

determining, using the neural network detection model, as trained, whether the at least one image comprises non-compliant content.

2. The system of claim 1 , wherein:

training the neural network detection model further comprises:

applying the one or more random transformations on a first set of images comprising the non-compliant content;

appending the edge case training images to the one or more the non-compliant content to a second set of compliant images; and

utilizing the transformation algorithm further comprises appending the edge case training images to the one or more compliant images in the second set of compliant images.

3. The system of claim 2 , wherein applying the one or more random transformations comprises one or more of:

randomly scaling the non-compliant content before appending the non-compliant content to the second set of compliant images;

randomly rotating the non-compliant content before appending the non-compliant content to the second set of compliant images;

randomly distorting the non-compliant content before appending the non-compliant content to the second set of compliant images; or

randomly determining where the non-compliant content is to be placed on the second set of compliant images.

4. The system of claim 1 , wherein training the neural network detection model with the training dataset further comprises:

receiving a first set of images comprising the non-compliant content;

receiving a second set of compliant images; and

receiving the edge case training images to tune an accuracy of the neural network detection model.

5. The system of claim 1 , wherein training the neural network detection model with the training dataset further comprises:

automatically generating and storing annotation information associated with the synthetic training images; and

utilizing the annotation information to train the neural network detection model during a training procedure.

6. The system of claim 5 , wherein:

the training procedure comprises applying active learning to train the neural network detection model; and

the edge case training images comprise compliant images that resemble one or more categories of the non-compliant content.

7. The system of claim 1 , wherein training the neural network detection model with the training dataset further comprises converting a first set of images to transparent images.

8. The system of claim 1 , wherein the non-compliant content comprises one or more of:

logos or badges identifying one or more brands;

logos or badges identifying one or more seller statuses;

logos or badges comprising marketing information; or

logos or badges identifying one or more countries.

9. The system of claim 1 , wherein the operations further comprise:

executing one or more corrective measures on the at least one image in response to determining that the at least one image comprises the non-compliant content, wherein the one or more corrective measures comprise removing the non-compliant content from the at least one image.

10. The system of claim 9 , wherein the one or more corrective measures comprises at least one of:

removing the at least one image from a database;

preventing the at least one image from being published on a website;

flagging the at least one image for manual review;

deleting the at least one image;

removing the non-compliant content from the at least one image; or

annotating the at least one image to block the non-compliant content.

11. A method being implemented via execution of computing instructions configured to run at one or more processors and configured to be stored at non-transitory computer-readable media, the method comprising:

training a neural network detection model with a training dataset comprising synthetic training images by:

using a transformation algorithm to create the synthetic training images by appending edge case training images to one or more compliant images; and

utilizing the transformation algorithm by at least applying one or more random transformations on the edge case training images;

receiving, at the neural network detection model, as trained, at least one image; and

determining, using the neural network detection model, as trained, whether the at least one image comprises non-compliant content.

12. The method of claim 11 , wherein:

training the neural network detection model further comprises:

applying the one or more random transformations on a first set of images comprising the non-compliant content;

appending the edge case training images to the one or more the non-compliant content to a second set of compliant images; and

utilizing the transformation algorithm further comprises appending the edge case training images to the one or more compliant images in the second set of compliant images.

13. The method of claim 12 , wherein applying the one or more random transformations comprises one or more of:

randomly scaling the non-compliant content before appending the non-compliant content to the second set of compliant images;

randomly rotating the non-compliant content before appending the non-compliant content to the second set of compliant images;

randomly distorting the non-compliant content before appending the non-compliant content to the second set of compliant images; or

randomly determining where the non-compliant content is to be placed on the second set of compliant images.

14. The method of claim 11 , wherein training the neural network detection model with the training dataset further comprises:

receiving a first set of images comprising the non-compliant content;

receiving a second set of compliant images; and

receiving the edge case training images to tune an accuracy of the neural network detection model.

15. The method of claim 11 , wherein training the neural network detection model with the training dataset further comprises:

automatically generating and storing annotation information associated with the synthetic training images; and

utilizing the annotation information to train the neural network detection model during a training procedure.

16. The method of claim 15 , wherein:

the training procedure comprises applying active learning to train the neural network detection model; and

the edge case training images comprise compliant images that resemble one or more categories of the non-compliant content.

17. The method of claim 11 , wherein training the neural network detection model with the training dataset further comprises converting a first set of images to transparent images.

18. The method of claim 11 , wherein the non-compliant content comprises one or more of:

logos or badges identifying one or more brands;

logos or badges identifying one or more seller statuses;

logos or badges comprising marketing information; or

logos or badges identifying one or more countries.

19. The method of claim 11 , further comprising:

executing one or more corrective measures on the at least one image in response to determining that the at least one image comprises the non-compliant content, wherein the one or more corrective measures comprise at least one of:

removing the non-compliant content from the at least one image;

removing the at least one image from a database;

preventing the at least one image from being published on a website;

flagging the at least one image for manual review;

deleting the at least one image;

removing the non-compliant content from the at least one image; or

annotating the at least one image to block the non-compliant content.

20. A non-transitory computer readable storage medium storing one or more computing instructions that, when run on one or more processors, cause the one or more processors to perform:

training a neural network detection model with a training dataset comprising synthetic training images by:

using a transformation algorithm to create the synthetic training images by appending edge case training images to one or more compliant images; and

utilizing the transformation algorithm by at least applying one or more random transformations on the edge case training images;

receiving, at the neural network detection model, as trained, at least one image; and

determining, using the neural network detection model, as trained, whether the at least one image comprises non-compliant content.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2023
From: GANDHI, SHREYANSH PRAKASH; MAGNANI, ALESSANDRO; KRISHNAN, ABHINANDAN; CHAUDHURI, ABON; KOKKULA, SAMRAT; KANDASWAMY, VENKATESH
To: WALMART APOLLO, LLC
Reel/Frame 062624/0193 →
Continuity (3)
Continuation 17174857 · Feb 12, 2021
Continuation 16262621 · Jan 30, 2019
Related Publication 20230177823A1 · Jun 8, 2023
References Cited (90)
US 6269351B1 · Black · 2001 [cited by applicant]
US 6904168B1 · Steinberg et al. · 2005 [cited by applicant]
US 7403928B2 · Chen · 2008 [cited by applicant]
US 7831530B2 · Hashimoto · 2010 [cited by applicant]
US 8086039B2 · Kletter · 2011 [cited by applicant]
US 8374974B2 · Chen et al. · 2013 [cited by applicant]
US 8411964B2 · Choi · 2013 [cited by applicant]
US 8611617B1 · Baluja et al. · 2013 [cited by applicant]
US 8644606B2 · White et al. · 2014 [cited by applicant]
US 9361503B2 · Deppieri · 2016 [cited by examiner]
US 9418458B2 · Chertok et al. · 2016 [cited by applicant]
US 9922272B2 · Cheng et al. · 2018 [cited by applicant]
US 10225086B2 · Benchetrit et al. · 2019 [cited by applicant]
US 10235601B1 · Wrenninge · 2019 [cited by examiner]
US 10250538B2 · Prabhu · 2019 [cited by applicant]
US 10346969B1 · Raghu · 2019 [cited by examiner]
US 10426442B1 · Schnorr · 2019 [cited by applicant]
US 10592732B1 · Sather · 2020 [cited by examiner]
US 10664722B1 · Sharma · 2020 [cited by examiner]
US 10679046B1 · Black · 2020 [cited by examiner]
US 10769524B1 · Natesh · 2020 [cited by examiner]
US 11900653B1 · Schoner · 2024 [cited by examiner]
US 11972343B2 · Markram · 2024 [cited by examiner]
US 20030009495A1 · Adjaoute · 2003 [cited by applicant]
US 20030126267A1 · Gutta et al. · 2003 [cited by applicant]
US 20050160258A1 · O'Shea et al. · 2005 [cited by applicant]
US 20070133947A1 · Armitage et al. · 2007 [cited by applicant]
US 20080134282A1 · Fridman et al. · 2008 [cited by applicant]
US 20080159627A1 · Sengamedu · 2008 [cited by applicant]
US 20110135204A1 · Choi · 2011 [cited by applicant]
US 20110142346A1 · Han et al. · 2011 [cited by applicant]
US 20130151609A1 · Rubinstein et al. · 2013 [cited by applicant]
US 20160275343A1 · Dinerstein et al. · 2016 [cited by applicant]
US 20160294755A1 · Prabhu · 2016 [cited by applicant]
US 20170032224A1 · Zhang et al. · 2017 [cited by applicant]
US 20170061625A1 · Estrada · 2017 [cited by examiner]
US 20170177997A1 · Karlinsky · 2017 [cited by examiner]
US 20170287137A1 · Lin · 2017 [cited by examiner]
US 20170289624A1 · Avila et al. · 2017 [cited by applicant]
US 20180032840A1 · Yu et al. · 2018 [cited by applicant]
US 20180129978A1 · Vigoda et al. · 2018 [cited by applicant]
US 20180137390A1 · Brundage · 2018 [cited by examiner]
US 20180152402A1 · Tsou · 2018 [cited by applicant]
US 20180211117A1 · Ratti · 2018 [cited by examiner]
US 20180253866A1 · Jain et al. · 2018 [cited by applicant]
US 20180276864A1 · Hayashi · 2018 [cited by applicant]
US 20180285696A1 · Eigen · 2018 [cited by examiner]
US 20180307942A1 · Pereira · 2018 [cited by examiner]
US 20180349527A1 · Li · 2018 [cited by examiner]
US 20180349735A1 · Lin et al. · 2018 [cited by applicant]
US 20190080456A1 · Song · 2019 [cited by examiner]
US 20190108396A1 · Dal Mutto · 2019 [cited by examiner]
US 20190156202A1 · Falk · 2019 [cited by examiner]
US 20190163804A1 · Linton et al. · 2019 [cited by applicant]
US 20190179796A1 · Lakhman et al. · 2019 [cited by applicant]
US 20190197358A1 · Madani · 2019 [cited by examiner]
US 20190197466A1 · Hand, III · 2019 [cited by examiner]
US 20190208412A1 · Lord et al. · 2019 [cited by applicant]
US 20190245754A1 · Greene, Jr. · 2019 [cited by applicant]
US 20190257767A1 · Shaubi · 2019 [cited by examiner]
US 20190297042A1 · Prabhu · 2019 [cited by applicant]
US 20190325183A1 · Tscherepanow · 2019 [cited by examiner]
US 20190356956A1 · Sheng et al. · 2019 [cited by applicant]
US 20200005046A1 · Attorre et al. · 2020 [cited by applicant]
US 20200092607A1 · Rechner et al. · 2020 [cited by applicant]
US 20200097724A1 · Chakravarty · 2020 [cited by examiner]
US 20200099783A1 · Liu et al. · 2020 [cited by applicant]
US 20200125639A1 · Doyle · 2020 [cited by applicant]
US 20200125888A1 · Hacker · 2020 [cited by examiner]
US 20200125928A1 · Doyle · 2020 [cited by applicant]
US 20200126533A1 · Doyle et al. · 2020 [cited by applicant]
US 20200151521A1 · Almazán · 2020 [cited by examiner]
US 20200154078A1 · Ahn et al. · 2020 [cited by applicant]
US 20200160612A1 · Bowen · 2020 [cited by examiner]
US 20200162412A1 · Mei et al. · 2020 [cited by applicant]
US 20200242395A1 · Kumar · 2020 [cited by examiner]
US 20200242407A1 · Gandhi · 2020 [cited by examiner]
US 20200242733A1 · Kumar · 2020 [cited by examiner]
US 20210204553A1 · Mehta · 2021 [cited by examiner]
CN 101923652 · 2010 [cited by applicant]
CN 102842032B · 2015 [cited by applicant]
EP 1107130 · 2001 [cited by applicant]
EP 3555816B1 · 2023 [cited by examiner]
WO 2017020515 · 2017 [cited by applicant]
WO 2017214970 · 2017 [cited by applicant]
Indyk et al., “Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality,” STOC '98: Proceedings of the 30th Annual ACM Symposium on Theory of Computing, May 1998, https://dl.acm.org/doi/proceedings/10… [cited by applicant]
Simonyan et al., “Very Deep Convolutional Networks for Large-Scale Image Recognition,” Proceedings of ICLR, 2015, arXiv:1409.1556v6 [cs.CV], https://arxiv.org/pdf/1409.1556.pdf, 2015, 2015. [cited by applicant]
Mu et al., “Fast and Exact Nearest Neighbor Search in Hamming Space on Full-Text Search Engines,” Proceedings of SISAP, 2019, arXiv:1902.08498v2 [cs.IR], https://arxiv.org/abs/1902.08498, 2019, 2019. [cited by applicant]
Neurohive, “VGG16—Convolutional Network for Classification and Detection,” https://neurohive.io/en/popular-networks/vgg16/, accessed Apr. 27, 2020, 2020. [cited by applicant]
Van Durme et al., “Online Generation of Locality Sensitive Hash Signatures,” http://www.cs.jhu.edu/˜vandurme/papers/VanDurmeLallACL10-slides.pdf, accessed Apr. 27, 2020, 2020. [cited by applicant]