Heterogenous content collection package compliance testing processes
Methods are disclosed for performing compliance testing using machine learning models, such as language models or computer vision models. A computer-implemented method may include, for example, accessing a first portion of a content collection package, accessing a classification ruleset specifying a set of criteria that describes identifying features associated with specific content items in the content collection package and types of data extractable from the specific content items, executing a content item classification and data extraction process on the first portion implemented using a first set of machine learning models by classifying the specific content items in the content collection package, determining a first content item comprises first target information and extracting the first target information, generating a prompt, processing the prompt with a compliance checker, receiving a compliance determination dataset from the compliance checker, and generating an output for display based on the compliance determination dataset.
1 . A computer-implemented method of testing compliance of a content collection package having a plurality of heterogenous content items, the computer-implemented method comprising:
by a computing system comprising one or more hardware processors,
receiving a request to perform content compliance testing of the content collection package;
accessing a first portion of the content collection package;
accessing a classification ruleset specifying a set of at least partially different criteria that describe identifying features associated with specific content items in the content collection package and types of data extractable from the specific content items, and wherein the classification ruleset comprises configuration parameters for configuring a first set of machine learning models;
executing, on the first portion of the content collection package, a content item classification and data extraction process implemented using the first set of machine learning models that are configured based on the configuration parameters, wherein the configuration parameters specify a set of instructions that instruct the first set of machine learning models on operations to perform with respect to the classification ruleset and the content collection package, and wherein the content item classification and data extraction process comprises:
classifying the specific content items in the content collection package;
determining that a first content item of the specific content items comprises first target information, wherein the first target information is usable to assess compliance of the content collection package with a compliance ruleset, and wherein the compliance ruleset specifies rules for determining whether the content collection package satisfies a set of compliance criteria; and
in response to determining that the first content item comprises the first target information, extracting the first target information into a compliance audit package, wherein the compliance audit package comprises a subset of the content collection package having the first target information and at least second target information that, when processed using the compliance ruleset, is indicative of whether the content collection package satisfies the compliance ruleset, and wherein the compliance audit package omits a subset of the content collection package that, when processed using the compliance ruleset, is not indicative of whether the content collection package satisfies the compliance ruleset;
processing the compliance audit package using the compliance ruleset to obtain a compliance indication result, wherein processing the compliance audit package comprises:
generating a prompt based at least in part on the compliance audit package and the compliance ruleset; and
processing the prompt using a second set of machine learning models that are configured based on second configuration parameters; and
generating an output for display on a user interface based on the compliance indication result.
2 . The computer-implemented method of claim 1 , wherein processing the compliance audit package comprises assessing consistency between the first target information and the at least second target information.
3 . The computer-implemented method of claim 1 , wherein the second configuration parameters specify a second set of instructions that instruct the second set of machine learning models on operations to perform with respect to the compliance ruleset and the compliance audit package.
4 . The computer-implemented method of claim 3 , wherein the second set of instructions is configured to maintain accuracy of the second set of machine learning models at or above an accuracy threshold.
5 . The computer-implemented method of claim 1 , further comprising:
accessing a second portion of the content collection package; and
executing the content item classification and data extraction process on the second portion of the content collection package.
6 . The computer-implemented method of claim 5 , wherein the second portion of the content collection package corresponds to a different content item than the first portion of the content collection package.
7 . The computer-implemented method of claim 5 , wherein the first portion of the content collection package and the second portion of the content collection package correspond to the same content item of the content collection package.
8 . The computer-implemented method of claim 5 , wherein the at least second target information is extracted from the second portion of the content collection package.
9 . The computer-implemented method of claim 1 , wherein, in response to determining that the compliance indication result indicates that the compliance audit package does not satisfy one or more criteria within the compliance ruleset, the computer-implemented method further comprises determining a resolution action to resolve noncompliance of the content collection package.
10 . The computer-implemented method of claim 9 , wherein the output comprises an identity of the resolution action.
11 . The computer-implemented method of claim 1 , wherein the first set of machine learning models comprises a computer vision machine learning model.
12 . The computer-implemented method of claim 1 , further comprising chunking the content collection package into a plurality of portions that satisfy a size threshold, wherein the plurality of portions includes the first portion of the content collection package.
13 . A compliance testing system configured to test compliance of a content item, the compliance testing system comprising:
a memory configured to store computer-executable instructions; and
one or more hardware processors configured to execute the computer-executable instructions to at least:
receive a request to perform content compliance testing of a content item;
access the content item;
identify a type of the content item using a first machine learning model;
extract target information from the content item, wherein the target information is useable to assess compliance of the content item with a compliance ruleset, wherein the target information is identified based at least in part on the type of the content item;
access the compliance ruleset that specifies a set of criteria that evaluate compliance of content items with a set of constraints, and wherein the compliance ruleset comprises configuration parameters for configuring a second machine learning model;
execute a compliance checker implemented using the second machine learning model, wherein the second machine learning model is configured based on the configuration parameters, wherein the configuration parameters specify a set of instructions that instruct the second machine learning model on operations to perform with respect to the compliance ruleset and the content item, and wherein the set of instructions are configured to maintain accuracy of the compliance checker at or above an accuracy threshold;
generate a prompt comprising the target information extracted from the content item and the compliance ruleset;
process the prompt using the compliance checker, wherein the compliance checker uses the second machine learning model to verify compliance of the content item based at least in part on the compliance ruleset;
receive a compliance determination dataset from the compliance checker that indicates whether the content item satisfies one or more criteria within the compliance ruleset; and
generate an output for display on a user interface based at least in part on the compliance determination dataset.
14 . The compliance testing system of claim 13 , wherein determining whether the content items satisfies one or more criteria within the compliance ruleset comprises determining whether the target information satisfies the one or more criteria.
15 . The compliance testing system of claim 13 , wherein the prompt further comprises at least a portion of the content item.
16 . A compliance testing system configured to test compliance of a content item, the compliance testing system comprising:
a memory configured to store computer-executable instructions; and
one or more hardware processors configured to execute the computer-executable instructions to at least:
receive a request to perform content compliance testing of a content item;
access the content item;
identify a type of the content item using a first machine learning model;
access a compliance ruleset that specifies a set of criteria that evaluate compliance of content items with a set of constraints, and wherein the compliance ruleset comprises configuration parameters for configuring a second machine learning model, and wherein the compliance ruleset is selected based at least in part on the type of the content item;
execute a compliance checker implemented using the second machine learning model, wherein the second machine learning model is configured based on the configuration parameters, wherein the configuration parameters specify a set of instructions that instruct the second machine learning model on operations to perform with respect to the compliance ruleset and the content item, and wherein the set of instructions are configured to maintain accuracy of the compliance checker at or above an accuracy threshold;
generate a prompt comprising the compliance ruleset;
process the prompt using the compliance checker, wherein the compliance checker uses the second machine learning model to verify compliance of the content item based at least in part on the compliance ruleset;
receive a compliance determination dataset from the compliance checker that indicates whether the content item satisfies one or more criteria within the compliance ruleset; and
generate an output for display on a user interface based at least in part on the compliance determination dataset.
17 . The compliance testing system of claim 16 , wherein the prompt comprises at least a portion of the content item.
18 . The compliance testing system of claim 16 , wherein the one or more hardware processors are further configured to execute the computer-executable instructions to at least divide the content item into at least two overlapping sections, wherein generating the prompt comprises generating a prompt for each overlapping section of the content item.
19 . The compliance testing system of claim 18 , wherein receiving a compliance determination dataset comprises receiving an indication of whether an output associated with each prompt for each of the overlapping sections of the content item are consistent.