IP Library Granted Patent US 12688368
Granted Patent B2
US 12688368 · App. 18/341,207 · Granted Jul 21, 2026

Tagging for subject matter or learning schema

Inventors: Jesse Andrew Lewis Hamer (Iowa City, IA); Tiago Lima Almeida (Garopaba, BR); Charles Foster (Oakland, CA); Nicholas Koprowicz (Denver, CO); Lucas Cezimbra (Garopaba, BR); Claudio Souza (Garopaba, BR); Saad Masood Khan (Yardley, PA); Sara Vispoel (Iowa City, IA)
Assignee: Prometric LLC
G06F40/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12688368
App. No.
18/341,207
Filed
Jun 26, 2023
Granted
Jul 21, 2026
Kind
B2
Art Unit
2693
USPC
704/9
Abstract

An example method of classifying user content based on a set of standards is described herein. The method includes receiving the user content, generating numeric representations of the user content, and identifying one or more standards of the set of standards applying to the user content based on a comparison of the numeric representation of the user content and numeric representations of the one or more standards.

Claims (58)

1 . A method of classifying user content based on a hierarchical set of educational standards, the method comprising:

receiving the user content via a user interface operating on a computer system having one or more processors and memory, wherein the computer system is configured for:

generating numeric representations of the user content, wherein the numeric representations of the user content comprise multidimensional feature vectors representing the user content;

identifying one or more standards of the hierarchical set of educational standards for applying to the user content;

generating numeric representations of the one or more identified standards, wherein the numeric representations of the one or more identified standards comprise multidimensional feature vectors representing the one or more identified standards; and

classifying the user content based on a comparison of the numeric representations of the user content and the numeric representations of the one or more identified standards, wherein the comparison comprises a similarity function reflecting similarity or dissimilarity and semantic dependence or independence of the user content and the one or more identified standards; and

outputting a metric based on the similarity function via the user interface, wherein the metric quantifies a match between the user content and one or more of the identified standards based on a distance determined between the multidimensional feature vectors representing the user content and the one or more identified standards;

wherein the computer system is further configured for executing:

an insights and visualization sub-module configured for creating and communicating display results for display via the user interface, wherein the display results include a histogram view and/or density plot, automatic taxonomic gap detection, and/or automatic data-analytic insights based on such display results; and

a user feedback module configured to receive user validation and/or feedback via the user interface, and to determine based on said validation and/or feedback to validate the match as appropriate or inappropriate.

2 . The method of claim 1 , wherein generating the numeric representations of the user content comprises using a transformer model to generate the multidimensional feature vectors representing the user content and the one or more identified standards.

3 . The method of claim 1 , further comprising pre-processing the user content to create query tokens and converting the query tokens into the multidimensional feature vector representing the user content.

4 . The method of claim 1 , further comprising outputting the one or more identified standards via the user interface, wherein the standards are further identified based on the distance determined between the multidimensional feature vectors representing the user content and the respective standard, wherein one of the standards is identified as a primary standard that is closest to the user content based on the respective distance.

5 . The method of claim 3 , further comprising:

processing the multidimensional feature vector representing the user content to create a conditioned probability token for each query token, wherein the conditioned probability tokens are conditioned on the user content;

comparing the conditioned probability tokens to unconditioned probability tokens, wherein the unconditioned probability tokens are not conditioned on the user content; and

aggregating a result of said comparing into the metric.

6 . The method of claim 5 , wherein the user content comprises an input string including text, wherein the unconditioned probability tokens are generated by processing the input string, and wherein the conditioned probability tokens are conditioned on the text.

7 . A method of classifying user content based on a hierarchical taxonomic content map enumerating units of taxonomic content for a subject matter area, the method comprising:

receiving the user content via a user interface operating on a computer system having one or more processors and memory, wherein the computer system is configured for:

identifying one or more units of the user content;

generating numeric representations comprising multidimensional feature vectors representing the one or more units of the user content;

identifying one or more of the units of the taxonomic content in the taxonomic content map;

generating numeric representations comprising multidimensional feature vectors representing the one or more units of taxonomic content identified in the taxonomic content map; and

identifying for the one or more units of the user content a similarity or semantic match relationship with the one or more units of taxonomic content identified in the taxonomic content map based on a comparison of the numeric representations of the one or more units of the user content and the numeric representations of the one or more units of taxonomic content identified in the taxonomic content map, wherein the comparison comprises a similarity function reflecting similarity or dissimilarity and semantic dependence or independence of the user content and the one or more identified units of taxonomic content;

displaying via the user interface at least a portion of the taxonomic content map enumerating one or more of the identified units of taxonomic content; and

for each of said one or more identified units of taxonomic content, displaying via the user interface a metric based on the similarity function, wherein the metric quantifies the similarity or semantic match relationship based on a distance determined between the multidimensional feature vectors representing the one or more units of the user content and the one or more identified units of taxonomic content;

wherein the computer system is further configured for executing:

an insights and visualization sub-module configured for creating and communicating display results for display via the user interface, wherein the display results include a histogram view and/or density plot, automatic taxonomic gap detection, and/or automatic data-analytic insights based such display results; and

a user feedback module configured to receive user validation and/or feedback via the user interface, and to determine based on said validation and/or feedback to validate the similarity or semantic match relationship as appropriate or inappropriate.

8 . The method of claim 7 , wherein the comparison of the numeric representations of the one or more units of the user content and the numeric representations of the one or more units of taxonomic content identified in the taxonomic content map comprises a set of similarity functions, and further comprising aggregating the similarity functions to generate the metric.

9 . The method of claim 7 , wherein generating numeric representations of the one or more units of the user content comprises using a transformer model to generate the multidimensional feature vectors representing the one or more units of the user content.

10 . The method of claim 7 , wherein the one or more identified units of the taxonomic content are comprised within a hierarchical set of educational standards for the subject matter area and the one or more units of the user content comprise at least one unit of learning content or assessment content for the hierarchical set of educational standards.

11 . The method of claim 7 , wherein identifying the similarity or semantic match relationship comprises computing the metric in an n-dimensional space based on the distance between the multidimensional feature vectors representing the one or more units of the user content and the one or more identified units of the taxonomic content.

12 . The method of claim 7 , wherein the metric comprises a confidence measure representing an extent to which the user content is semantically associated or disassociated with the taxonomic content.

13 . The method of claim 7 , further comprising pre-processing the units of user content to create query tokens and converting the query tokens into the multidimensional feature vectors representing the units of the user content.

14 . The method of claim 13 , further comprising:

processing the multidimensional feature vectors representing the units of the user content to create conditioned probability tokens for each query token, wherein the conditioned probability tokens are conditioned on the user content;

comparing the conditioned probability tokens to unconditioned probability tokens, wherein the unconditioned probability tokens are not conditioned on the user content; and

aggregating a result of said comparing into the metric.

15 . The method of claim 14 , wherein the user content comprises an input string including text, wherein the unconditioned probability tokens are generated by processing the input string, and wherein the conditioned probability tokens are conditioned on the text.

16 . A method for classifying user content based on a hierarchical set of educational standards represented by a hierarchical map enumerating units of taxonomic content, the method comprising:

receiving a piece of the user content via a user interface operating on a computer system having one or more processors and memory, wherein the computer system is configured for:

identifying a standard of the set of educational standards for applying to the piece of the user content;

generating a numeric representation comprising a multidimensional feature vector representing the piece of user content;

generating a numeric representations comprising multidimensional feature vectors representing the units of taxonomic content enumerated in the hierarchical map representing the identified standard;

comparing the numeric representation of the piece of user content and the numeric representation of the identified standard via a similarity function reflecting similarity or dissimilarity and semantic dependence or independence of the piece of user content and the identified standard;

providing the piece of the user content and the identified standard to a generative model;

receiving output from the generative model; and

determining if the identified standard applies to the piece of the user content based on the similarity function and the output received from the generative model, wherein the output comprises a confidence measure representing an extent to which the piece of user content is semantically associated or disassociated with one or more of the respective units of taxonomic content enumerated in the hierarchical map representing the identified standard; and further comprising:

displaying, via the user interface, at least a portion of the hierarchical map enumerating the one or more respective units of taxonomic content; and

for each of said one or more respective units of taxonomic content, displaying, via the user interface, the confidence measure and a metric representing the similarity function, wherein the metric quantifies a match between the piece of content and the identified standard based on a distance determined between the multidimensional feature vectors representing the piece of user content and the one or more respective units of taxonomic content; wherein the computer system is further configured for executing:

an insights and visualization sub-module configured for creating and communicating display results for display via the user interface, wherein the display results include a histogram view and/or density plot, automatic taxonomic gap detection, and/or automatic data-analytic insights based on such display results; and

a user feedback module configured to receive user validation and/or feedback via the user interface, and to determine based on said validation and/or feedback to validate the similarity or semantic match relationship as appropriate or inappropriate.

17 . The method of claim 16 , further comprising preprocessing the piece of user content to create query tokens prior to providing the piece of the user content to the generative model.

18 . The method of claim 16 , further comprising outputting the identified standard via the user interface, wherein the standard is further identified as a primary standard that is closest to the user content based on the distance.

19 . The method of claim 17 , wherein providing the piece of the user content to the generative model comprises prompting the generative model to process the query tokens to generate the multidimensional feature vector representing the piece of user content, wherein the output indicates whether the identified standard applies to the piece of the user content based on said multidimensional feature vector.

20 . The method of claim 19 , wherein determining if the standard applies to the piece of the user content is further based on a similarity measure between the standard and the piece of the user content, wherein the similarity measure quantifies the distance between the multidimensional feature vectors representing the piece of user content and the identified standard, and wherein the distance is generated by a second generative model.