Granted Patent B1

US 12,277,154 · App. 18/827,005 · Granted Apr 15, 2025

Inventors: Vinod Khosla (Portola Valley, CA); Neeru Khosla (Portola Valley, CA); Miral Shah (San Jose, CA); Reza Shahbazi (Hillsborough, CA)

Assignee: CK12 Foundation

G06F16/3344G06F40/35

Abstract

Claims (63)

1. A non-transitory processor-readable medium storing code representing instructions to be executed by one or more processors, the instructions comprising code to cause the one or more processors to:

receive, a plurality of inputs associated with a behavior of a user;

classify, based on a first machine learning model and using at least one input rubric, each input from the plurality of inputs into an input type from a plurality of input types, the first machine learning model configured to extract relevant classification data from the plurality of inputs;

define, based on the input type of each input from the plurality of inputs, a first set of inputs associated with a first evaluation type and a second set of inputs associated with a second evaluation type, wherein the second evaluation type is associated with a lower computational cost than the first evaluation type;

select, based on the first evaluation type, a second machine learning model, the second machine learning model trained based on analytical data associated with the user;

extract, using the second machine learning model, from the first set of inputs a pattern associated with the user;

evaluate a first state of the user based on the pattern and a second state of the user based on the second set of inputs; and

generate a grade of the user based on the first state and the second state.

2. The non-transitory processor-readable medium of claim 1 , further comprising code to cause the one or more processors to:

send, to a user device, the grade.

3. The non-transitory processor-readable medium of claim 1 , further comprising code to cause the one or more processors to:

determine, based on the first state and the second state, missing performance data based on a third machine learning model, the third machine learning model using a rubric mapped to the first state and the second state;

generate user prompts associated with the missing performance data;

send the user prompts to the user;

receive, from the user, additional inputs associated with the missing performance data; and

evaluate an updated state of the user based on the plurality of inputs and the additional inputs.

4. The non-transitory processor-readable medium of claim 1 , wherein the first evaluation type is a machine learning evaluation and the second evaluation type is a heuristic evaluation.

5. The non-transitory processor-readable medium of claim 1 , wherein the second machine learning model includes a large language model (LLM).

6. The non-transitory processor-readable medium of claim 1 , wherein the plurality of inputs includes at least one of user questions, user answers, generated questions, or generated answers.

7. The non-transitory processor-readable medium of claim 1 , further comprising code to cause the one or more processors to:

provide the plurality of inputs as an input to a third machine learning model trained with representations of common misinformation;

identify, based on an output of the third machine learning model, user misinformation associated with the user; and

provide feedback associated with the user misinformation to the user.

8. A method, comprising:

receiving, a plurality of inputs associated with a behavior of a user;

classifying, based on a first machine learning model, each input from the plurality of inputs into an input type from a plurality of input types;

defining, based on the input type of each input from the plurality of inputs, a first set of inputs associated with a first evaluation type and a second set of inputs associated with a second evaluation type;

selecting, based on the first evaluation type, a second machine learning model;

extracting, using the second machine learning model, from the first set of inputs a pattern associated with the user;

evaluating a first state of the user based on the pattern and a second state of the user based on the second set of inputs;

determining, based on the first state and the second state, if the first state and the second state satisfy one or more rubric condition;

responsive to determining that the one or more rubric condition is not satisfied, generating, using a third machine learning model, one or more outputs;

receiving, at least one update input associated with the one or more outputs;

updating at least one of the first state or the second state based on the at least one update input; and

responsive to determining that the one or more rubric condition is satisfied, generating a grade of the user based on the first state and the second state.

9. The method of claim 8 , wherein the first evaluation type is a machine learning evaluation and the second evaluation type is a heuristic evaluation.

10. The method of claim 8 , wherein the second evaluation type is associated with a lower computational cost than the first evaluation type.

11. The method of claim 8 , wherein the third machine learning model includes at least one of a large language model or a reinforcement learning model.

12. The method of claim 8 , wherein the one or more rubric condition includes a total amount of data associated with a subject exceeding a predetermined threshold.

13. The method of claim 12 , wherein the one or more outputs is associated with the subject.

14. The method of claim 8 , further comprising:

receiving a visual signal associated with the user, the visual signal associated with a user engagement;

determining, based on a vision transformer model, a visual user behavior; and

updating, based on the visual user behavior, at least one of the first state or the second state.

15. The method of claim 14 , further comprising:

determining, using the third machine learning model and based on the visual user behavior, a user response behavior associated with the plurality of inputs; and

determining, based on the user response behavior, if the user response behavior satisfied the one or more rubric condition.

16. A method, comprising:

training, a first machine learning model, based on a set of representations of misinformation;

receiving a plurality of inputs associated with behavior of a user;

filtering the plurality of inputs to remove duplicate inputs;

clustering, using the first machine learning model, the plurality of inputs based on input type to define clustered inputs;

extracting, from the clustered inputs using the first machine learning model, one or more identified misinformation;

categorizing the one or more identified misinformation to one or more categories to define categorized misinformation;

training the first machine learning model based on the categorized misinformation to update the set of representations of misinformation with current information;

generating, using a second machine learning model and based on the one or more categories, at least one output; and

sending, to a user device, the at least one output.

17. The method of claim 16 , wherein the first machine learning model is a large language model.

18. The method of claim 16 , wherein the at least one output includes at least one of a report, information, sources, or an indication of identified misinformation.

19. The method of claim 16 , further comprising:

generating a feedback report based on the categorized misinformation and information associated with the categorized misinformation.

20. The method of claim 19 , further comprising:

customizing the feedback report based on the user.

Assignments (1)

Continuity (1)

References Cited (37)

Cited By (2)

US 12,524,708 US 12,711,552