Learning content evaluation apparatus, system, and operation method for evaluating problem based on predicted probability of correct answer for problem content added without solving experience

1 . A learning content evaluation apparatus for evaluating characteristics of a problem on the basis of a probability of a correct answer of a user for content of the problem, the learning content evaluation apparatus comprising:

a processor; and

a memory storing instructions executable by the processor,

wherein the processor is configured to:

generate a problem embedding vector of pre-collected problem content by using bidirectional encoder representations from transformers (BERT) embedding in an unsupervised manner on the basis of only problem information included in the pre-collected problem content without using solution result data of the problem information,

train an AI model on the basis of (i) the problem embedding vector in which the solution result data of the problem information is not reflected and (ii) a user embedding vector in which solution result data of users for the pre-collected problem content is reflected,

generate AI learning information including a weight determined using a result of training the AI model,

generate a problem embedding vector of an added problem not solved by any of the users by using only problem information of the added problem without using solution result data of the added problem,

calculate correct answer probability information about a probability of being answered correctly by the user for the added problem not solved by any of the users by an artificial neural network model implemented by a program that is one of a recurrent neural network (RNN), a long/short-term memory (LSTM), a bidirectional LSTM, a transformer structure-artificial neural network, and BERT on the basis of the problem embedding vector of the added problem not solved by any of the users and the AI learning information, wherein the calculating of the correct answer probability information comprises calculating the correct answer probability information by inputting the problem embedding vector of the added problem not solved by any of the users into the artificial neural network model,

on the basis of the correct answer probability information of the added problem, generate problem evaluation information of the added problem not solved by any of the users, the problem evaluation information including one or more of a difficulty, a probability of being answered correctly by guessing and a discrimination power of the added problem, and

provide the generated problem evaluation information to a user terminal of the user.

2 . The learning content evaluation apparatus of claim 1 , wherein the processor is further configured to generate the problem embedding vector of the pre-collected problem content using a word embedding when text of a problem used for education or evaluation is given, and

the problem embedding vector of the pre-collected problem content is generated using the pre-collected problem content as an input such that an embedding value is adjusted to predict a correct answer of the pre-collected problem content.

3 . The learning content evaluation apparatus of claim 2 , wherein the processor is further configured to perform training to allow the AI model, which is trained with the problem embedding vector generated using the solution result data and the user embedding vector including a response of the user to additionally learn the added problem.

4 . The learning content evaluation apparatus of claim 3 , wherein the processor is further configured to predict a test score or grade to be given to the user after solving a problem in addition to the probability of a correct answer, and

the test score or grade reflects a result of solving one or more problems to be updated whenever the user solves the one or more problems.

5 . The learning content evaluation apparatus of claim 1 , wherein the processor is further configured to generate the problem evaluation information using an item response theory (IRT) representing a functional relationship between characteristics of the added problem and the probability of being answered correctly for the added problem.

6 . An operation method of a learning content evaluation apparatus for evaluating characteristics of a problem on the basis of a probability of a correct answer of a user for content of the problem, the operation method comprising:

generating a problem embedding vector of pre-collected problem content by using bidirectional encoder representations from transformers (BERT) embedding in an unsupervised manner on the basis of only problem information included in the pre-collected problem content without using solution result data of the problem information;

training an AI model on the basis of (i) the problem embedding vector in which the solution result data of the problem information is not reflected and (ii) a user embedding vector in which solution result data of users for the pre-collected problem content is reflected;

generating artificial intelligence (AI) learning information including a weight determined using a result of training the AI model;

generating a problem embedding vector of an added problem not solved by any of the users by using only problem information of the added problem without using solution result data of the added problem;

calculating correct answer probability information about a probability of being answered correctly by the user for the added problem not solved by any of the users by using an artificial neural network model implemented by a program that is one of a recurrent neural network (RNN), a long/short-term memory (LSTM), a bidirectional LSTM, a transformer structure-artificial neural network, and BERT on the basis of the problem embedding vector of the added problem and the AI learning information, wherein the calculating of the correct answer probability information comprises calculating the correct answer probability information by inputting the problem embedding vector of the added problem not solved by any of the users into the artificial neural network model;

on the basis of the correct answer probability information of the added problem, generating problem evaluation information of the added problem not solved by any of the users, the problem evaluation information including one or more of a difficulty, a probability of being answered correctly by guessing and a discrimination power of the added problem; and

providing the generated problem evaluation information to a user terminal of the user.