Systems and methods of detecting chatbots
Some embodiments determine whether an entity engaging in online conversations comprises a chatbot. A challenge message is constructed according to a current context of an ongoing conversation, by employing a generative language model to determine a plausible conversation sequel and subsequently distorting the respective sequel. The challenge is added as a new message to the respective conversation. The chatbot detector may then determine whether a conversation partner is a chatbot according to a similarity between a response to the challenge received from the respective conversation partner and an artificially generated, plausible response to the undistorted conversation sequel. Distortions applied to construct the challenge are deliberately crafted to send a chatbot on a markedly different trajectory.
1 . A computer system comprising at least one hardware processor configured to:
apply a generative language model to generate a surrogate conversation sequel and a surrogate response, wherein:
the surrogate conversation sequel comprises a predicted continuation of an ongoing online conversation comprising a sequence of electronic messages, and
the surrogate response comprises a predicted response from a conversation partner to the surrogate conversation sequel;
in response, distort the surrogate conversation sequel to produce a challenge message;
add the challenge message to the ongoing online conversation; and
in response to receiving a partner response from the conversation partner, the partner response comprising a response to the challenge message, determine whether the conversation partner comprises a robot according to a similarity between the partner response and the surrogate response.
2 . The computer system of claim 1 , wherein the at least one hardware processor is configured to determine whether the conversation partner comprises the robot according to a result of comparing a similarity measure and a pre-determined threshold, wherein the similarity measure quantifies the similarity between the partner response and surrogate response.
3 . The computer system of claim 1 , wherein the at least one hardware processor is configured to determine whether the conversation partner comprises the robot further according to a similarity between the partner response and another surrogate response comprising a predicted response from the conversation partner to the challenge message.
4 . The computer system of claim 3 , wherein the at least one hardware processor is configured to determine whether the conversation partner comprises the robot according to a result of comparing a first similarity measure to a second similarity measure, wherein the first similarity measure quantifies the similarity between the partner response and the surrogate response and the second similarity measure quantifies the similarity between the partner response and the other surrogate response.
5 . The computer system of claim 1 , wherein distorting the surrogate conversation sequel comprises an item selected from a set consisting of replacing a selected token of the surrogate conversation sequel with a substitute token, adding a token to the surrogate conversation sequel, rephrasing the surrogate conversation sequel as a question, and rephrasing the surrogate conversation sequel to change a sentiment of the surrogate conversation sequel.
6 . The computer system of claim 1 , wherein the at least one hardware processor is further configured to:
in response to producing the challenge message, determine whether the challenge message satisfies a quality condition according to a similarity between the challenge message and the surrogate conversation sequel, and
in response, add the challenge message to the ongoing online conversation only if the challenge message satisfies the quality condition.
7 . The computer system of claim 6 , wherein the at least one hardware processor is configured to determine whether the challenge message satisfies the quality condition according to a result of comparing a similarity measure to a pre-determined threshold, wherein the similarity measure quantifies the similarity between the challenge message and the surrogate conversation sequel.
8 . The computer system of claim 6 , wherein the at least one hardware processor is configured to determine whether the challenge message satisfies the quality condition further according to a similarity between the surrogate response and another surrogate response comprising another predicted response from the conversation partner to the challenge message.
9 . The computer system of claim 1 , wherein the ongoing online conversation comprises an item selected from a group consisting of an exchange of messages carried out via an instant messaging application executing on the computer system, a sequence of messages posted to an online forum, and a sequence of messages posted to a social media page.
10 . The computer system of claim 1 , wherein applying the generative language model comprises transmitting an encoding of a fragment of the ongoing online conversation to a remote chatbot, and in response, receiving the surrogate conversation sequel or the surrogate response from the remote chatbot.
11 . A chatbot detection method comprising employing at least one hardware processor of a computer system to:
apply a generative language model to generate a surrogate conversation sequel and a surrogate response, wherein:
the surrogate conversation sequel comprises a predicted continuation of an ongoing online conversation comprising a sequence of messages, and
the surrogate response comprises a predicted response from a conversation partner to the surrogate conversation sequel;
in response, distort the surrogate conversation sequel to produce a challenge message;
add the challenge message to the ongoing online conversation; and
in response to receiving a partner response from the conversation partner, the partner response comprising a response to the challenge message, determine whether the conversation partner comprises a robot according to a similarity between the partner response and the surrogate response.
12 . The method of claim 11 , comprising determining whether the conversation partner comprises the robot according to a result of comparing a similarity measure and a pre-determined threshold, wherein the similarity measure quantifies the similarity between the partner response and surrogate response.
13 . The method of claim 11 , comprising determining whether the conversation partner comprises the robot further according to a similarity between the partner response and another surrogate response comprising a predicted response from the conversation partner to the challenge message.
14 . The method of claim 13 , comprising determining whether the conversation partner comprises the robot according to a result of comparing a first similarity measure to a second similarity measure, wherein the first similarity measure quantifies the similarity between the partner response and the surrogate response and the second similarity measure quantifies the similarity between the partner response and the other surrogate response.
15 . The method of claim 11 , wherein distorting the surrogate conversation sequel comprises an item selected from a set consisting of replacing a selected token of the surrogate conversation sequel with a substitute token, adding a token to the surrogate conversation sequel, rephrasing the surrogate conversation sequel as a question, and rephrasing the surrogate conversation sequel to change a sentiment of the surrogate conversation sequel.
16 . The method of claim 11 , further comprising employing the at least one hardware processor to:
in response to producing the challenge message, determine whether the challenge message satisfies a quality condition according to a similarity between the challenge message and the surrogate conversation sequel, and
in response, add the challenge message to the ongoing online conversation only if the challenge message satisfies the quality condition.
17 . The method of claim 16 , comprising determining whether the challenge message satisfies the quality condition according to a result of comparing a similarity measure to a pre-determined threshold, wherein the similarity measure quantifies the similarity between the challenge message and the surrogate conversation sequel.
18 . The method of claim 16 , comprising determining whether the challenge message satisfies the quality condition further according to a similarity between the surrogate response and another surrogate response comprising another predicted response from the conversation partner to the challenge message.
19 . The method of claim 11 , wherein the ongoing online conversation comprises an item selected from a group consisting of an exchange of messages carried out via an instant messaging application executing on the computer system, a sequence of messages posted to an online forum, and a sequence of messages posted to a social media page.
20 . The method of claim 11 , wherein applying the generative language model comprises employing the at least one hardware processor to transmit an encoding of a fragment of the ongoing online conversation to a remote chatbot, and in response, receive the surrogate conversation sequel or the surrogate response from the remote chatbot.
21 . A non-transitory computer-readable medium storing instructions which, when executed by at least one hardware processor of a computer system, cause the computer system to:
apply a generative language model to generate a surrogate conversation sequel and a surrogate response, wherein:
the surrogate conversation sequel comprises a predicted continuation of an ongoing online conversation comprising a sequence of messages, and
the surrogate response comprises a predicted response from a conversation partner to the surrogate conversation sequel;
in response, distort the surrogate conversation sequel to produce a challenge message;
add the challenge message to the ongoing online conversation; and
in response to receiving a partner response from the conversation partner, the partner response comprising a response to the challenge message, determine whether the conversation partner comprises a robot according to a similarity between the partner response and the surrogate response.