Miscategorized outlier detection using unsupervised SLM-GBM approach and structured data
In an example, one or more leaf category specific unsupervised statistical language model (SLM) models are trained using sample item listings corresponding to each of one or more leaf categories and structured data about the one or more leaf categories, the training including calculating an expected perplexity and a standard deviation for item listing titles. A perplexity for a title of a particular item listing is calculated and a perplexity deviation signal is generated based on a difference between the perplexity for the title of the particular item listing and the expected perplexity for item listing titles in a leaf category of the particular item listing and based on the standard deviation for item listing titles in the leaf category of the particular item listing. A gradient boosting machine (GBM) fuses the perplexity deviation signal with one or more other signals to generate a miscategorization classification score corresponding to the particular item listing.
1. A system comprising:
one or more processors;
a statistical language model (SLM) training component executable by the one or more processors and configured to train one or more leaf-category-specific unsupervised statistical language model (SLM) models using sample item listings corresponding to each of one or more leaf categories and structured data about the one or more leaf categories, the training including calculating an expected perplexity and a standard deviation for item listing titles;
a perplexity deviation signal generator configured to, in response to a request for a miscategorization classification score corresponding to a particular item listing:
calculate a perplexity for a title of the particular item listing, and
generate a perplexity deviation signal based on a difference between the perplexity for the title of the particular item listing and the expected perplexity for item listing titles in a leaf category of the particular item listing and based on the standard deviation for item listing titles in the leaf category of the particular item listing;
a gradient boosting machine (GBM) configured to fuse the perplexity deviation signal with one or more other signals to generate a miscategorization classification score corresponding to the particular item listing; and
a graphical user interface including a first section where a user can enter textual data pertaining to the particular item listing, including a title of the particular item listing, and a second section where top n categories are presented, the top n categories being selected from the one or more leaf categories and representing n leaf categories having lowest miscategorization score for a first item listing, the second section further displaying the miscategorization classification score next to each of the top n categories and a subsection where the user can select one of the top n categories to assign to the particular item listing.
2. The system of claim 1 , wherein the training further includes generating an SLM for each leaf category for structured data, an SLM for each leaf category's queries, and an SLM for each leaf category's titles, and interpolating the SLM for each leaf category for structured data, the SLM for each leaf category's queries, and the SLM for each leaf category's titles into an SLM for each leaf category.
3. The system of claim 2 , wherein the training further includes generating an expected perplexity and a standard deviation for each leaf category based on the SLM for each leaf category and perplexity and standard deviation calculations for each sample item listing.
4. The system of claim 1 , wherein the generating the perplexity deviation signal includes computing a sentence log probability.
5. The system of claim 1 , further comprising:
a GBM training component configured to:
create a tuning set of item listings by labeling item listings as miscategorized or non-miscategorized based on application of filters to the item listings; and
feed the tuning set of item listings to the GBM for tuning of a GBM model used by the GBM.
6. The system of claim 1 , wherein the GBM takes a product type signal as input.
7. A method comprising:
training one or more leaf-category-specific unsupervised statistical language model (SLM) models using sample item listings corresponding to each of one or more leaf categories and structured data about the one or more leaf categories, the training including calculating an expected perplexity and a standard deviation for item listing titles;
in response to a request for a miscategorization classification score corresponding to a particular item listing, calculating a perplexity for a title of the particular item listing and generating a perplexity deviation signal based on a difference between the perplexity for the title of the particular item listing and the expected perplexity for item listing titles in a leaf category of the particular item listing and based on the standard deviation for item listing titles in the leaf category of the particular item listing;
using a gradient boosting machine (GBM) to fuse the perplexity deviation signal with one or more other signals to generate a miscategorization classification score corresponding to the particular item listing; and
causing a graphical user interface to be rendered, the graphical user interface including a first section where a user can enter textual data pertaining to the particular item listing, including a title of the particular item listing, and a second section where top n categories are presented, the top n categories being selected from the one or more leaf categories and representing n leaf categories having lowest miscategorization score for a first item listing, the second section further displaying the miscategorization classification score next to each of the top n categories and a subsection where the user can select one of the top n categories to assign to the particular item listing.
8. The method of claim 7 , wherein the training comprises calculating a sentence perplexity PP(S) for each sequence S of N words {w 1 , w 2 , . . . , w N } in each title of each of the sample item listings according to the following formula:
PP
(
S
)
=
P
(
w
1
…
w
N
)
-
1
/
N
=
∏
i
=
1
N
1
P
(
w
1
|
w
1
…
w
i
-
1
)
N
.
9. The method of claim 7 , wherein the training further includes generating an SLM for each leaf category for structured data, an SLM for each leaf category's queries, and an SLM for each leaf category's titles, and interpolating the SLM for each leaf category for structured data; the SLM for each leaf category's queries, and the SLM for each leaf category's titles into an SIM for each leaf category.
10. The method of claim 9 , wherein the training further includes generating an expected perplexity and a standard deviation for each leaf category based on the SLM for each leaf category and perplexity and standard deviation calculations for each sample item listing.
11. The method of claim 7 , wherein the generating the perplexity deviation signal includes computing a sentence log probability.
12. The method of claim 7 , further comprising:
creating a tuning set of item listings by labeling item listings as miscategorized or non-miscategorized based on application of filters to the item listings; and
feeding the tuning set of item listings to a GBM for tuning of a GBM model used by the GBM.
13. The method of claim 7 , wherein the GBM takes a product type signal as input.
14. A non-transitory machine-readable storage medium having instruction data to cause a machine to perform operations comprising:
training one or more leaf-category-specific unsupervised statistical language model (SLM) models using sample item listings corresponding to each of one or more leaf categories and structured data about the one or more leaf categories, the training including calculating an expected perplexity and a standard deviation for item listing titles;
in response to a request for a miscategorization classification score corresponding to a particular item listing, calculating a perplexity for a title of the particular item listing and generating a perplexity deviation signal based on a difference between the perplexity for the title of the particular item listing and the expected perplexity for item listing titles in a leaf category of the particular item listing and based on the standard deviation for item listing titles in the leaf category of the particular item listing;
using a gradient boosting machine (GBM) to fuse the perplexity deviation signal with one or more other signals to generate a miscategorization classification score corresponding to the particular item listing; and
causing a graphical user interface to be rendered, the graphical user interface including a first section where a user can enter textual data pertaining to the particular item listing, including a title of the particular item listing, and a second section where top n categories are presented, the top n categories being selected from the one or more leaf categories and representing n leaf categories having lowest miscategorization score for a first item listing, the second section further displaying the miscategorization classification score next to each of the top n categories and a subsection where the user can select one of the top n categories to assign to the particular item listing.
15. The non-transitory machine-readable storage medium of claim 14 , wherein the training comprises calculating a sentence perplexity PP(S) for each sequence S of N words {w 1 , w 2 , . . . , w N } in each title of each of the sample item listings according to the following formula:
PP
(
S
)
=
P
(
w
1
…
w
N
)
-
1
/
N
=
∏
i
=
1
N
1
P
(
w
1
|
w
1
…
w
i
-
1
)
N
.
16. The non-transitory machine-readable storage medium of claim 14 , wherein the training further includes generating an SLM for each leaf category for structured data, an SLM for each leaf category's queries, and an SLM for each leaf category's titles, and interpolating the SLM for each leaf category for structured data, the SLM for each leaf category's queries, and the SLM for each leaf category's titles into an SLM for each leaf category.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the training further includes generating an expected perplexity and a standard deviation for each leaf category based on the SLM for each leaf category and perplexity and standard deviation calculations for each sample item listing.
18. The non-transitory machine-readable storage medium of claim 14 , wherein the generating the perplexity deviation signal includes computing a sentence log probability.
19. The non-transitory machine-readable storage medium of claim 14 , wherein the operations further comprise:
creating a tuning set of item listings by labeling item listings as miscategorized or non-miscategorized based on application of filters to the item listings; and
feeding the tuning set of item listings to a GBM for tuning of a GBM model used by the GBM.
20. The non-transitory machine-readable storage medium of claim 14 , wherein the GBM takes a product type signal as input.