Text mining-based attribute analysis method for internet media users
View Patent ↗A text mining-based attribute analysis method for Internet media users comprises a first steps of establishing a label main corpus and a feature corpus sequentially, and updating and maintaining the label main corpus and the feature corpus respectively, and a second step of extracting all history article samples of Internet users, and cleaning out videos, audios and pictures in the samples. The text mining-based attribute analysis method can form attributes of browsed sample articles for each Internet media user, and analyze accurately weights of interesting categories, to identify deeply, analyze, and mine the user attributes, and the basic attributes of the Internet users can also be analyzed.
1. A text mining-based attribute analysis method for Internet media users comprising:
block 1 , establishing a label main corpus and a feature corpus sequentially, and updating and maintaining the label main corpus and the feature corpus respectively;
block 2 , extracting all history article samples of the Internet media users, and cleaning out videos, audios and pictures of the history article samples;
block 3 , performing on the extracted history article samples dynamic clustering and fuzzy clustering synchronously, and performing word frequency classification, semantic analysis, class feature analysis, class parameter amendment, and density noise reduction processing, sequentially, to generate an ordered classified text and calculate a first noise level A according to a first result of the ordered classified text;
block 4 , comparing the first noise level A with a threshold a to determine whether the first noise level A is less than the threshold a; if yes, the method goes to block 5 ; if no, the method goes to block 3 ;
block 5 , when the first noise level A is less than the threshold a, performing model clustering, semantic analysis, class feature analysis, and density noise reduction processing, sequentially, to generate a second ordered classified text and calculate a second noise level B according to a second result of the second ordered classified text;
block 6 , comparing the second noise level B with the threshold a to determine whether the second noise level B is less than the threshold a; if yes, the method goes to block 5 ; if no, amend the class parameters, then the method goes to block 5 ; and
block 7 , performing model classification to form an attribute collection of the Internet media users;
wherein the threshold a is greater than 0 and less than 0.4.
2. The method of claim 1 , wherein the step of establishing the label main corpus comprising:
block a 1 , extracting article samples, cleaning the article samples, and cleaning out audio, video, pictures, incomplete articles, garbled, and illegal characters;
block a 2 , classifying artificially according to a label library;
block a 3 , the article samples being clustered dynamically and clustered fuzzily at the same time, and setting the cluster parameters;
block a 4 , performing semantic analysis, cluster feature analysis, cluster parameter amendment, and density noise reduction processing, sequentially, to generate a first main corpus ordered classified text and calculating a third noise level M according to a third result of the first main corpus ordered classified text;
block a 5 , comparing the third noise level M with a threshold a to determine whether the third noise level M is less than the threshold a; if yes, the method goes to block a 6 ; if no, the method goes to block a 3 ;
block a 6 , performing model clustering, semantic analysis, class feature analysis, class parameter amendment, and density noise reduction processing, sequentially, to generate a second main corpus ordered classified text and calculating a fourth noise level N according to a fourth result of the second main corpus ordered classified text;
block a 7 , comparing the fourth noise level N with the threshold a to determine whether the fourth noise level N is less than the threshold a; if yes, the method goes to block a 8 ; if no, amend the label library, then the method goes to block a 6 ; and
block a 8 , performing model classification to form the label main corpus.
3. The method of claim 2 , wherein the step of establishing the feature corpus comprising:
block b 1 , the label main corpus being performed sample frequency analysis and semantic analysis in turn;
block b 2 , performing high word frequency classification; and
block b 3 , applying a mapping model of feature words and the label library to form the feature corpus.
4. The method of claim 2 , wherein the step of updating and maintaining the label main corpus and the feature corpus respectively comprising:
block c 1 , extracting the total number of classified article samples;
block c 2 , performing word frequency classification, semantic analysis, density noise reduction processing, and noise data cleaning in turn, classifying samples, and updating the label main corpus or the feature corpus; and
block c 3 , collecting new labels and extracting article samples with new labels.
5. The method of claim 4 , wherein each of the noise level A, B, M, and N is calculated using the following unified algorithm:
considering the given object set D, k-distance neighborhood of an object o being marked as disk k(o), is a distance dist(o, p) between the object o and another object p (p∈D), thus,
there are at least K objects o′, and o′∈D, thus, disk (o, o′)≤dist(o, p);
there are at least K−1 objects o′, and o′∈D, thus, disk (o, o′)≤dist(o, p);
thus: Nk(o)={o′|o′∈D, dist(o, o′)≤distk(o)};
there are two objects o and o′, if dist(o, o′)>distk(o), a reachable distance between the object o and the object o′ is dist(o, o′); if dist(o, o′)≤distk(o), the reachable distance between the object o and the object o′ is distk(o);
thus, reachdist k(o←o′)=max{distk(o), dist(o, o′)};
a local reachable density of the objects o is shown as follows:
lrdk
(
o
)
=
Nk
(
o
)
∑
o
′
∈
Nk
(
o
)
reachdistk
(
o
←
o
′
)
;
thus, the local outlier factor of the objects o is defined as follows:
LOFk
(
o
)
=
∑
o
′
∈
Nk
(
o
)
lrdk
(
o
)
lrdk
(
o
′
)
Nk
(
o
)
.
6. The method of claim 1 , wherein the step of establishing the feature corpus comprising:
block b 1 , the label main corpus being performed sample frequency analysis and semantic analysis in turn;
block b 2 , performing high word frequency classification; and
block b 3 , a mapping model of feature words and the label library being established, to form the feature corpus.
7. The method of claim 1 , wherein the step of updating and maintaining the label main corpus and the feature corpus respectively comprising:
block c 1 , extracting the total number of classified article samples;
block c 2 , performing word frequency classification, semantic analysis, density noise reduction processing, and noise data cleaning in turn, classifying samples, and updating the label main corpus or the feature corpus; and
block c 3 , collecting new labels and extracting article samples with new labels.
8. The method of claim 7 , wherein each of the noise level A, B, M, and N is calculated using the following unified algorithm:
considering the given object set D, k-distance neighborhood of an object o being marked as disk k(o), is a distance dist(o, p) between the object o and another object p (p D), thus,
there are at least K objects o′, and o′∈D, thus, disk (o, o′)≤dist(o, p);
there are at least K−1 objects o′, and o′∈D, thus, disk (o, o′)≤dist(o, p);
thus: Nk(o)={o′|o′∈D, dist(o, o′)≤distk(o)};
there are two objects o and o′, if dist(o, o′)>distk(o), a reachable distance between the object o and the object o′ is dist(o, o′); if dist(o, o′)≤distk(o), the reachable distance between the object o and the object o′ is distk(o);
thus, reachdist k(o←o′)=max{distk(o), dist(o, o′)};
a local reachable density of the objects o is shown as follows:
lrdk
(
o
)
=
Nk
(
o
)
∑
o
′
∈
Nk
(
o
)
reachdistk
(
o
←
o
′
)
;
thus, the local outlier factor of the objects o is defined as follows:
LOFk
(
o
)
=
∑
o
′
∈
Nk
(
o
)
lrdk
(
o
)
lrdk
(
o
′
)
Nk
(
o
)
.