Method and apparatus for categorizing and presenting documents of a distributed database
View Patent ↗Described herein are methods for creating categorized documents, categorizing documents in a distributed database and categorizing Resulting Pages. Also described herein is an apparatus for searching a distributed database. The method for creating categorized documents generally comprises: initially assuming all documents are of type 1; filtering out all type 2 documents and placing them in a first category; filtering out all type 3 documents and placing them in a second category; and defining all remaining documents as type 4 documents and placing all type 4 documents in a third category. The apparatus for searching a distributed database generally comprises at least one memory device; a computing apparatus; an indexer; a transactional score generator; and a category assignor; a search server; and a user interface in communication with the search server.
1. A computer-implemented method for categorizing Resulting Pages into categories, comprising:
designating a first category as commercial pages and a second category as informational pages;
determining a quality score q(wi) for each Resulting Page;
determining a transactional rating τ(w i )for each Resulting Page,
deriving a propagation matrix P;
determining a commercial score κ for each Resulting Page;
filtering out all Resulting Pages that meet or exceed a commercial score threshold value;
wherein the Resulting Pages that meet or exceed the commercial page threshold value are placed in the first category and all remaining Resulting Pages are placed in the second category,
wherein the determining the transactional rating τ(w i ) comprises
determining whether each Resulting Page meets select criteria,
determining how strongly each Resulting Page meets the select criteria,
determining a transactional score for each page, and
determining the transactional rating for each page from the transactional score, and
wherein determining a transactional score for each page comprises creating a vector for each Resulting Page αk(w i ), wherein each vector contains a plurality of elements αk n (w i ), wherein each of the plurality of elements αk n (w i ) is a Boolean value that reflects how strongly each of the Resulting Pages meets each of the select criteria.
2. A computer-implemented method for categorizing Resulting Pages into categories, comprising:
designating a first category as commercial pages and a second category as informational pages;
determining a quality score q(wi) for each Resulting Page;
determining a transactional rating τ(w i ) for each Resulting Page,
deriving a propagation matrix P;
determining a commercial score κ for each Resulting Page;
filtering out all Resulting Pages that meet or exceed a commercial score threshold value;
wherein the Resulting Pages that meet or exceed the commercial page threshold value are placed in the first category and all remaining Resulting Pages are placed in the second category,
wherein the determining the transactional rating τ(w i ) comprises
determining whether each Resulting Page meets select criteria,
determining how strongly each Resulting Page meets the select criteria,
determining a transactional score for each page, and
determining the transactional rating for each page from the transactional score, and
wherein determining a transactional score for each page comprises creating a vector for each Resulting Page βk(w i ), wherein each vector contains a plurality of elements βk n (w i ), wherein each of the plurality of elements βk n (w i ) is a weighted value that reflects how strongly each of the Resulting Pages meets each of the select criteria.
3. A computer-implemented method for categorizing Resulting Pages into categories, comprising:
designating a first category as commercial pages and a second category as informational pages;
determining a quality score q(wi) for each Resulting Page;
determining a transactional rating τ(w i ) for each Resulting Page,
deriving a propagation matrix P;
determining a commercial score κ for each Resulting Page;
filtering out all Resulting Pages that meet or exceed a commercial score threshold value;
wherein the Resulting Pages that meet or exceed the commercial page threshold value are placed in the first category and all remaining Resulting Pages are placed in the second category,
wherein the determining the transactional rating τ(w i ) comprises
determining whether each Resulting Page meets select criteria,
determining how strongly each Resulting Page meets the select criteria,
determining a transactional score for each page, and
determining the transactional rating for each page from the transactional score, and
wherein determining the transactional rating τ(w i ) for each page from the transactional score comprises evaluating a relationship between the transactional rating τ(w i ), and a p-norm of a vector for each Resulting Page αk(w i ) wherein the relationship is defined by
τ
(
w
i
)
=
α
(
w
i
)
ρ
=
(
∑
i
=
1
n
α
(
w
i
)
ρ
)
-
ρ
.
4. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 3 wherein ρ=2.
5. A computer-implemented method for categorizing Resulting Pages into categories, comprising:
designating a first category as commercial pages and a second category as informational pages;
determining a quality score q(wi) for each Resulting Page;
determining a transactional rating τ(w i ) for each Resulting Page,
deriving a propagation matrix P;
determining a commercial score κ for each Resulting Page;
filtering out all Resulting Pages that meet or exceed a commercial score threshold value;
wherein the Resulting Pages that meet or exceed the commercial page threshold value are placed in the first category and all remaining Resulting Pages are placed in the second category,
wherein the determining the transactional rating τ(w i ) comprises
determining whether each Resulting Page meets select criteria,
determining how strongly each Resulting Page meets the select criteria,
determining a transactional score for each page, and
determining the transactional rating for each page from the transactional score, and
wherein determining the transactional rating τ(w i ) for each page from the transactional score comprises evaluating a relationship between the transactional rating τ(w i ) and a p-norm of a vector for each Resulting Page βk(w i ) wherein the relationship is defined by
τ
(
w
i
)
=
β
(
w
i
)
ρ
=
(
∑
i
=
1
n
β
(
w
i
)
ρ
)
-
ρ
.
6. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 5 wherein ρ=2.
7. A computer-implemented method for categorizing Resulting Pages into categories, comprising:
designating a first category as commercial pages and a second category as informational pages;
determining a quality score q(wi) for each Resulting Page;
determining a transactional rating for each Resulting Page τ(w i );
deriving a propagation matrix; P
determining a commercial score κ for each Resulting Page;
filtering out all Resulting Pages that meet or exceed a commercial score threshold value;
wherein the Resulting Pages that meet or exceed the commercial page threshold value are placed in the first category and all remaining Resulting Pages are placed in the second category
wherein deriving a propagation matrix, comprises:
creating a hyperlink connectivity matrix C containing elements Ci,j;
calculating a plurality of authority scores ai and a plurality of hub scores hi;
calculating a plurality of transition counts Ti,j and a plurality of pageviews vi for each Resulting Page; and
creating the propagation matrix P containing propagation matix elements Pi,j.
8. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 7 , wherein creating a hyperlink connectivity matrix C comprises: representing a link structure of the Internet in a matrix.
9. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 8 , wherein the link structure if the Internet is represented by examining a number of links from each Resulting Page to each Resulting Page.
10. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 7 , wherein the plurality of hub scores hi and the plurality of authority scores are related to the hyperlink connectivity matrix C and wherein the plurality of authority scores ai are defined as:
a
i
=
∑
j
C
j
,
i
and wherein the plurality of hub scores are defined as:
h
i
=
∑
j
C
i
,
j
,
respectively.
11. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 7 , wherein the plurality of pageviews vi are related to the plurality of transition counts Ti,j and are defined by:
v
i
=
∑
j
T
i
,
j
.
12. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 11 , wherein the propagation matrix is a function of the hyperlink connectivity matrix, the plurality of hub scores, the plurality of authority scores, the plurality of transition counts and the plurality pageviews.
13. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 11 , wherein calculating the propagation matrix further comprises weighting the plurality of hub scores, the plurality of authority scores, and the plurality pageviews.
14. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 11 , wherein the propagation matrix P is a further function of weighing functions F(hi), G(ai) and H(vi), and wherein the propagation matrix P is defined as:
P
i
,
j
=
f
(
C
i
,
j
)
+
g
(
C
i
,
j
,
a
i
)
+
h
(
T
i
,
j
,
v
i
)
F
(
h
i
)
+
G
(
a
i
)
+
H
(
v
i
)
.
15. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 14 , wherein each of the weighting functions comprises a step function.
16. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 15 , wherein the commercial score κ for each Resulting Page wi is determined recursively.
17. A computer-implemented method for categorizing Resulting Pages into categories, as claimed in claim 16 , wherein the commercial score κ is recursively determined over t iterations from a transverse of the propagation matrix P T , a propagation matrix weight η, and a commercial score initial value κ′(0), wherein κ′(0) is weighted by select quantities A and B and defined as:
κ
′
(
0
)
=
A
τ
(
w
i
)
+
Bq
(
w
i
)
+
σ
(
w
i
)
A
+
B
+
1
and a prior iteration of the commercial score κ′(t), wherein κ′(t) is defined as: κ′(t)=ηP T κ′(t−1)+(1−η)κ′(0), and wherein κ=κ′(t′).
18. A computer-implemented method for categorizing Resulting Pages into categories, comprising:
designating a first category as commercial pages and a second category as informational pages;
determining a quality score q(wi) for each Resulting Page;
determining a transactional rating for each Resulting Page τ(w i );
deriving a propagation matrix; P
determining a commercial score κ for each Resulting Page;
filtering out all Resulting Pages that meet or exceed a commercial score threshold value;
wherein the Resulting Pages that meet or exceed the commercial page threshold value are placed in the first category and all remaining Resulting Pages are placed in the second category, and
designating a third category as spam pages; and determining a spam score σ(wi) for each Resulting Page;
wherein determining the commercial score κ for each Resulting Page is recursively determined over t iterations from a transverse of the propagation matrix P T , propagation matrix weight η and commercial score initial value κ′(0), wherein κ′(0) is weighted by select quantities A and B and defined as:
κ
′
(
0
)
=
A
τ
(
w
i
)
+
Bq
(
w
i
)
+
σ
(
w
i
)
A
+
B
+
1
and a prior iteration of the commercial score κ′(t), wherein κ′(t) is defined as: κ′(t)=ηP T κ′(t−1)+(1−η)κ′(0), and wherein κ=κ′(t′).