System and method for analysing natural language
A computer implemented method for analyzing natural language to determine a sentiment between two entities discussed in the natural language, comprising the following steps: receiving the natural language at a processing circuitry; analyzing the natural language to determine a syntactic representation which shows syntactic constituents of the analyzed natural language and to determine a sentiment score of each constituent; determining which constituents link the two entities; and calculating an overall sentiment score for the sentiment between the two entities by processing the sentiment score of each constituent of the constituents determined to link the two entities.
1. A computer implemented method for analysing natural language contained in electronic text to determine a sentiment between two entities discussed in the natural language, comprising the following steps:
receiving, via an input module, the electronic text containing the natural language at a processing circuitry;
using an input/output subsystem of the processing circuitry to move the received natural language to a data storage;
analysing the natural language in the data storage to determine a syntactic representation which shows syntactic constituents of the analysed natural language and to determine a sentiment score of each constituent, wherein each constituent is a sentiment (sub)context, and wherein determining a sentiment score of each constituent comprises:
determining a polarity distribution of each (sub)context, and
calculating an entity sentiment score for each entity from the cumulative polarity distributions D 1 . . . D n across all (sub)contexts z 1 . . . z n , the entity sentiment score for polarity p of entity e in sentence s with n (sub)contexts being obtained through the equation:
e_scr
(
p
,
e
)
=
∑
i
=
1
n
fD
i
wherein
z is Constituent as sentiment (sub)context
p is Polarity pε{POS, NTR, NEG} assigned to z, by sentiment grammar
D
i
is
Polarity
distribution
of
z
j
:=
{
f
if
p
is
POS
f
if
p
is
NEG
0.5
·
f
if
p
is
NTR
s is Sentence
d is Dijkstra's shortest path distance between sentence root and head of z i
β is Coverage coefficient:
length(z i )/length(s)
f is Weight ƒ or z i :
d
+
β
length
(
s
)
;
determining which constituents link the two entities;
calculating an overall sentiment score for the sentiment between the two entities by processing the sentiment score of each constituent of the constituents determined to link the two entities; and
outputting, by a computer processor, the overall sentiment score for the sentiment between the two entities discussed by the natural language contained in the electronic text.
2. A method according to claim 1 wherein the syntactic representation is a tree showing how the entities within the natural language are connected to one another.
3. A method according to claim 2 , wherein the determination as to which constituents link the two entities comprises performing a tree search to determine a shortest path.
4. A method according to claim 1 wherein a sentiment score for a constituent is determined from an entity sentiment score of an entity within the natural language.
5. A method according to claim 1 , wherein processing the sentiment score of each constituent of the constituents determined to link the two entities comprises using a windowed method to include a plurality of entities.
6. A method according to claim 5 wherein the windowed method comprises using a set of rules to provide a score for the arrangement of entities within the window.
7. A non-transitory computer-readable medium storing executable computer program code for analysing natural language contained in electronic text to determine a sentiment between two entities discussed in the natural language, the computer program code executable to perform steps comprising:
receiving, via an input module, the elecronic text containing the natural language at a processing circuitry;
using an input/output subsystem of the processing circuitry to move the received natural language to a data storage;
analysing the natural language in the data storage to determine a syntactic representation which shows syntactic constituents of the analysed natural language and to determine a sentiment score of each constituent, wherein each constituent is a sentiment (sub)context, and wherein determining a sentiment score of each constituent comprises:
determining a polarity distribution of each (sub)context, and
calculating an entity sentiment score for each entity from the cumulative polarity distributions D 1 . . . D n across all (sub)contexts z 1 . . . z n , the entity sentiment score for polarity p of entity e in sentence s with n (sub)contexts being obtained through the equation:
e_scr
(
p
,
e
)
=
∑
i
=
1
n
fD
i
wherein
z is Constituent as sentiment (sub)context
p is Polarity pε{POS, NTR, NEG} assigned to z i by sentiment grammar
D
i
is
Polarity
distribution
of
z
j
:=
{
f
if
p
is
POS
f
if
p
is
NEG
0.5
·
f
if
p
is
NTR
s is Sentence
d is Dijkstra's shortest path distance between sentence root and head of z i
β is Coverage coefficient:
length(z i )/length(s)
f is Weight ƒ or z i :
d
+
β
length
(
s
)
;
determining which constituents link the two entities;
calculating an overall sentiment score for the sentiment between the two entities by processing the sentiment score of each constituent of the constituents determined to link the two entities; and
outputting the overall sentiment score for the sentiment between the two entities discussed by the natural language contained in the electronic text.
8. A computer system for analyzing natural language contained in electronic text to determine a sentiment between two entities discussed in the natural language comprising:
a computer processor for executing computer program code; and
a non-transitory computer-readable storage medium storing executable computer program code comprising:
an input module arranged to receive the electronic text containing the natural language at a processing circuitry;
an input/output subsystem of the processing circuitry arranged to move the received natural language to a data storage;
an analysing module arranged to analyse the natural language in the data storage to determine a syntactic representation which shows syntactic constituents of the analysed natural language to determine a sentiment score of each constituent t, wherein each constituent is a sentiment (sub)context, and wherein determining a sentiment score of each constituent comprises:
determining a polarity distribution of each (sub)context, and
calculating an entity sentiment score for each entity from the cumulative polarity distributions D 1 . . . D n across all (sub)contexts z 1 . . . z n , the entity sentiment score for polarity p of entity e in sentence s with n (sub)contexts being obtained through the equation:
e_scr
(
p
,
e
)
=
∑
i
=
1
n
fD
i
wherein
z is Constituent as sentiment (sub)context
p is Polarity pε{POS, NTR, NEG} assigned to z, by sentiment grammar
D
i
is
Polarity
distribution
of
z
j
:=
{
f
if
p
is
POS
f
if
p
is
NEG
0.5
·
f
if
p
is
NTR
s is Sentence
d is Dijkstra's shortest path distance between sentence root and head of z i
β is Coverage coefficient:
length(z i )/length(s)
f is Weight ƒ or z i :
d
+
β
length
(
s
)
;
a link determining module arranged to determine which constituents link the two entities;
a calculating module arranged to calculate an overall sentiment score for the sentiment between the two entities by processing the sentiment score of each constituent of the constituents determined to link the two entities; and
an outputting module arranged to output the overall sentiment score for the sentiment between the two entities discussed by the natural language contained in the electronic text.
9. A computer implemented method for analysing natural language contained in electronic text to determine a sentiment between two entities discussed in the natural language, comprising the following steps:
receiving, via an input module, the electronic text containing the natural language at a processing circuitry;
using an input/output subsystem of the processing circuitry to move the received natural language to a data storage;
analysing the natural language in the data storage to determine a syntactic representation which shows the syntactic constituents of the analysed natural language together with determining a sentiment score of each constituent, wherein each constituent is a sentiment (sub)context;
wherein the determining a sentiment score of each constituent comprises using a relation classifier to traverse a dependency path using a limited sliding window, each window position of which represents a (sub)context triple (z i−1 , z i , z i+1 ), and to determine a polarity distribution of each (sub)context;
determining which constituents link the two entities;
calculating an overall sentiment score for the sentiment between the two entities by processing the sentiment score of each constituent of the constituents determined to link the two entities, wherein calculating an overall sentiment score for the sentiment between the two entities comprises:
calculating the overall sentiment scores for the sentiment between the two entities from the cumulative polarity distributions D 1 . . . D n across all (sub)contexts z 1 . . . z n , respectively, the cumulative scores for the three polarity counts (r.pos, r.ntr, r.neg) for sentiment r between the two entities (e 1 , e 2 ) in sentence s with n (sub)contexts being obtained through the following equation:
r_scr
(
r
,
e
1
,
e
2
)
=
∑
i
=
1
n
fD
i
wherein
z i is Constituent as sentiment (sub)context
t i is Triple of previous current, and next (subcontext) along the path between e 1 and e 2 :
(z i−1 , z i , z i+1 )
p is Polarity pε{POS, NTR, NEG} assigned to t, by sentiment grammar
D
is
Polarity
distribution
of
z
i
:=
{
r
·
pos
=
α
,
r
·
ntr
=
1
-
α
if
p
is
POS
r
·
neg
=
α
,
r
·
ntr
=
1
-
α
if
p
is
NEG
r
·
ntr
=
α
if
p
is
NTR
d is Dijkstra's shortest path distance between e 1 and e 2 heads
α
is
(
Sub
)
context
score
:=
1
log
2
(
d
)
.
outputting, by a computer processor, the overall sentiment score for the sentiment between the two entities discussed by the natural language contained in the electronic text.
10. A method according to claim 9 wherein the syntactic representation is a tree showing how the entities within the natural language are connected to one another.
11. A method according to claim 9 wherein the sentiment score for each constituent is determined from a sentiment score from the or each entity within the natural language.
12. A method according to claim 10 , wherein the determination as to which constituents linking the two entities comprises performing a tree search to determine a shortest path.
13. A method according to claim 9 , wherein processing the sentiment score of each constituent of the constituents determined to link the two entities includes using a windowed method to include a plurality of entities.
14. A method according to claim 13 wherein the windowed method includes using a set of rules to provide a score for the arrangement of entities within the window.
15. A non-transitory computer-readable medium storing executable computer program code for analysing natural language contained in electronic text to determine a sentiment between two entities discussed in the natural language, the computer program code executable to perform steps comprising:
receiving, via an input module, the electronic text containing the natural language at a processing circuitry;
using an input/output subsystem of the processing circuitry to move the received natural language to a data storage;
analysing the natural language in the data storage to determine a syntactic representation which shows the syntactic constituents of the analysed natural language together with determining a sentiment score of each constituent, wherein each constituent is a sentiment (sub)context,
wherein the determining a sentiment score of each constituent comprises using a relation classifier to traverse a dependency path using a limited sliding window, each window position of which represents a (sub)context triple (z i−1 , z i , z i+1 ), and to determine a polarity distribution of each (sub)context;
determining which constituents link the two entities;
calculating an overall sentiment score for the sentiment between the two entities by processing the sentiment score of each constituent of the constituents determined to link the two entities, wherein the calculating an overall sentiment score for the sentiment between the two entities comprises:
calculating the overall sentiment scores for the sentiment between the two entities from the cumulative polarity distributions D 1 . . . D n across all (sub)contexts z 1 . . . z n , respectively, the cumulative scores for the three polarity counts (r.pos, r.ntr, r.neg) for sentiment r between the two entities (e 1 , e 2 ) in sentence s with n (sub)contexts being obtained through the following equation:
r
scr
(
r
,
e
1
,
e
2
)
=
∑
i
=
1
n
fD
i
wherein
z i is Constituent as sentiment (sub)context
t i is Triple of previous current, and next (subcontext) along the path between e i and e 2 :
(z i−1 , z i , z i+1 )
p is Polarity pε{POS, NTR, NEG} assigned to t i by sentiment grammar
D
is
Polarity
distribution
of
z
i
:=
{
r
·
pos
=
α
,
r
·
ntr
=
1
-
α
if
p
is
POS
r
·
neg
=
α
,
r
·
ntr
=
1
-
α
if
p
is
NEG
r
·
ntr
=
α
if
p
is
NTR
d is Dijkstra's shortest path distance between e 1 and e 2 heads
α
is
(
Sub
)
context
score
:=
1
log
2
(
d
)
.
outputting the overall sentiment score for the sentiment between the two entities discussed by the natural language contained in the electronic text.