Page 229 - Artificial Intelligence for the Internet of Everything

210 Artificial Intelligence for the Internet of Everything

the positive dataset from textual customer complaints dataset (Galitsky et al.,
2009; Github, 2018) scraped from consumer advocacy site PlanetFeedback.
com. This dataset is used for both argument detection (first step) and argument
validity (second step) tasks. For argument detection we attempt to identify if a
given paragraph of text contains an argument, in a domain-independent
manner. For argument validation, in the second step, if we detected an argu-
ment in the first step we try to validate it having the domain-ontology built in
a given vertical domain such as a landlord-tenant dispute. If an argument
has not been detected in the first step, we have nothing to validate.
For the negative dataset, only for the affective argument detection task, we
used Wikipedia, factual news sources, and also the component of Lee’s (2001)
dataset that includes such sections of the corpus as: [“tells”], instructions for
how to use software; [“tele”], instructions for how to use hardware; and
[news], a presentation of a news article in an objective, independent manner,
and others. Further details on the data set are available in Galitsky et al. (2015).
Each row indicates a method used to detect the presence of argumen-
tationinaparagraph. We startwithbaselinemethods based onkeywords
and their frequencies (second and third row on the top, Table 11.2). The
second column shows precision (P), the third recall, and the fourth F1
measure. Frequently, a coordinated pair of communicative actions (so that
at least one has a negative sentiment polarity related to an opponent) is a
hint that logical argumentation is present. This naı ¨ve approach is outper-
formed by the top-performing TK-learning CDT approach by 29%. SVM
TK of CDT outperforms SVM TK for RST+CA and RST+full parse
trees (Galitsky et al., 2018) by about 5% due to noisy syntactic data, which
is frequently redundant for argumentation detection.
The SVM TK approach provides an acceptable F-measure but does not
help to explain how exactly the affective argument identification problem is
solved, providing only final scoring and class labels. The nearest-neighbor
maximal common sub-graph algorithm is much more fruitful in this respect

Table 11.2 Evaluation results for argument detection
Method/sources P R F1
Bag-of-words 57.2 53.1 55.07
WEKA-Naı ¨ve Bayes 59.4 55.0 57.12
SVM TK for RST and CA (full parse trees) 77.2 74.4 75.77
SVM TK for DT 63.6 62.8 63.20
SVM TK for CDT 82.4 77.0 79.61