NLP Notes

Abstract

We describe MITRE’s two submissions to
the RTE Challenge, intended to exemplify
two different ends of the spectrum of possibilities.
The first submission is a traditional
system based on linguistic analysis
and inference, while the second is inspired
by alignment approaches from machine
translation. We also describe our
efforts to build our own entailment corpus.
Finally, we discuss our investigations
and reflections on the strengths and
weaknesses of the evaluation itself.

The Paper: http://www.cs.biu.ac.il/~glikmao/rte05/bayer_et_al.pdf

CLaC PASCAL Algorithm:

- Use the coreference resolution system to produce coreference chains for T, H, T and H as a unit.

- For each sentence, determine Predicate-Argument-Structures (PAS), based on the parsing, NP chunking and verb grouping results.

- Apply cardinality filter (if there exists a numeric value N in H that does not exist in T, return false)

- Convert passive voices into active voices.

- For each PAS pair, compute WN distance for verbs in T and H
If WN distance <= threshold & both PASs are in comparable structures, and If there is coreference between corresponding parts, return true - Apply Be-Heuristic Algorithm if H contains the pattern "X is Y", and
(X is in H) & (X' is in T) and {X, X'} belong to the same inter-sentence coreference chain, and
(Y is in H) & (Y' is in T) and {Y, Y'} belong to the same inter-sentence coreference chain, and
X' corefers with Y'
then return true
return false

Precision: around 0.5
Recall: around 0.2
Accuracy: around 0.55
Confidence-Weighted Score (cws): around 0.55


The Paper: http://www.cs.biu.ac.il/~glikmao/rte05/andreevskaia_et_al.pdf

Abstract

The CLaC Lab’s system for the PASCAL
RTE challenge explores the potential
of simple general heuristics and
a knowledge-poor approach for recognising
paraphrases, using NP coreference,
NP chunking, and two parsers
(RASP and Link) to produce Predicate
Argument Structures (PAS) for each of
the pair components. WordNet lexical
chains and a few specialised heuristics
are used to establish semantic similarity
between corresponding components
of the PAS from the pair. We discuss
the results and potential of this
approach.

GATE is made up of three elements:

• An architecture describing how language processing systems are made up of components.

• A framework (or class library, or SDK), written in Java and tested on Linux, Windows and Solaris.

• A graphical development environment built on the framework.

http://gate.ac.uk/

http://www.cs.biu.ac.il/~glikmao/rte05/akhmatova.pdf

Kullanilan kaynaklar:

• Wordnet
• Link Parser
  
• OTTER

Kullanilan Teknikler:

• first-order logic forms
• Wordnet Relatedness Algorithm
  

Detaylar:

Text (T) ve Hypothesis (H) 'te proposition'lar detect ediliyor. Eger H'teki her bir proposition icin T'de bir matching proposition var ise, T entails H.

- Asagidaki sekilde logical formlar kullaniyorlar:
3 tip object var: Subj(x), Obj(x), Pred(x)
and a meaning attaching element: iq(x, )
ORNEK:
"Somali capital" -- Subj(x) & iq(x, 'capital') & attr(x, y) & Subj(y) & iq(y, 'somali')
"a zoo in Berlin" -- Obj(x) & iq(x, 'zoo') &amp; prep(x, y) & Obj(y) & iq(y, 'Berlin')

- Wordnet 'teki relation'lardan logic formula'lar generate ediyorlar.
ORNEK:
Semantic synonymy is expressed as an equivalence:

http://www.cs.biu.ac.il/~glikmao/rte05/akhmatova.pdf

Abstract
This paper presents an approach to solving
the problem of textual entailment
recognition and describes the computer
application built to demonstrate the
performance of the proposed approach.
The method presented here is based on
syntax-driven semantic analysis and uses
the notion of atomic proposition as its
main element for entailment recognition.
The idea is to find the entailment relation
in the sentence pairs by comparing the
atomic propositions contained in the text
and hypothesis sentences.
The comparison of atomic propositions is
performed via an automated deduction
system OTTER; the propositions are
extracted from the output of the Link
Parser; and semantic knowledge is taken
from the WordNet database. On its current
stage the system is capable to recognize
basic semantically and syntactically based
entailments and is potentially capable to
use more external and internal knowledge
to deal with more complex entailments.

• Every gardener likes the sun.
  (Ax) gardener(x) => likes(x,Sun)
• You can fool some of the people all of the time.
  (Ex) (person(x) ^ (At)(time(t) => can-fool(x,t)))
• You can fool all of the people some of the time.
  (Ax) (person(x) => (Et) (time(t) ^ can-fool(x,t)))
• All purple mushrooms are poisonous.
  (Ax) (mushroom(x) ^ purple(x)) => poisonous(x)
• No purple mushroom is poisonous.
  ~(Ex) purple(x) ^ mushroom(x) ^ poisonous(x)
  or, equivalently,
  (Ax) (mushroom(x) ^ purple(x)) => ~poisonous(x)
• There are exactly two purple mushrooms.
  (Ex)(Ey) mushroom(x) ^ purple(x) ^ mushroom(y) ^ purple(y) ^ ~(x=y) ^ (Az) (mushroom(z) ^ purple(z)) => ((x=z) v (y=z))
• Deb is not tall.
  ~tall(Deb)
• X is above Y if X is on directly on top of Y or else there is a pile of one or more other objects directly on top of one another starting with X and ending with Y.
  (Ax)(Ay) above(x,y) <=> (on(x,y) v (Ez) (on(x,z) ^ above(z,y)))

http://www.cs.wisc.edu/~dyer/cs540/notes/fopc.html

Text'i cumlelere ayirmak icin MXTERMINATOR kullanilmis.

Her bir cumleyi parse etmek icin de MINIPAR kullanilmis.

Cumlelerdeki her bir "node" (noktalama isaretleri dahil) su attribute'lara sahip:
- node (e.g. 7)
- word (e.g. "declined")
- lemma (e.g. "decline")
- pos (e.g. v)
- relation (e.g. i) (bkz. MINIPAR Grammatical Categories)
- parent node (for the relation) (e.g. E3)

Parent Node iliskisi kullanilarak tree yapisi elde edilebilir.

RTE1 Dataset / Development: http://www.cs.biu.ac.il/~glikmao/dev2.zip
RTE1 Dataset / Test: http://www.cs.biu.ac.il/~glikmao/test.zip
RTE1 Dataset / Annotated: http://www.cs.biu.ac.il/~glikmao/annotated_test.zip

RTE 2 Dataset: http://ir-srv.cs.biu.ac.il:64080/RTE2/PASCAL/index.php?action=download

The meanings of grammatical categories are explained as follows:

Det: Determiners
PreDet: Pre-determiners (search for PreDet in data/wndict.lsp for instances)
PostDet: Post-determiners (search for PostDet in data/wndict.lsp for instances)
NUM: numbers
C: Clauses
I: Inflectional Phrases
V: Verb and Verb Phrases
N: Noun and Noun Phrases
NN: noun-noun modifiers
P: Preposition and Preposition Phrases
PpSpec: Specifiers of Preposition Phrases (search for PpSpec
in data/wndict.lsp for instances)
A: Adjective/Adverbs
Have: have
Aux: Auxilary verbs, e.g. should, will, does, ...
Be: Different forms of be: is, am, were, be, ...
COMP: Complementizer
VBE: be used as a linking verb. E.g., I am hungry
V_N verbs with one argument (the subject), i.e., intransitive verbs
V_N_N verbs with two arguments, i.e., transitive verbs
V_N_I verbs taking small clause as complement

GRAMMATICAL RELATIONSHIPS

The following is a list of all the grammatical relationships in
Minipar. Search for (dep-type relation) in data/minipar.lsp for the
meaning of relation.

appo "ACME president, --appo-> P.W. Buckman"
aux "should <-aux-- resign"
be "is <-be-- sleeping"
c "that <-c-- John loves Mary"
comp1 first complement
det "the <-det `-- hat"
gen "Jane's <-gen-- uncle"
have "have <-have-- disappeared"
i the relationship between a C clause and its I clause
inv-aux inverted auxiliary: "Will <-inv-aux-- you stop it?
inv-be inverted be: "Is <-inv-be-- she sleeping"
inv-have inverted have: "Have <-inv-have-- you slept"
mod the relationship between a word and its adjunct modifier
pnmod post nominal modifier
p-spec specifier of prepositional phrases
pcomp-c clausal complement of prepositions
pcomp-n nominal complement of prepositions
post post determiner
pre pre determiner
pred predicate of a clause
rel relative clause
vrel passive verb modifier of nouns
wha, whn, whp: wh-elements at C-spec positions
obj object of verbs
obj2 second object of ditransitive verbs
subj subject of verbs
s surface subject

MINIPAR Homepage:
http://www.cs.ualberta.ca/~lindek/minipar.htm

(Information is from the README file of the program.)

MINIPAR is a broad-coverage parser for the English language. An
evaluation with the SUSANNE corpus shows that MINIPAR achieves about
88% precision and 80% recall with respect to dependency
relationships. MINIPAR is very efficient, on a Pentium II 300 with
128MB memory, it parses about 300 words per second.

MINIPAR Homepage:
http://www.cs.ualberta.ca/~lindek/minipar.htm

* System complexity and sophistication of inference did not correlate fully with performance.

* Naive lexically-based systems returned the best results.

For more information:
The PASCAL Recognising Textual Entailment Challenge: http://www.cs.biu.ac.il/~glikmao/rte05/dagan_et_al.pdf

- word overlap between T and H, possibly including stemming, lemmatization,
part of speech tagging, and applying statistical word weighting such as idf.
(for example using simple decision tree trained by the development set: accuracy = 56%)

- consider relationships between words that may reflect entailment, based either on statistical methods or WordNet.

- measure degree of match between T and H, based on some distance criteria.

- few systems incorporated some form of "world knowledge."

- some of them applyied a logical prover.

- other techniques used are:
- probabilistic models
- probabilistic Machine Translation models
- supervised learning methods
- logical inference
- various specific scoring mechanisms

For more information:
The PASCAL Recognising Textual Entailment Challenge: http://www.cs.biu.ac.il/~glikmao/rte05/dagan_et_al.pdf