CONTEXTUAL

VOCABULARY ACQUISITION:

A Computational Theory and Educational Curriculum

William J. Rapaport

SNePS Research Group (SNeRG) Dept. of Computer Science & Engineering

Department of Philosophy Center for Cognitive Science

SUNY Buffalo rapaport@cse.buffalo.edu

http://www.cse.buffalo.edu/∼rapaport/

Michael W. Kibby

Dept. of Learning & Instruction Center for Literacy & Reading Instruction

SUNY Buffalo mwkibby@acsu.buffalo.edu

http://www.gse.buffalo.edu/FAS/Kibby/

Karen Ehrlich,

Marc Broklawski, Karen Wieland

NSF ROLE Grant REC-0106338

A Computational Theory of Contextual Vocabulary Acquisition

Computational cognitive theory of how NLU system can learn word meanings …

• … from context:

=df grammatical info + text

+ background knowledge

• no external sources (human, on-line) – unavailable, incomplete, or misleading

• domain-independent

• ”definition” =

current hypothesis about word’s meaning - revised each time word is seen

MOTIVATIONS & APPLICATIONS

• part of cog sci projects:

– narrative text understanding

– syntactic semantics (contra Searle’s CRA)

• important for:

– message-processing systems

– intelligent agents ∗ there can be no complete lexicon

∗ such systems shouldn’t have to stop to ask questions

• other applications:

– L1 & L2 acquisition research

– computational lexicography

– ** education: teaching reading **

What does ‘brachet’ mean?

4

(From Malory’s Morte D’Arthur [page # in brackets])

1. ... there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them ... [66] 1. ... there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them ... [66]

2. ... as [the hart] went by the sideboard, the white brachet bit him .... [66]

5-a

1. ... there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them ... [66]

2. ... as [the hart] went by the sideboard, the white brachet bit him .... [66]

3. ... the knight arose, took up the brachet , ... and rode away with the brachet . [66] 1. ... there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them ... [66]

5-b

2. ... as [the hart] went by the sideboard, the white brachet bit him .... [66]

3. ... the knight arose, took up the brachet , ... and rode away with the brachet . [66]

4. ... a lady came in ... and cried aloud to King Arthur, “Sire, ... the brachet is mine ...” [66]

5-c

1. ... there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them ... [66]

2. ... as [the hart] went by the sideboard, the white brachet bit him .... [66]

3. ... the knight arose, took up the brachet , ... and rode away with the brachet . [66]

4. ... a lady came in ... and cried aloud to King Arthur, “Sire, ... the brachet is mine ...” [66]

10. ... there was the white brachet which bayed at him fast. [72]

5-d

1. ... there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them ... [66]

2. ... as [the hart] went by the sideboard, the white brachet bit him .... [66]

3. ... the knight arose, took up the brachet , ... and rode away with the brachet . [66]

4. ... a lady came in ... and cried aloud to King Arthur, “Sire, ... the brachet is mine ...” [66]

10. ... there was the white brachet which bayed at him fast. [72]

18. ... the hart lay dead ...; a brachet was biting on his throat, and other hounds came behind. [86]

5-e

DEMO:

Backgd info re: harts, animals, KA...; No info re: brachets I/P: formal-lang version of simplified Eng.

...there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them...

A hart runs into King Arthur’s hall.

• In the story, B17 is a hart.

• In the story, B17 runs into B18.

• In the story, B18 is a hall.

• In the story, B18 is KA’s hall.

A white brachet is next to the hart.

• In the story, B19 is a brachet.

• In the story, B19 has the prop “white”.

• Brachets ⊂ physical obj. (deduced since only phys objs have color)

--> (defn noun 'brachet)
((CLASS INCLUSION    = (PHYS OBJ))
 structure           = nil
 function            = nil
 actions             = (nil)
 ownership           = nil
 POSSIBLE PROPERTIES = ((WHITE))
 synonyms            = nil)

... The brachet bites the hart’s buttock.

--> (defn_noun 'brachet)
((CLASS INCLUSION    = (ANIMAL))
structure           = nil
function            = nil
ACTIONS =
 ((POSSIBLE ACTIONS = (BITE)))
ownership           = nil
POSSIBLE PROPERTIES = ((WHITE))
synonyms            = nil)

... The knight picks up the brachet . ... The knight carries the brachet .

--> (defn_noun 'brachet)
((CLASS INCLUSION    = (ANIMAL))
structure           = nil
function            = nil
ACTIONS =
 ((POSSIBLE ACTIONS = (BITE)))
ownership           = nil
POSSIBLE PROPERTIES = ((SMALL WHITE))
synonyms            = nil)

... The lady says that she wants the brachet .

--> (defn_noun 'brachet)
((CLASS INCLUSION    = (ANIMAL))
structure           = nil
function            = nil
ACTIONS =
 ((POSSIBLE ACTIONS = (BITE)))
ownership           = nil
POSSIBLE PROPERTIES = ((SMALL VALUABLE WHITE))
synonyms            = nil)

... The brachet bays in the direction of Sir Tor.

--> (defn_noun 'brachet)
((A BRACHET IS A KIND OF (DOG)
ACTIONS =
 ((POSSIBLE ACTIONS = (BAY BITE)))
FUNCTION            = ((HUNT))
structure           = nil
ownership           = nil
synonyms            = nil)

• SYSTEM’S BEHAVIOR ≈ HUMAN PROTOCOLS

• OED:

A brachet is “a kind of hound which hunts by scent”

A Computational Theory of Vocabulary Acquisition

• 3 kinds of words:

– unknown, misunderstood, new use (‘brachet’, ‘smite’, ‘dress’)

• Initial hypothesis

– “Tommy broke a vase.”

∗ ‘vase’ =_{df} something that Tommy broke;

∗ ‘vase’ =_{df} phys. obj. breakable by human

Revision(s) upon further encounter(s) Converges to dictionary-like def. Settles down, but subject to revision

• Implementation:

– SNePS + SNeBR + SNePSwD (+ GATN)

Long-term goal of the SNePS Research Group: Development of a Computational Cognitive Agent:

“Cassie”

• to understand intelligent cognitive processes

• by developing & experimenting with a computational cognitive agent

• who can use & understand NL

• who can reason & solve problems

• who can plan & execute actions, as well as reason about the plans The SNePS Knowledge Representation,

Reasoning, & Acting System

Stuart C. Shapiro & SNeRG

• KR:

– Propositional semantic network

(propositions are represented by nodes)

– Fully intensional KR

(∴ can handle non-referring terms)

• Reasoning:

– node-based reasoning (w/ rules)

– path-based reasoning (inheritance)

– belief revision (relevance logic)

• Acting:

– SNePS Rational Engine

• Interfaces:

– SNePSUL (Lisp-like)

– SNePSLOG (logic-like)

– Natural language understanding & generation

12 13 14 15

Related Work in AI/CL: Distinguish WSD (multiple-choice test) from CVA (essay question) CVA in AI/CL (highlights):

• Granger 1977: Foul-Up (script-based)

• Berwick 1983 (causal inferences, synonymy)

• Zernik & Dyer 85,87: Self-Extending Lexicon – L2 acquisition, figurative phrases, human informant

• Hastings et al. 91,94,96:

– KB = taxonomy

– tries to locate new word in taxonomy

– … “correctly”

• Siskind 92,96:

– model of L1 acquisition

– I/P = utterance + visual perception of situation + mental representation of situation + assumption that utterance means rep’n

– O/P = “correct word to meaning maps”

– uses semantics to learn syntax

(we use syntax to learn semantics)

• Ehrlich 95 @ Buffalo More complete list: http://www.cse.buffalo.edu/∼rapaport/refs.vocab.html

Related Work in Psychology:

• Elshout-Mohr & van Daalen-Kapteijns 81,87

– reader’s model of new word = updatable frame w/ slots & defaults

• Sternberg et al. 83,87

– cues to look for (= slots for frame): ∗ spatio/temporal cues

∗ value cues

∗ properties

∗ functions

∗ cause/enablement info

∗ class memberships

∗ synonyms/antonyms

• Johnson-Laird 87

– word understanding ≠ definition

– definitions aren’t stored Elshout-Mohr & van Daalen-Kapteijns:

• Experiments w/ neologisms in contexts

– When you are used to a view it is depressing when you live in a room with kolpers . (superordinate info)

– At home he had to work by artificial light because of those kolpers . (distinguishing info)

– I was afraid the room might have kolpers , but plenty of sunlight came into it. (counterex.)

* kolper: window transmitting little light because of something outside it.

• Good readers model new word on known word & look for differences.

• Model is a frame-like structure, with slots & defaults Sternberg:

The couple there on the blind date was not enjoying the festivities in the least. An acapnotic , he disliked her smoking; and when he removed his hat, she, who preferred “ageless” men, eyed his increasing phalacrosis and grimaced.

• To acquire new words from context: – distinguish relevant/irrelevant info

– selectively combine relevant info

– compare this info with previous beliefs

• and look for:

– spatio/temporal cues

– value cues

– properties

– functions

– cause/enablement info

– class memberships

– synonyms/antonyms Related Work in Education:

• VA in context ≠ VA from context!

• Seibert 45:

Guessing Word Meanings from Context • Ames 66,70:

Classification Scheme of Contextual Aids • Nagy et al. 84ff:

“incidental” CVA explains vocab growth • Schatz & Baldwin 86:

Context Clues Are Unreliable Predictors of Word Meanings

• Hulstijn 92: Inferred Word Meanings

• At UB:

Fisher 86, Friedland 92, Pullano 92, Kibby 95

Education, cont’d.

An “algorithm”(?) [Nation & Coady 88]:

“The following strategy is an elaboration of one described by Clarke & Nation 80. This strategy consists of five steps:

1. Finding the part of speech of the unknown word.

2. Looking at the immediate context of the unknown word and simplifying this context if necessary.

3. Looking at the wider context of the unknown word. This means looking at the relationship between the clause containing the unknown word and surrounding clauses and sentences.

4. Guessing the meaning of the unknown word.

5. Checking that the guess is correct.”

Our theory:

not “guessing”, not even (merely) “inferring”, but …

… computing the meaning of unknown words from context. here, need info re computational impln

22

[Background info re: harts, animals, KA, etc.— but no info re: brachets]

Right so as they sat, there came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them with a great cry. [p.66]

A hart runs into King Arthur’s hall.

;M233: In the story, B17 is a hart.
(M233! (KN_CAT story)
       (OBJECT1 B17)
       (OBJECT2 (M1 (LEX hart)))
       (REL ISA))
;M344: In the story, B17 runs into B18.
(M344! (ACT (M194 (LEX run)))
       (AGENT B17)
       (INTO B18)
       (KN_CAT story))
;M347: In the story, B18 is a hall.
(M347! (CLASS (M4 (LEX hall)))
       (KN_CAT story)
       (MEMBER B18))
;M350: In the story, the hall (B18) is a hall of its
;      possessor, King Arthur (B3).
(M350! (KN_CAT story)
       (OBJECT B18)
       (POSSESSOR B3)
       (REL (M4 (LEX hall))))

HERE PUT SNEPS NETS

24

A white brachet is next to the hart

;M352: In the story, B19 is a brachet.
(M352! (KN_CAT story)
       (OBJECT1 B19)
       (OBJECT2 B17)
       (REL (M354 (LEX next to))))
;M356: In the story,
;      the brachet has the property "white".
(M356! (KN_CAT story)
       (OBJECT B19)
       (PROPERTY (M97 (LEX white))))
;M358: Brachets are a subclass of physical object
;      (deduced since only physical objects
;      have color).
(M358! (SUBCLASS (M351 (LEX brachet)))
       (SUPERCLASS (M38 (LEX phys obj))))

--> (defn_noun 'brachet)
((CLASS INCLUSION= (phys obj))  STRUCTURE= NIL
FUNCTION= NIL ACTIONS= (NIL)  OWNERSHIP= NIL
POSSIBLE PROPERTIES= ((WHITE)) SYNONYMS= NIL)

• At this stage, Cassie says that ‘brachet’ means: “a physical obj that can be white”

• Because Cassie:

– believes ∃ a brachet (B19)

– believes B19 is white (*)

– has inferred (from (*)) that brachets are physical objs

• N.B.: The “definition” does not appear in the net!

Search Algorithms for Nouns & Verbs

• Algorithms search net for fillers for slots in definition frame

– search guided by desired slots

– e.g., ignores particular info if general info is present, else takes what it can get

• Each request for def starts search from scratch with larger context

– If queried at each occurrence, frame “develops” dynamically

– If queried only at end, then only “final” frame is O/P Noun search algorithm:

• Look for: – definitely-true general rules – presumably-true general rules – individual instances

• Infer class inclusions (via node-based inference)

– especially basic-level

– check for animacy

• Search (via path-based inference) for: – properties

– structure

– functions

– actions performed

– being object of an action

– ownership

• Infer “synonyms” (if class inclusions found)

• If no class, or too-high level, & no structure, functions, or acts, then look for noun as object, else as anything else.

Verb search algorithm: Find or infer information about:

• predicate structure – categorization of arguments/case frame

• results of <verb>ing:

– effects caused

– state changes that follow

• enabling conditions for <verb>

• classification of verb-type?

• synonyms?

3 Kinds of Vocabulary Acquisition:

• Construct new definition of unknown word (e.g., ‘brachet’)

• Fully revise definition of misunderstood word (e.g., ‘smite’)

• Expand definition of word used in new sense (e.g., ‘dress’)

Belief Revision:

SNeBR (Martins & Shapiro 88) SNePSwD (Martins & Cravo 91)

• If inference leads to a contradiction, then:

– SNeBR asks user to remove culprit(s)

– & automatically removes all inferred propositions

• Used to revise defs of words used differently from current meaning hypothesis.

• Removal & revision being automated via SNePSwD by ranking all propositions with

  kn cat:

most  intrinsic:  info re: lang; fund. bkgd info
certain  (“before” is transitive)

|

| story: info in text
|  (King Lot rode to town)

|

| life: bkgd info w/o vars or inf
|  (dogs are animals)

|

| story-comp: info inferred from text
|  (King Lot is a king, rode on a horse)

|

| life-rule1: everyday CS bkgd info
|  (x bears young → x is a mammal”)

|

| life-rule2: specialized bkgd info
|  (x smites y → x kills y by hitting y)

V

least  questionable: already-revised life-rule2;

certain  not part of I/P

Belief revision: ‘smite’

• misunderstood word; 2-stage “subtractive” revision

[background info includes: (*) smite(x, y, t) →

hit(x, y, t) & dead(y, t) & cause(hit(x, y, t),dead(y, t))]

P1: King Lot smote down King Arthur D1: If person x smites person y at time t,

then x hits y at t, and y is dead at t Q1: What properties does KA have? R1: King Arthur is dead. P2: King Arthur drew Excalibur Q2: When did King Arthur do this?

SNeBR is invoked:

• KA’s drawing E is incons w/ being dead

• (*) replaced w/: – smiting only poss’ly entails killing

– if smiting results in death, then the hitting caused the death

D2: If person x smites person y at time t,

then x hits y at t, & ⋄(y is dead at t) P3: [another passage in which smiting ↛ death] D3: If person x smites person y at time t,

then x hits y at t.

Belief revision: ‘dress’

• additive revision

[background info includes:

(1) dresses(x, y) → ∃z[clothing(z) & wears(y, z)

(2) spears don’t wear clothing (both kn cat=life rule1)]

P1: King Arthur dressed himself D1: A person can dress itself; result: it wears clothing P2: King Claudius dressed his spear [Cassie infers: KC’s spear wears clothing] Q2: What wears clothing?

SNeBR invoked:

• Contra!

• (1) replaced w/: dresses(x, y) → ∃z[...] ∨ S

• replace (1), not (2), bec. of verb in antecedent (Gentner)

P3: [passages in wh. dressing spears precedes fighting] D3: A person can dress a spear or a person;

result: person wears clothing or person is enabled to fight

CVA Pilot Project:

Development of Computational Theory & Educational Curriculum

2-way interaction between CS & Ed “teams”:

• CS team:

– extend & develop algorithms for computational CVA (N,V,Adj)

– translate SNePS algorithms into English; Ed team converts them into curriculum

∗ for enhancing middle-school/college students’ abilities to use deliberate CVA strategies in reading STEM texts

∗ more explicit than Nation & Coady; more realistic than Huntington Learning Center:

Does Your Child Need a Vocabulary Boost? Huntington Can Help!

Huntington Learning Center offers a program called Practicing Vocabulary in Context which teaches the student to use the context of a sentence or paragraph to determine the meaning of a word. The student reads the story out loud and is asked to think about the meaning of the underlined words.

• Ed team:

– formal oral protocols with passages …

∗ that Cassie already handles:

cf. to Cassie’s results; use for algorithm improvement

∗ that KR students are coding:

use for algorithm improvement

∗ that are new: use to help code new passages

• both teams: – use new passages

– unify the psych, L1, L2, reading, AI/CL literatures ”proliferate” ”sedative” ”oam” ”taciturn” ”proximity” ”estuary”

A decade ago research on lab animals revealed that stem cells taken from animal embryos are astoundingly versatile. They grow in the lab, PROLIFERATE like rabbits and turn into specialized cells such as neurons.

Adult stem cells seem to PROLIFERATE more slowly than the embryonic ones, and so might not provide an ever-renewing source of new cells to replace those lost to, say, arthritis.

Relief is now available for the anxiety many children suffer when undergoing difficult medical procedures or facing surgery. The U.S. Food and Drug Administration (FDA) granted marketing clearance today to a SEDATIVE for use in children.

The medication, Versed..., can be used to calm and SEDATE children who must undergo procedures such as bone-marrow aspiration or spinal tap. Sedating a child also may spare parents the trauma of watching their child being physically restrained during procedures. Without SEDATION, such restraint is often necessary, and parents may even participate in restraining their child, which can be very distressing. Versed also can be used to help calm children facing surgery, who might otherwise be terrified by the mask used to administer anesthesia, or by separation from their parents.

Versed now is also available for use in infants and children of all ages in critical care settings. This is the first time a SEDATIVE has been approved for use of any kind in newborns. Without SEDATION, babies and children may have a difficult time tolerating a breathing tube or ventilator – and ”fighting” a ventilator can be life-threatening. [ETC.]

Two ill-dressed people ... sat around a fire where the common meal was almost ready. The mother ... peered at her son through the OAM of the bubbling stew.

Unlike his brothers, who were noisy, outgoing, and very talkative, Fred was quite TACITURN.

Galileo recently flew just 120 miles (200 kilometers) above Europa, a PROXIMITY which allowed the spacecraft to take the most detailed pictures ever of the mysterious satellite. [ETC.] An ESTUARY is a coastal area where fresh water from rivers and streams mixes with salt water from the ocean. Many bays, sounds, and lagoons are ESTUARIES. ESTUARIES provide safe spawning grounds and nurseries and are critical for fish, birds, and other wildlife.

How should the passage be represented?

• Correctly?

– Long-term goal

• To work with the Ehrlich algorithm? – Short-term goal How much background knowledge is needed?

• ∀ known concept, C, in the passage, provide at least 2 meaning postulates:

1. C → N

2. S → C

• The unknown concept (X) will be linked to some known concept C, thereby linking X to N & to S.

• But defining ‘coastal’ only in terms of ‘coast’ is not suff;

– need also to define ‘coast’.

How much background knowledge do kids need?

• Cassie can do inferences ”immediately”:

– If she knows that:

– x is white

– white is a color

– colored things are physical objects,

she immediately/automatically infers:

x is a physical object

• But: kids may need to be led through them:

– Need to be ”told” that x is white

– Need to be ”told” that white is a color

– Need to be ”told” that colored things are physical objects

– Need to be ”told” to infer that x is a physical object

How much background knowledge do people need?

• Cassie can do inferences ”immediately”:

– If she knows that:

∗ only small objects can be picked up ∗ the knight picks up the brachet

she immediately/automatically infers:

∗ the brachet is small

• But: people may need to be led through them:

– Need to be ”told” that only small objects can be picked up.

– Need to be ”told” to infer that the brachet is small.

WHAT IS A.I.?

• “The science of making machines do things that would require intelligence if done by humans.” – Marvin Minsky

Using humans to tell us how to program computers.

• “The use of computer programs and programming techniques to cast light on the principles of intelligence in general and human thought in particular.”

– Margaret Boden

Using computers to tell us something about humans.

• In fact: it’s both—a 2-way street Kibby:

We are teaching a machine, to see if what we learn in teaching it can help us teach students better.

• cf. Boden’s def of AI

Web Sites

– all at http://www.cse.buffalo.edu/…

• Bibliography of

Contextual Vocabulary Acquisition:

… ∼rapaport/refs.vocab.html • SNePS Research Group (SNeRG):

… sneps

• CVA:

… ∼rapaport/cva.html