Skip to content

Commit 828139a

Browse files
vagmcsvagmcs
committed
structure learning examples documentation
1 parent 2a34028 commit 828139a

File tree

2 files changed

+304
-1
lines changed

2 files changed

+304
-1
lines changed

‎doc/4_1_structure_learning_examples.md

+302-1
Original file line numberDiff line numberDiff line change
@@ -1 +1,302 @@
1-
TODO
1+
## Online Structure Learning Examples
2+
3+
Below we provide online structure learning examples for OSL and OSLa, using the LoMRF structure learning command-line tool:
4+
5+
## OSL in Natural Language Processing
6+
7+
We would like to perform an information extraction task, called field segmentation, a real-world problem where the data
8+
contain many, relatively small, structured examples in the form of short documents. A document is represented as sequence
9+
of tokens, and the goal is to segment the text into fields, i.e. label each token in the document with a particular field.
10+
We use the CiteSeer dataset that contains 1563 bibliographic citations. The dataset has four disjoint subsets consisting
11+
of citations in four different research areas. The task is to segment each citation into three fields: Author, Title and Venue.
12+
13+
### Knowledge base (empty.mln)
14+
15+
We begin from an empty knowledge base that specify only the predicate schema.
16+
17+
*Predicate schema:*
18+
```lang-none
19+
// Predicate definitions
20+
Token(token,position,bib)
21+
FollowBy(bib,position,token)
22+
Next(position,position)
23+
LessThan(position,position)
24+
IsAlphaChar(token)
25+
IsDate(token)
26+
IsDigit(token)
27+
FirstNonAuthorTitleTkn(bib,position)
28+
FirstIn(bib,position)
29+
LastInitial(bib,position)
30+
Center(bib,position)
31+
HasPunc(bib,position)
32+
HasComma(bib,position)
33+
34+
// Non-Evidence
35+
InField(bib,field,position)
36+
```
37+
38+
### Mode Declarations (citeceer.modes)
39+
40+
We define a set of mode declarations, a form of language bias, to constrain the
41+
structure of clauses learned. Each mode definitions begins either with the symbol `modeP`
42+
or `modeF` to specify whether it refers to a predicate or function. In this example we
43+
only have predicates and therefore function mode declarations are not required. Each mode
44+
declaration has a recall number (1st argument), that restricts the number of times the
45+
specific predicate can appear in a clause, and a predicate symbol having as arguments
46+
placemarkers. Placemarkers can be input `+`, output `-`, or ignore `.`.Finally, symbol
47+
`#` specifies that this argument will remain constant after varabilization.
48+
49+
*Mode declarations:*
50+
```lang-none
51+
modeP(2, Token(#-,+,.))
52+
modeP(2, FollowBy(.,+,#.))
53+
modeP(1, Next(-,-))
54+
modeP(0, LessThan(-,-))
55+
modeP(2, IsAlphaChar(#+))
56+
modeP(2, IsDate(#+))
57+
modeP(2, IsDigit(#+))
58+
modeP(2, FirstNonAuthorTitleTkn(.,+))
59+
modeP(2, FirstIn(.,+))
60+
modeP(2, LastInitial(.,+))
61+
modeP(2, Center(.,+))
62+
modeP(2, HasPunc(.,+))
63+
modeP(2, HasComma(.,+))
64+
modeP(2, InField(.,#.,+))
65+
```
66+
67+
### Training micro-batches
68+
69+
The training data is composed of ground facts of the input predicates `Token/3`, `IsDigit/1`, `IsAlphaChar/1`, `IsDate/1`, `FirstNonAuthorTitleTkn/2`, `FollowBy/3`
70+
`FirstIn/2`, `LastInitial/2`, `Center/2`, `HasPunc/2`, `HasComma/2`, `Next/2` and ground facts of the annotation predicates `InField/3`.
71+
72+
```lang-none
73+
IsDigit(T0044)
74+
IsDigit(T01)
75+
IsAlphaChar(Tg)
76+
IsAlphaChar(Th)
77+
IsDate(Tjanuary)
78+
IsDate(Tjuly)
79+
//
80+
// ... sequence of facts ...
81+
//
82+
Next(P01,P00)
83+
Next(P02,P01)
84+
//
85+
// ... sequence of facts ...
86+
//
87+
Token(Tj,P00,B0013)
88+
FollowBy(B0013,P00,TPERIOD)
89+
InField(B0013,Fauthor,P00)
90+
Token(Tjaffar,P01,B0013)
91+
FollowBy(B0013,P01,TCOMMA)
92+
HasPunc(B0013,P01)
93+
HasComma(B0013,P01)
94+
//
95+
// ... sequence of facts ...
96+
//
97+
LastInitial(B0013,P00)
98+
FirstIn(B0013,P08)
99+
FirstNonAuthorTitleTkn(B0013,P19)
100+
Center(B0013,P04)
101+
```
102+
103+
### Structure Learning
104+
105+
The files of this example are the following:
106+
* Initially empty MLN file: [empty.mln](/Examples/Structure_Learning/OSL_NLP/empty.mln)
107+
* Mode declaration file: [citeceer.mln](/Examples/Structure_Learning/OSL_NLP/citeceer.mln)
108+
* Training files for online learning: [micro-batches](/Examples/Structure_Learning/OSL_NLP/training/)
109+
110+
Parameters:
111+
* Non-evidence predicates: `InField/3`
112+
* Max clause length: `12`
113+
* AdaGrad learning rate: `0.001`
114+
* Evaluation threshold: `2`
115+
116+
***OSL Learning***
117+
118+
```lang-none
119+
lomrf-slearn -i empty.mln -t ./training/ -o learned.mln -m citeceer.modes -ne InField/3 -maxLength 12 -lambda 0.001 -threshold 2
120+
```
121+
122+
## OSLa in Activity Recognition
123+
124+
In this example we demonstrate how to perform structure learning for activity recognition, using a fragment of the first
125+
set of the [CAVIAR dataset](http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/). We use the same Probabilistic Event Calculus
126+
formalism as presented in the [Quick Start](0_quick_start.md) section and an empty knowledge base having only the Probabilistic
127+
Event Calculus axioms.
128+
129+
### Knowledge base (empty.mln)
130+
131+
We begin from an empty knowledge base that specify only the predicate and function schema, as well as the Probabilistic
132+
Event Calculus axioms. We present below the initial knowledge base for the `meet` fluent.
133+
134+
```lang-none
135+
// ----------------------------------------------------------------------------
136+
// --- Domain definitions
137+
//
138+
// time: the domain of time-points
139+
// fluent: the domain of fluent, that is composite events (CE)
140+
// event: the domain of events, that is the input simple, derived events (SDE)
141+
// id: domain entities, e.g. persons or objects
142+
// ----------------------------------------------------------------------------
143+
144+
// Query Predicates:
145+
HoldsAt(fluent, time)
146+
147+
// Template Predicates:
148+
InitiatedAt(fluent, time)
149+
TerminatedAt(fluent, time)
150+
151+
// Evidence Predicates (closed-world assumption):
152+
Happens(event, time)
153+
Close(id, id, distance, time)
154+
OrientationMove(id, id, time)
155+
Next(time, time)
156+
StartTime(time)
157+
158+
// Function definitions for simple, derived events (SDE):
159+
event enter(id)
160+
event exit(id)
161+
event walking(id)
162+
event running(id)
163+
event active(id)
164+
event inactive(id)
165+
166+
// Function definitions for composite events (CE):
167+
fluent meet(id, id)
168+
169+
// ----------------------------------------------------------------------------
170+
// --- Probabilistic Event Calculus (domain-independent axioms)
171+
// ----------------------------------------------------------------------------
172+
173+
// When a fluent holds:
174+
Next(t1, t0) ^ InitiatedAt(f, t0) => HoldsAt(f, t1)
175+
176+
// When a fluent does not hold:
177+
Next(t1, t0) ^ TerminatedAt(f, t0) => !HoldsAt(f, t1)
178+
179+
// The inertia of holdsAt, i.e. a fluent continues to hold unless it is terminated:
180+
Next(t1, t0) ^ HoldsAt(f, t0) ^ !TerminatedAt(f, t0) => HoldsAt(f, t1).
181+
182+
// The inertia of !holdsAt, i.e. a fluent continues not to hold unless it is initiated:
183+
Next(t1, t0) ^ !HoldsAt(f, t0) ^ !InitiatedAt(f, t0) => !HoldsAt(f, t1).
184+
185+
// ----------------------------------------------------------------------------
186+
// --- Other domain-dependent rules
187+
// ----------------------------------------------------------------------------
188+
189+
// --- The meeting CE cannot performed by a single person:
190+
!HoldsAt(meet(p, p), t).
191+
192+
// --- Initially nothing holds
193+
StartTime(t) => !HoldsAt(f, t).
194+
```
195+
196+
### Mode Declarations (citeceer.modes)
197+
198+
Similar to OSL, we define a set of mode declarations, a form of language bias, to constrain the
199+
structure of clauses learned. In this case we also have mode declarations for the functions.
200+
201+
*Mode declarations:*
202+
```lang-none
203+
// Modes declarations for predicates
204+
modeP(2, Happens(-,+))
205+
modeP(0, HoldsAt(+,+))
206+
modeP(0, Next(-,+))
207+
modeP(0, OrientationMove(.,.,+))
208+
modeP(0, StartTime(+))
209+
modeP(1, Close(.,.,#.,+))
210+
211+
// Mode declarations for functions
212+
modeF(2, inactive(.))
213+
modeF(2, walking(.))
214+
modeF(2, active(.))
215+
modeF(0, running(.))
216+
modeF(1, meet(.,.))
217+
modeF(2, enter(.))
218+
modeF(2, exit(.))
219+
```
220+
221+
###Training data
222+
223+
The training micro-batches are composed of ground facts of the input predicates `StartTime/1`, `Happens/2`, `Close/4`, `OrientationMove/3`, `Next/2`
224+
ground facts of the annotation predicates `HoldsAt/2`, as well as ground function mappings. For example, consider the following fragment:
225+
226+
```lang-none
227+
// Input events:
228+
Enter_ID0 = enter(ID0)
229+
Enter_ID1 = enter(ID1)
230+
Exit_ID0 = exit(ID0)
231+
Exit_ID1 = exit(ID1)
232+
Walking_ID0 = walking(ID0)
233+
Walking_ID1 = walking(ID1)
234+
Running_ID0 = running(ID0)
235+
Running_ID1 = running(ID1)
236+
Active_ID0 = active(ID0)
237+
Active_ID1 = active(ID1)
238+
Inactive_ID0 = inactive(ID0)
239+
Inactive_ID1 = inactive(ID1)
240+
241+
// Output composite events (fluents):
242+
Move_ID0_ID0 = move(ID0, ID0)
243+
Move_ID0_ID1 = move(ID0, ID1)
244+
Move_ID1_ID0 = move(ID1, ID0)
245+
Move_ID1_ID1 = move(ID1, ID1)
246+
247+
Meet_ID0_ID0 = meet(ID0, ID0)
248+
Meet_ID0_ID1 = meet(ID0, ID1)
249+
Meet_ID1_ID0 = meet(ID1, ID0)
250+
Meet_ID1_ID1 = meet(ID1, ID1)
251+
252+
// Facts
253+
StartTime(0)
254+
255+
HoldsAt(Meet_ID0_ID1, 174)
256+
HoldsAt(Meet_ID0_ID1, 175)
257+
HoldsAt(Meet_ID0_ID1, 176)
258+
//
259+
// ... sequence of facts ...
260+
//
261+
Happens(Walking_ID0, 100)
262+
Happens(Walking_ID1, 100)
263+
OrientationMove(ID0, ID1, 100)
264+
Close(ID0, ID1, 34, 100)
265+
//
266+
// ... sequence of facts ...
267+
//
268+
Happens(Active_ID0, 170)
269+
Happens(Active_ID1, 170)
270+
//
271+
// ... sequence of facts ...
272+
```
273+
274+
### Weight Learning
275+
276+
The files of this example are the following:
277+
* Knowledge base files:
278+
* Main MLN file in CNF: [theory_cnf.mln](/Examples/Weight_Learning/Activity_Recognition/theory.mln)
279+
* Definitions of moving activity: [definitions/moving.mln](/Examples/Weight_Learning/Activity_Recognition/definitions/moving.mln)
280+
* Definitions of meeting activity: [definitions/meeting.mln](/Examples/Weight_Learning/Activity_Recognition/definitions/meeting.mln)
281+
* Training file for batch learning: [training.db](/Examples/Weight_Learning/Activity_Recognition/training/batch/training.db)
282+
* Training files for online learning: [micro-batches](/Examples/Weight_Learning/Activity_Recognition/training/online/)
283+
284+
285+
### Structure Learning
286+
287+
The files of this example are the following:
288+
* Initially empty MLN file: [meet.mln](/Examples/Structure_Learning/OSLa_CAVIAR/meet.mln)
289+
* Mode declaration file: [meet.modes](/Examples/Structure_Learning/OSLa_CAVIAR/meet.modes)
290+
* Training files for online learning: [micro-batches](/Examples/Structure_Learning/OSLa_CAVIAR/training/meet)
291+
292+
Parameters:
293+
* Non-evidence predicates: `HoldsAt/2`
294+
* Template predicates: `InitiatedAt/2`, `TerminatedAt/2`
295+
* Max clause length: `8`
296+
* Evaluation threshold: `1`
297+
298+
***OSLa Learning***
299+
300+
```lang-none
301+
lomrf-slearn -i meet.mln -o learned_meet.mln -t ./training/meet -m meet.modes -ne HoldsAt/2 -template InitiatedAt/2,TerminatedAt/2 -maxLength 8 -threshold 1
302+
```

‎doc/4_structure_learning.md

+2
Original file line numberDiff line numberDiff line change
@@ -95,3 +95,5 @@ literal become unit clauses with positive literal and inverted sign in their cor
9595
## References
9696

9797
* Tuyen N. Huynh and Raymond J. Mooney. Online Structure Learning for Markov Logic Networks (2011). In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2011), Vol. 2, pp. 81-96. ([link](http://www.cs.utexas.edu/users/ai-lab/?huynh:ecml11))
98+
99+
* Michelioudakis E., Skarlatidis A., Paliouras G. and Artikis A. OSLa: Online Structure Learning using Background Knowledge Axiomatization. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2016).

0 commit comments

Comments
 (0)