git @ Cat's Eye Technologies Tamsin / 1fa7832
Compiler supports escape sequences. Tweak README, add examples. Cat's Eye Technologies 10 years ago
7 changed file(s) with 103 addition(s) and 82 deletion(s). Raw diff Collapse all Expand all
+44
-53
README.markdown less more
3232 | main = 'Hello, world!'.
3333 = Hello, world!
3434
35 Copy input to output, not unlike unix `cat`.
36
37 | main = T ← '' & {any → S & T ← T + S} & T.
38 + This file
39 + gets catted.
40 = This file
41 = gets catted.
42
43 Parse an algebraic expression for correctness.
44
45 | main = (expr0 & eof & 'ok').
46 | expr0 = expr1 & {"+" & expr1}.
47 | expr1 = term & {"*" & term}.
48 | term = "x" | "y" | "z" | "(" & expr0 & ")".
49 + x+y*(z+x+y)
50 = ok
51
52 | main = (expr0 & eof & 'ok').
53 | expr0 = expr1 & {"+" & expr1}.
54 | expr1 = term & {"*" & term}.
55 | term = "x" | "y" | "z" | "(" & expr0 & ")".
56 + x+xy
57 ? expected EOF found 'y'
58
59 Parse an algebraic expression to a syntax tree.
60
61 | main = expr0.
62 | expr0 = expr1 → E1 & {"+" & expr1 → E2 & E1 ← add(E1,E2)} & E1.
63 | expr1 = term → E1 & {"*" & term → E2 & E1 ← mul(E1,E2)} & E1.
64 | term = "x" | "y" | "z" | "(" & expr0 → E & ")" & E.
65 + x+y*(z+x+y)
66 = add(x, mul(y, add(add(z, x), y)))
67
68 Translate an algebraic expression to RPN (Reverse Polish Notation).
69
70 | main = expr0 → E & walk(E).
71 | expr0 = expr1 → E1 & {"+" & expr1 → E2 & E1 ← add(E1,E2)} & E1.
72 | expr1 = term → E1 & {"*" & term → E2 & E1 ← mul(E1,E2)} & E1.
73 | term = "x" | "y" | "z" | "(" & expr0 → E & ")" & E.
74 | walk(add(L,R)) = walk(L) → LS & walk(R) → RS & return LS+RS+' +'.
75 | walk(mul(L,R)) = walk(L) → LS & walk(R) → RS & return LS+RS+' *'.
76 | walk(X) = return ' '+X.
77 + x+y*(z+x+y)
78 = x y z x + y + * +
79
80 Make a story more exciting — a one-liner!
35 Make a story more exciting!
8136
8237 | main = S ← '' & {("." & '!' | "?" & '?!' | any) → C & S ← S + C} & S.
8338 + Chapter 1
9045 = It was raining! She knocked on the door! She heard
9146 = footsteps inside! The door opened! The butler peered
9247 = out! "Hello," she said! "May I come in?!"
48
49 Parse an algebraic expression for syntactic correctness.
50
51 | main = (expr0 & eof & 'ok').
52 | expr0 = expr1 & {"+" & expr1}.
53 | expr1 = term & {"*" & term}.
54 | term = "x" | "y" | "z" | "(" & expr0 & ")".
55 + x+y*(z+x+y)
56 = ok
57
58 Parse an algebraic expression to a syntax tree.
59
60 | main = expr0.
61 | expr0 = expr1 → E1 & {"+" & expr1 → E2 & E1 ← add(E1,E2)} & E1.
62 | expr1 = term → E1 & {"*" & term → E2 & E1 ← mul(E1,E2)} & E1.
63 | term = "x" | "y" | "z" | "(" & expr0 → E & ")" & E.
64 + x+y*(z+x+y)
65 = add(x, mul(y, add(add(z, x), y)))
66
67 Translate an algebraic expression to RPN (Reverse Polish Notation).
68
69 | main = expr0 → E & walk(E).
70 | expr0 = expr1 → E1 & {"+" & expr1 → E2 & E1 ← add(E1,E2)} & E1.
71 | expr1 = term → E1 & {"*" & term → E2 & E1 ← mul(E1,E2)} & E1.
72 | term = "x" | "y" | "z" | "(" & expr0 → E & ")" & E.
73 | walk(add(L,R)) = walk(L) → LS & walk(R) → RS & return LS+RS+' +'.
74 | walk(mul(L,R)) = walk(L) → LS & walk(R) → RS & return LS+RS+' *'.
75 | walk(X) = return ' '+X.
76 + x+y*(z+x+y)
77 = x y z x + y + * +
9378
9479 Reverse a list.
9580
127112 | extract(lines(Lines, Line)) = extract(Lines) & extract_field(Line).
128113 | extract(L) = L.
129114 | extract_field(fields(Last, fields(This, X))) = print This.
115 | extract_field(X) = return X.
130116 + Harold,1850,"21 Baxter Street",burgundy
131117 + Smythe,1833,"31 Little Street, St. James",mauve
132118 + Jones,1791,"41 ""The Gardens""",crimson
151137 Quick Start
152138 -----------
153139
140 This repository contains the reference implementation of Tamsin, written in
141 Python. It can interpret a Tamsin program, and compile a program written
142 in (core) Tamsin to C. To start using it,
143
154144 * Install [toolshelf](https://github.com/catseye/toolshelf).
155145 * `toolshelf dock gh:catseye/tamsin`
156146
157147 Or just clone this repo and make a symbolic link to `bin/tamsin` somewhere
158148 on your path (or alter your path to contain the `bin/` directory of this repo.)
149
150 Then you can run `tamsin` like so:
151
152 * `tamsin run eg/csv_extract.tamsin < eg/names.csv`
159153
160154 Design Goals
161155 ------------
167161 parsing techniques.
168162 * Provide means to solve practical problems.
169163 * Keep the language simple (grammar should fit on a page)
170 * Have a relatively simple reference implementation (currently ~1000 lines
171 of Python, not counting debugging).
164 * Have a relatively simple reference implementation (currently ~1700 lines,
165 including everything — debugging and the C runtime used by the compiler.)
172166
173167 License
174168 -------
185179 * `bin/tamsin runast astfile.txt` -- for testing the meta-circular parser
186180 * `bin/tamsin runscan scanfile.txt` -- for testing the meta-circular scanner
187181 * `bin/tamsin scan file.tamsin` -- to generate a scanfile
188
189 ### compiler ###
190
191 * refactor analyzer to find locals for each branch-prod
192 * handle evaluation
182 * refactor nastier bits of the compiler
193183
194184 ### document ###
195185
216206 expr1 = term → E & fold ("*" & term) E mul.
217207 term = "x" | "y" | "z" | "(" & expr0 → E & ")" & E.
218208
209 * auto-generate terms from productions, like Rooibos does
219210 * `;` = `&`?
220211 * pretty-print AST for error messages
221212 * have analyzer, interpreter, compiler all inherit from ASTWalker or smth?
+0
-24
doc/Advanced_Features.markdown less more
33 This document is a **work in progress**.
44
55 -> Tests for functionality "Intepret Tamsin program"
6
7 Escape Characters
8 -----------------
9
10 A literal string may contain escape sequences. Note, I hate escape sequences!
11 So I might not leave this feature in, or, at least, not quite like this.
12
13 | main = "a" & "\"" & "b" & return 'don\'t'.
14 + a"b
15 = don't
16
17 | main = "a" & "\\" & "b" & return 'don\\t'.
18 + a\b
19 = don\t
20
21 | main = "a" & "\n" & "b" & return 'don\nt'.
22 + a
23 + b
24 = don
25 = t
26
27 | main = "a" & "\t" & "b" & return 'don\tt'.
28 + a b
29 = don t
306
317 Advanced Scanning
328 -----------------
+23
-0
doc/Tamsin.markdown less more
426426 | main = this_prod.
427427 | this_prod = set Var_name = this_atom & return Var_name.
428428 = this_atom
429
430 ### Escape Sequences ###
431
432 A literal string may contain escape sequences. Note, I hate escape sequences!
433 So I might not leave this feature in, or, at least, not quite like this.
434
435 | main = "a" & "\"" & "b" & return 'don\'t'.
436 + a"b
437 = don't
438
439 | main = "a" & "\\" & "b" & return 'don\\t'.
440 + a\b
441 = don\t
442
443 | main = "a" & "\n" & "b" & return 'don\nt'.
444 + a
445 + b
446 = don
447 = t
448
449 | main = "a" & "\t" & "b" & return 'don\tt'.
450 + a b
451 = don t
429452
430453 ### Examples using Terms ###
431454
+11
-0
eg/csv_extract.tamsin less more
0 main = line → L & L ← lines(nil, L) &
1 {"\n" & line → M & L ← lines(L, M)} & extract(L) & ''.
2 line = field → F & {"," & field → G & F ← fields(G, F)} & F.
3 field = strings | bare.
4 strings = string → T & {string → S & T ← T + '"' + S} & T.
5 string = "\"" & T ← '' & {!"\"" & any → S & T ← T + S} & "\"" & T.
6 bare = T ← '' & {!(","|"\n") & any → S & T ← T + S} & T.
7 extract(lines(Lines, Line)) = extract(Lines) & extract_field(Line).
8 extract(L) = L.
9 extract_field(fields(Last, fields(This, X))) = print This.
10 extract_field(X) = return X.⏎
+3
-0
eg/names.csv less more
0 Harold,1850,"21 Baxter Street",burgundy
1 Smythe,1833,"31 Little Street, St. James",mauve
2 Jones,1791,"41 ""The Gardens""",crimson
+18
-4
src/tamsin/compiler.py less more
0 # COMPLETELY EXPERIMENTAL.
10 # encoding: UTF-8
21
3 # spits out some kind of code based on a Tamsin AST.
4 # certainly does not support `using` or `@` at the moment.
2 # Copyright (c)2014 Chris Pressey, Cat's Eye Technologies.
3 # Distributed under a BSD-style license; see LICENSE for more information.
4
5 # Generates a C-language program which, when linked with -ltamsin, has
6 # the same (we hope) behaviour as interpreting the input Tamsin program.
7 # Does not support `using` or `@` at the moment.
58
69 from tamsin.term import Term, Variable, Concat
710
340343 else:
341344 self.emit('struct term *%s = %s;' % (name, term.name))
342345 else:
343 self.emit('struct term *%s = term_new("%s");' % (name, term.name))
346 self.emit('struct term *%s = term_new("%s");' %
347 (name, escaped(term.name))
348 )
344349 i = 0
345350 # TODO: reversed() is provisional
346351 for subterm in reversed(term.contents):
348353 i += 1
349354 self.emit_term(subterm, subname, pattern=pattern);
350355 self.emit("term_add_subterm(%s, %s);" % (name, subname))
356
357 def escaped(s):
358 escaped_name = s
359 escaped_name = escaped_name.replace("\\", r"\\")
360 escaped_name = escaped_name.replace("\n", r"\n")
361 escaped_name = escaped_name.replace("\t", r"\t")
362 escaped_name = escaped_name.replace('"', r'\"')
363 return escaped_name
364
+4
-1
test.sh less more
1111
1212 if [ x$1 = xcompiler ]; then
1313 ./build.sh
14 falderal --substring-error fixture/compiler.py.markdown doc/Tamsin.markdown
14 FILES="doc/Tamsin.markdown"
15 FILES="doc/Tamsin.markdown README.markdown"
16 #FILES="doc/Tamsin.markdown doc/Case_Study.markdown README.markdown"
17 falderal --substring-error fixture/compiler.py.markdown $FILES
1518 else
1619 falderal --substring-error fixture/tamsin.py.markdown $FILES
1720 fi