git @ Cat's Eye Technologies Falderal / 31694d9
Move obsolete Test.Falderal docs out of the way. Tweaks to others. --HG-- rename : doc/Quick_Start.markdown => impl/Test.Falderal/doc/Quick_Start.markdown rename : doc/Theory_of_Operation.markdown => impl/Test.Falderal/doc/Theory_of_Operation.markdown Cat's Eye Technologies 11 years ago
6 changed file(s) with 145 addition(s) and 136 deletion(s). Raw diff Collapse all Expand all
+13
-12
README.markdown less more
2626 You could write a bunch of standalone test sources, and store the output you
2727 expect from them in a bunch of other files, and write a shell script that runs
2828 each program and `diff`s the output with the expected output. But this is a
29 lot of clutter -- finding a particular example might not be so easy. Each
29 lot of clutter — finding a particular example might not be so easy. Each
3030 test source exists in a void, not necessarily logically grouped with other,
3131 similar tests. And any text you write describing a test needs to be in the
3232 comment syntax of your programming language (if your programming language
9393
9494 This distribution contains:
9595
96 * `doc` -- contains documents about Falderal. For the specification of
96 * `doc` — contains documents about Falderal. For the specification of
9797 the file format, see `doc/Falderal_Literate_Test_Format.markdown`.
9898 (Note that this specification should not be expected to remain stable
9999 through the 0.x version series.) There are other documents in there too.
100 * `bin/falderal` -- the reference implementation of Falderal, written in
100 * `bin/falderal` — the reference implementation of Falderal, written in
101101 Python and sometimes referred to as "py-falderal". It imports the
102102 sources in `src/falderal`. You don't need to install it; just add
103 the `bin` directory of this distribution to your `$PATH`.
104 * `impl/Test.Falderal` -s a (lagging, and not conformant) implementation of
103 the `bin` directory of this distribution to your `$PATH`. This
104 implementation is (somewhat) documented in `doc/py-falderal.markdown`.
105 * `impl/Test.Falderal` — a (lagging, and not conformant) implementation of
105106 Falderal in Haskell.
106 * `tests` -- a set of tests for Falderal itself. (Note that these are not
107 * `tests` — a set of tests for Falderal itself. (Note that these are not
107108 written in Falderal, as that would just be too confusing.)
108 * `HISTORY.markdown` -- changelog for releases of Falderal.
109 * `TODO.markdown` -- areas where Falderal and its implementations could be
109 * `HISTORY.markdown` — changelog for releases of Falderal.
110 * `TODO.markdown` — areas where Falderal and its implementations could be
110111 improved.
111112
112113 Development
117118 [git mirror of the repository on Github](https://github.com/catseye/Falderal).
118119
119120 Official release distfiles are available on the
120 [Falderal project page](http://catseye.tc/projects/falderal/) at
121 [Falderal project page](http://catseye.tc/node/Falderal) at
121122 [Cat's Eye Technologies](http://catseye.tc/).
122123
123124 Projects using Falderal
124125 -----------------------
125126
126127 (NOTE Actually, I'm sure this information can be extracted from Chrysoberyl
127 somehow, so just link to that here.)
128 somehow, so in the future, just link to that here.)
128129
129 Exanoke, Flobnar, Iphigeneia, Madison, Pail, Pixley, PL-{GOTO}.NET, Robin,
130 Exanoke, Flobnar, Hev, Iphigeneia, Madison, Pail, Pixley, PL-{GOTO}.NET, Robin,
130131 Quylthulg, Velo, and Xoomonk.
131132
132 Pail, Xoomonk, Madison, Velo, and Exanoke are good examples of how a literate
133 Xoomonk, Madison, Velo, and Exanoke are good examples of how a literate
133134 test suite can be useful in both describing a programming language through
134135 examples and testing that an implementation of the language does not violate
135136 the language specification.
+0
-62
doc/Quick_Start.markdown less more
0 Quick Start
1 ===========
2
3 First, install `Test.Falderal`.
4
5 % hg clone https://bitbucket.org/catseye/falderal/ -r rel_0_4
6 % cd falderal
7 % cabal install --prefix=$HOME --user
8
9 Define a programming language, or some other file format -- basically,
10 anything you can model as a function which takes strings to strings. In
11 Falderal terminology, this is a "functionality". Implement your
12 functionality in any programming language for which you can produce
13 executables for your system. (If you implement it in Haskell, you get
14 some side benefits, but it's not necessary.)
15
16 Often, depending on the syntax of your implementation language, you can
17 place your literate tests in the same file as your code. We'll use
18 Bird-style literate Haskell in this example.
19
20 module Gobvert
21
22 This is some kind of really trivial little language.
23
24 > gobvert "A" = "Z"
25 > gobvert "Z" = "A"
26
27 Then give your functionality a name, and write some tests for your
28 functionality. You use a Falderal pragma to identify which functionality
29 these tests are for.
30
31 -> Functionality "Gobvert a string" is implemented by
32 -> Haskell function Gobvert:gobvert
33
34 -> Tests for functionality "Gobvert a string"
35
36 The gobversion of A is Z.
37
38 | A
39 = Z
40
41 The gobversion of Z is A.
42
43 | Z
44 = A
45
46 The gobversions of other letters are not defined.
47
48 | Q
49 ? Not matched
50
51 Then, use the `falderal` tool to run these tests:
52
53 % falderal test Gobvert.lhs
54
55 All failures will be listed in a nicely-formatted report, including the
56 literate description that appears before each failing test.
57
58 You can also use the `falderal` tool to format your literate Haskell
59 file, including embedded tests, to a document format such as Markdown:
60
61 % falderal format markdown Gobvert.lhs >Gobvert.markdown
+0
-44
doc/Theory_of_Operation.markdown less more
0 Theory of Operation
1 ===================
2
3 The `falderal` tool from the `Test.Falderal` implementation of the
4 Falderal Literate Test Format allows the user to format Falderal tests
5 to different formats, and to run those tests and generate a report.
6
7 This document briefly describes how it works internally.
8
9 When `falderal` is asked to run a set of tests, first it formats them
10 to a set of programs which run the functionalities being tested with the
11 input text of the tests. These programs are called *results generators*.
12 Since each test may have one or more implementations, multiple results
13 generators may be generated, one for each implementation language
14 (currently Haskell and Bourne shell).
15
16 Each results generator runs many functions in a batch, for efficiency.
17 The results of running the functions are written to standard output
18 (which is redirected to a temporary file by `falderal`) in an intermediate
19 format. `falderal` then reads these temporary files, parses the
20 intermediate format, checks which of the test results do not match the
21 expected output, and generates a test report based on that.
22
23 The intermediate format is a simple text-based format containing groups of
24 lines. A single group may look like the following.
25
26 output
27 4
28 2
29 Test with ID 4 generated
30 two lines of output.
31
32 The first line is either `output` or `exception`. The next line contains
33 the ID of the test that generated this result. The line following that
34 contains the number of lines of text that the test generated (call it _n_).
35 The next _n_ lines contain the actual text generated. (If _n_ = 0, there
36 will be no such lines.) Immediately following this group will be either
37 another group, or the end-of-file.
38
39 The second and third lines in a group contain natural numbers; they may
40 contain arbitrary text after the final digit of the natural number, which is
41 ignored. (This is to simplify their generation from shell scripts, where
42 `wc -l` is used to produce the number of lines of output text, and where
43 `wc` also outputs the filename.)
+26
-18
doc/py-falderal.markdown less more
77
88 There are a few reasons I had for re-implementing Falderal in Python:
99
10 * The original Falderal implementation grew out of a Haskell-specific hack,
11 and it shows in how it's written.
10 * The original Falderal implementation grew out of a Haskell-specific hack,
11 and it shows in how it's written.
1212
13 * Fewer discrepancies between platforms. In particular, `ghc` for Windows
14 likes to operate in terms of MS-DOS end-of-lines (`CR`, `LF`), but I tend
15 to use it under Cygwin using Unix end-of-lines (`LF`).
13 * Fewer discrepancies between platforms. In particular, `ghc` for Windows
14 likes to operate in terms of MS-DOS end-of-lines (`CR`, `LF`), but I tend
15 to use it under Cygwin using Unix end-of-lines (`LF`).
1616
17 * Smaller install burden: Python sometimes comes bundled with the operating
18 system; Haskell rarely does.
17 * Smaller install burden: Python sometimes comes bundled with the operating
18 system; Haskell rarely does.
1919
20 * Haskell, being lazy, makes it harder to deal with exceptions; unless the
21 Haskell expression is being evaluated both strictly and deeply, exceptions
22 can slip by. For Falderal's purposes, this seems artificial, at best.
20 * Haskell, being lazy, makes it harder to deal with exceptions; unless the
21 Haskell expression is being evaluated both strictly and deeply, exceptions
22 can slip by. Haskell's facilities for forcing an expression to be
23 evaluated strictly and deeply are rather hacky (last I checked, the most
24 portable way to do so was to use `show`.) For Falderal's purposes, this
25 limitation seems artificial, at best.
2326
24 * Relatedly, Python (or CPython, anyway) has better error behavior than
25 Haskell (or `ghc`, anyway); when it crashes, it dumps a backtrace (which I
26 can then analyze), instead of just saying something pithy like `Prelude:
27 empty list` (which I can't.)
27 * Relatedly, Python (or CPython, anyway) has better error behavior than
28 Haskell (or `ghc`, anyway); when it crashes, it dumps a backtrace (which I
29 can then analyze), instead of just saying something pithy like `Prelude:
30 empty list` (which I can't.)
2831
29 * Any standard worth its salt should probably have more than one
30 implementation, anyway.
32 * Any standard worth its salt should probably have more than one
33 implementation, anyway. (Although, to be fair, the Haskell implementation
34 will probably wither in bitrot for a while before simply being forgotten.)
3135
3236 Features
3337 --------
6569 command "flargh"` anymore. I mean, it should certainly be the case that
6670 functionalities can have multiple implementations, but
6771 * Perhaps implementations should not be specified in Falderal documents
68 at all -- that approach is more abstract. But it requires them to be
72 at all — that approach is more abstract. But it requires them to be
6973 specified on the tool's command line, which in practice requires there
7074 to be a driver script to run the tests, for a particular implementation;
71 but this is not necessarily a bad thing.
75 but this is not necessarily a bad thing. (The effective practice has
76 been, actually, to write out a bit of "configuration Markdown" with
77 that "Functionality is implemented by" snippet in it, and to load that
78 before loading the main tests. Rather than to configure the functionality
79 implementation(s) on the command line itself.)
7280 * When `falderal` is run on more than one input file, what is the scope
7381 of a functionality, and what is the scope of the implementations of a
7482 functionality? Currently, in the Falderal semantics, that scope is
+62
-0
impl/Test.Falderal/doc/Quick_Start.markdown less more
0 Quick Start
1 ===========
2
3 First, install `Test.Falderal`.
4
5 % hg clone https://bitbucket.org/catseye/falderal/ -r rel_0_4
6 % cd falderal
7 % cabal install --prefix=$HOME --user
8
9 Define a programming language, or some other file format -- basically,
10 anything you can model as a function which takes strings to strings. In
11 Falderal terminology, this is a "functionality". Implement your
12 functionality in any programming language for which you can produce
13 executables for your system. (If you implement it in Haskell, you get
14 some side benefits, but it's not necessary.)
15
16 Often, depending on the syntax of your implementation language, you can
17 place your literate tests in the same file as your code. We'll use
18 Bird-style literate Haskell in this example.
19
20 module Gobvert
21
22 This is some kind of really trivial little language.
23
24 > gobvert "A" = "Z"
25 > gobvert "Z" = "A"
26
27 Then give your functionality a name, and write some tests for your
28 functionality. You use a Falderal pragma to identify which functionality
29 these tests are for.
30
31 -> Functionality "Gobvert a string" is implemented by
32 -> Haskell function Gobvert:gobvert
33
34 -> Tests for functionality "Gobvert a string"
35
36 The gobversion of A is Z.
37
38 | A
39 = Z
40
41 The gobversion of Z is A.
42
43 | Z
44 = A
45
46 The gobversions of other letters are not defined.
47
48 | Q
49 ? Not matched
50
51 Then, use the `falderal` tool to run these tests:
52
53 % falderal test Gobvert.lhs
54
55 All failures will be listed in a nicely-formatted report, including the
56 literate description that appears before each failing test.
57
58 You can also use the `falderal` tool to format your literate Haskell
59 file, including embedded tests, to a document format such as Markdown:
60
61 % falderal format markdown Gobvert.lhs >Gobvert.markdown
+44
-0
impl/Test.Falderal/doc/Theory_of_Operation.markdown less more
0 Theory of Operation
1 ===================
2
3 The `falderal` tool from the `Test.Falderal` implementation of the
4 Falderal Literate Test Format allows the user to format Falderal tests
5 to different formats, and to run those tests and generate a report.
6
7 This document briefly describes how it works internally.
8
9 When `falderal` is asked to run a set of tests, first it formats them
10 to a set of programs which run the functionalities being tested with the
11 input text of the tests. These programs are called *results generators*.
12 Since each test may have one or more implementations, multiple results
13 generators may be generated, one for each implementation language
14 (currently Haskell and Bourne shell).
15
16 Each results generator runs many functions in a batch, for efficiency.
17 The results of running the functions are written to standard output
18 (which is redirected to a temporary file by `falderal`) in an intermediate
19 format. `falderal` then reads these temporary files, parses the
20 intermediate format, checks which of the test results do not match the
21 expected output, and generates a test report based on that.
22
23 The intermediate format is a simple text-based format containing groups of
24 lines. A single group may look like the following.
25
26 output
27 4
28 2
29 Test with ID 4 generated
30 two lines of output.
31
32 The first line is either `output` or `exception`. The next line contains
33 the ID of the test that generated this result. The line following that
34 contains the number of lines of text that the test generated (call it _n_).
35 The next _n_ lines contain the actual text generated. (If _n_ = 0, there
36 will be no such lines.) Immediately following this group will be either
37 another group, or the end-of-file.
38
39 The second and third lines in a group contain natural numbers; they may
40 contain arbitrary text after the final digit of the natural number, which is
41 ignored. (This is to simplify their generation from shell scripts, where
42 `wc -l` is used to produce the number of lines of output text, and where
43 `wc` also outputs the filename.)