diff --git a/README.markdown b/README.markdown index e6a5694..0182422 100644 --- a/README.markdown +++ b/README.markdown @@ -27,7 +27,7 @@ You could write a bunch of standalone test sources, and store the output you expect from them in a bunch of other files, and write a shell script that runs each program and `diff`s the output with the expected output. But this is a -lot of clutter -- finding a particular example might not be so easy. Each +lot of clutter — finding a particular example might not be so easy. Each test source exists in a void, not necessarily logically grouped with other, similar tests. And any text you write describing a test needs to be in the comment syntax of your programming language (if your programming language @@ -94,20 +94,21 @@ This distribution contains: -* `doc` -- contains documents about Falderal. For the specification of +* `doc` — contains documents about Falderal. For the specification of the file format, see `doc/Falderal_Literate_Test_Format.markdown`. (Note that this specification should not be expected to remain stable through the 0.x version series.) There are other documents in there too. -* `bin/falderal` -- the reference implementation of Falderal, written in +* `bin/falderal` — the reference implementation of Falderal, written in Python and sometimes referred to as "py-falderal". It imports the sources in `src/falderal`. You don't need to install it; just add - the `bin` directory of this distribution to your `$PATH`. -* `impl/Test.Falderal` -s a (lagging, and not conformant) implementation of + the `bin` directory of this distribution to your `$PATH`. This + implementation is (somewhat) documented in `doc/py-falderal.markdown`. +* `impl/Test.Falderal` — a (lagging, and not conformant) implementation of Falderal in Haskell. -* `tests` -- a set of tests for Falderal itself. (Note that these are not +* `tests` — a set of tests for Falderal itself. (Note that these are not written in Falderal, as that would just be too confusing.) -* `HISTORY.markdown` -- changelog for releases of Falderal. -* `TODO.markdown` -- areas where Falderal and its implementations could be +* `HISTORY.markdown` — changelog for releases of Falderal. +* `TODO.markdown` — areas where Falderal and its implementations could be improved. Development @@ -118,19 +119,19 @@ [git mirror of the repository on Github](https://github.com/catseye/Falderal). Official release distfiles are available on the -[Falderal project page](http://catseye.tc/projects/falderal/) at +[Falderal project page](http://catseye.tc/node/Falderal) at [Cat's Eye Technologies](http://catseye.tc/). Projects using Falderal ----------------------- (NOTE Actually, I'm sure this information can be extracted from Chrysoberyl -somehow, so just link to that here.) +somehow, so in the future, just link to that here.) -Exanoke, Flobnar, Iphigeneia, Madison, Pail, Pixley, PL-{GOTO}.NET, Robin, +Exanoke, Flobnar, Hev, Iphigeneia, Madison, Pail, Pixley, PL-{GOTO}.NET, Robin, Quylthulg, Velo, and Xoomonk. -Pail, Xoomonk, Madison, Velo, and Exanoke are good examples of how a literate +Xoomonk, Madison, Velo, and Exanoke are good examples of how a literate test suite can be useful in both describing a programming language through examples and testing that an implementation of the language does not violate the language specification. diff --git a/doc/Quick_Start.markdown b/doc/Quick_Start.markdown deleted file mode 100644 index 737270e..0000000 --- a/doc/Quick_Start.markdown +++ /dev/null @@ -1,62 +0,0 @@ -Quick Start -=========== - -First, install `Test.Falderal`. - - % hg clone https://bitbucket.org/catseye/falderal/ -r rel_0_4 - % cd falderal - % cabal install --prefix=$HOME --user - -Define a programming language, or some other file format -- basically, -anything you can model as a function which takes strings to strings. In -Falderal terminology, this is a "functionality". Implement your -functionality in any programming language for which you can produce -executables for your system. (If you implement it in Haskell, you get -some side benefits, but it's not necessary.) - -Often, depending on the syntax of your implementation language, you can -place your literate tests in the same file as your code. We'll use -Bird-style literate Haskell in this example. - - module Gobvert - - This is some kind of really trivial little language. - - > gobvert "A" = "Z" - > gobvert "Z" = "A" - -Then give your functionality a name, and write some tests for your -functionality. You use a Falderal pragma to identify which functionality -these tests are for. - - -> Functionality "Gobvert a string" is implemented by - -> Haskell function Gobvert:gobvert - - -> Tests for functionality "Gobvert a string" - - The gobversion of A is Z. - - | A - = Z - - The gobversion of Z is A. - - | Z - = A - - The gobversions of other letters are not defined. - - | Q - ? Not matched - -Then, use the `falderal` tool to run these tests: - - % falderal test Gobvert.lhs - -All failures will be listed in a nicely-formatted report, including the -literate description that appears before each failing test. - -You can also use the `falderal` tool to format your literate Haskell -file, including embedded tests, to a document format such as Markdown: - - % falderal format markdown Gobvert.lhs >Gobvert.markdown diff --git a/doc/Theory_of_Operation.markdown b/doc/Theory_of_Operation.markdown deleted file mode 100644 index 6f645a1..0000000 --- a/doc/Theory_of_Operation.markdown +++ /dev/null @@ -1,44 +0,0 @@ -Theory of Operation -=================== - -The `falderal` tool from the `Test.Falderal` implementation of the -Falderal Literate Test Format allows the user to format Falderal tests -to different formats, and to run those tests and generate a report. - -This document briefly describes how it works internally. - -When `falderal` is asked to run a set of tests, first it formats them -to a set of programs which run the functionalities being tested with the -input text of the tests. These programs are called *results generators*. -Since each test may have one or more implementations, multiple results -generators may be generated, one for each implementation language -(currently Haskell and Bourne shell). - -Each results generator runs many functions in a batch, for efficiency. -The results of running the functions are written to standard output -(which is redirected to a temporary file by `falderal`) in an intermediate -format. `falderal` then reads these temporary files, parses the -intermediate format, checks which of the test results do not match the -expected output, and generates a test report based on that. - -The intermediate format is a simple text-based format containing groups of -lines. A single group may look like the following. - - output - 4 - 2 - Test with ID 4 generated - two lines of output. - -The first line is either `output` or `exception`. The next line contains -the ID of the test that generated this result. The line following that -contains the number of lines of text that the test generated (call it _n_). -The next _n_ lines contain the actual text generated. (If _n_ = 0, there -will be no such lines.) Immediately following this group will be either -another group, or the end-of-file. - -The second and third lines in a group contain natural numbers; they may -contain arbitrary text after the final digit of the natural number, which is -ignored. (This is to simplify their generation from shell scripts, where -`wc -l` is used to produce the number of lines of output text, and where -`wc` also outputs the filename.) diff --git a/doc/py-falderal.markdown b/doc/py-falderal.markdown index 70db252..3ef56c3 100644 --- a/doc/py-falderal.markdown +++ b/doc/py-falderal.markdown @@ -8,27 +8,31 @@ There are a few reasons I had for re-implementing Falderal in Python: -* The original Falderal implementation grew out of a Haskell-specific hack, - and it shows in how it's written. +* The original Falderal implementation grew out of a Haskell-specific hack, + and it shows in how it's written. -* Fewer discrepancies between platforms. In particular, `ghc` for Windows - likes to operate in terms of MS-DOS end-of-lines (`CR`, `LF`), but I tend - to use it under Cygwin using Unix end-of-lines (`LF`). +* Fewer discrepancies between platforms. In particular, `ghc` for Windows + likes to operate in terms of MS-DOS end-of-lines (`CR`, `LF`), but I tend + to use it under Cygwin using Unix end-of-lines (`LF`). -* Smaller install burden: Python sometimes comes bundled with the operating - system; Haskell rarely does. +* Smaller install burden: Python sometimes comes bundled with the operating + system; Haskell rarely does. -* Haskell, being lazy, makes it harder to deal with exceptions; unless the - Haskell expression is being evaluated both strictly and deeply, exceptions - can slip by. For Falderal's purposes, this seems artificial, at best. +* Haskell, being lazy, makes it harder to deal with exceptions; unless the + Haskell expression is being evaluated both strictly and deeply, exceptions + can slip by. Haskell's facilities for forcing an expression to be + evaluated strictly and deeply are rather hacky (last I checked, the most + portable way to do so was to use `show`.) For Falderal's purposes, this + limitation seems artificial, at best. -* Relatedly, Python (or CPython, anyway) has better error behavior than - Haskell (or `ghc`, anyway); when it crashes, it dumps a backtrace (which I - can then analyze), instead of just saying something pithy like `Prelude: - empty list` (which I can't.) +* Relatedly, Python (or CPython, anyway) has better error behavior than + Haskell (or `ghc`, anyway); when it crashes, it dumps a backtrace (which I + can then analyze), instead of just saying something pithy like `Prelude: + empty list` (which I can't.) -* Any standard worth its salt should probably have more than one - implementation, anyway. +* Any standard worth its salt should probably have more than one + implementation, anyway. (Although, to be fair, the Haskell implementation + will probably wither in bitrot for a while before simply being forgotten.) Features -------- @@ -66,10 +70,14 @@ command "flargh"` anymore. I mean, it should certainly be the case that functionalities can have multiple implementations, but * Perhaps implementations should not be specified in Falderal documents - at all -- that approach is more abstract. But it requires them to be + at all — that approach is more abstract. But it requires them to be specified on the tool's command line, which in practice requires there to be a driver script to run the tests, for a particular implementation; - but this is not necessarily a bad thing. + but this is not necessarily a bad thing. (The effective practice has + been, actually, to write out a bit of "configuration Markdown" with + that "Functionality is implemented by" snippet in it, and to load that + before loading the main tests. Rather than to configure the functionality + implementation(s) on the command line itself.) * When `falderal` is run on more than one input file, what is the scope of a functionality, and what is the scope of the implementations of a functionality? Currently, in the Falderal semantics, that scope is diff --git a/impl/Test.Falderal/doc/Quick_Start.markdown b/impl/Test.Falderal/doc/Quick_Start.markdown new file mode 100644 index 0000000..737270e --- /dev/null +++ b/impl/Test.Falderal/doc/Quick_Start.markdown @@ -0,0 +1,62 @@ +Quick Start +=========== + +First, install `Test.Falderal`. + + % hg clone https://bitbucket.org/catseye/falderal/ -r rel_0_4 + % cd falderal + % cabal install --prefix=$HOME --user + +Define a programming language, or some other file format -- basically, +anything you can model as a function which takes strings to strings. In +Falderal terminology, this is a "functionality". Implement your +functionality in any programming language for which you can produce +executables for your system. (If you implement it in Haskell, you get +some side benefits, but it's not necessary.) + +Often, depending on the syntax of your implementation language, you can +place your literate tests in the same file as your code. We'll use +Bird-style literate Haskell in this example. + + module Gobvert + + This is some kind of really trivial little language. + + > gobvert "A" = "Z" + > gobvert "Z" = "A" + +Then give your functionality a name, and write some tests for your +functionality. You use a Falderal pragma to identify which functionality +these tests are for. + + -> Functionality "Gobvert a string" is implemented by + -> Haskell function Gobvert:gobvert + + -> Tests for functionality "Gobvert a string" + + The gobversion of A is Z. + + | A + = Z + + The gobversion of Z is A. + + | Z + = A + + The gobversions of other letters are not defined. + + | Q + ? Not matched + +Then, use the `falderal` tool to run these tests: + + % falderal test Gobvert.lhs + +All failures will be listed in a nicely-formatted report, including the +literate description that appears before each failing test. + +You can also use the `falderal` tool to format your literate Haskell +file, including embedded tests, to a document format such as Markdown: + + % falderal format markdown Gobvert.lhs >Gobvert.markdown diff --git a/impl/Test.Falderal/doc/Theory_of_Operation.markdown b/impl/Test.Falderal/doc/Theory_of_Operation.markdown new file mode 100644 index 0000000..6f645a1 --- /dev/null +++ b/impl/Test.Falderal/doc/Theory_of_Operation.markdown @@ -0,0 +1,44 @@ +Theory of Operation +=================== + +The `falderal` tool from the `Test.Falderal` implementation of the +Falderal Literate Test Format allows the user to format Falderal tests +to different formats, and to run those tests and generate a report. + +This document briefly describes how it works internally. + +When `falderal` is asked to run a set of tests, first it formats them +to a set of programs which run the functionalities being tested with the +input text of the tests. These programs are called *results generators*. +Since each test may have one or more implementations, multiple results +generators may be generated, one for each implementation language +(currently Haskell and Bourne shell). + +Each results generator runs many functions in a batch, for efficiency. +The results of running the functions are written to standard output +(which is redirected to a temporary file by `falderal`) in an intermediate +format. `falderal` then reads these temporary files, parses the +intermediate format, checks which of the test results do not match the +expected output, and generates a test report based on that. + +The intermediate format is a simple text-based format containing groups of +lines. A single group may look like the following. + + output + 4 + 2 + Test with ID 4 generated + two lines of output. + +The first line is either `output` or `exception`. The next line contains +the ID of the test that generated this result. The line following that +contains the number of lines of text that the test generated (call it _n_). +The next _n_ lines contain the actual text generated. (If _n_ = 0, there +will be no such lines.) Immediately following this group will be either +another group, or the end-of-file. + +The second and third lines in a group contain natural numbers; they may +contain arbitrary text after the final digit of the natural number, which is +ignored. (This is to simplify their generation from shell scripts, where +`wc -l` is used to produce the number of lines of output text, and where +`wc` also outputs the filename.)