Update spec to describe freestyle blocks (could use more work.)
Chris Pressey
8 years ago
16 | 16 | Overview |
17 | 17 | -------- |
18 | 18 | |
19 | A Falderal Literate Test Suite is a text file where some of the lines | |
19 | A Falderal Literate Test Suite is a plain text file where some of the lines | |
20 | 20 | have special meaning to the Falderal Literate Test Format. Certain |
21 | groupings of the lines defined by this format are intended to denote tests. | |
22 | ||
23 | A tool which claims to understand this format may choose to extract these | |
24 | tests, run them, and report which and how many of those tests passed or | |
25 | failed. That would be a typical application for this format. However, it is | |
26 | not required to do this with the tests; it may, for example, reformat the | |
21 | groupings of the lines defined by this format are intended to denote tests, | |
22 | while others denote pragmas which specify how those tests should be run. | |
23 | Outside of these groupings, lines have no meaning to the Falderal Literate | |
24 | Test Format other than that they are presumed to be mainly descriptive text. | |
25 | ||
26 | The plain text file may also be formatted in some other format, such as | |
27 | Markdown, although this is not required. | |
28 | ||
29 | A tool which claims to understand the Falderal Literate Test Format may | |
30 | choose to extract the tests from such a document, run them, and report which | |
31 | and how many of them passed or failed. That would be a typical application | |
32 | for this format. However, no tool is strictly required to do this with the | |
33 | tests found in a document; a tool may, for example, simply reformat the | |
27 | 34 | tests to produce an output file in a different format. |
28 | 35 | |
29 | 36 | In the syntax templates below, words in `/slashes/` indicate a variable |
32 | 39 | Syntax |
33 | 40 | ------ |
34 | 41 | |
35 | Lines which have special meaning to the Falderal Literate Test Format | |
36 | always begin with an indent of four (4) spaces from the leftmost column | |
37 | followed immediately by a series of distinguished characters, called an | |
38 | _introducer_. The introducers which have meaning to the Falderal Literate | |
39 | Test Format are as follows: | |
40 | ||
41 | * `->` (hyphen, greater-than sign): pragma | |
42 | * `| ` (vertical bar, space): test body text | |
43 | * `+ ` (plus sign, space): test input text | |
44 | * `= ` (equals sign, space): expected output text | |
45 | * `? ` (question mark, space): expected error text | |
46 | * `> ` (greater-than sign, space): Bird-style embedded code | |
42 | Each grouping of lines which has special meaning to the Falderal Literate | |
43 | Test Format always begins with an indent of four (4) spaces from the | |
44 | leftmost column, preceded by non-indented text, and followed by either | |
45 | non-indented text or the end of the file. Such a grouping of lines is | |
46 | called a _block_. | |
47 | ||
48 | There are two general formats to any block. In the first, "verbose" format, | |
49 | each indent of 4 spaces is followed immediately on that line by distinguished | |
50 | sequence of characters, called an _introducer_. The introducers which have | |
51 | meaning to the Falderal Literate Test Format are as follows: | |
52 | ||
53 | * `->` (hyphen, greater-than sign): pragma | |
54 | * `| ` (vertical bar, space): test body text | |
55 | * `+ ` (plus sign, space): test input text | |
56 | * `= ` (equals sign, space): expected output text | |
57 | * `? ` (question mark, space): expected error text | |
47 | 58 | |
48 | 59 | If the same introducer occurs on multiple adjacent lines, the text after |
49 | 60 | each introducer is concatenated to form one chunk of relevant text. This |
50 | 61 | allows, for example, multi-line text to be given as the body, the input, |
51 | or the expected output of a test. Lines without introducers are called | |
52 | _intervening text_. | |
62 | or the expected output of a test. | |
53 | 63 | |
54 | 64 | There are some restrictions on the order in which introducers can sensibly |
55 | occur: | |
56 | ||
57 | * Test body text should occur after intervening text. | |
65 | occur in a block: | |
66 | ||
67 | * Test body text should occur at the start of a block. | |
58 | 68 | * Test body test may be optionally followed by test input text. |
59 | * The first test input text must be immediately preceded by test body text, | |
60 | with no intervening text. | |
69 | * The first test input text must be immediately preceded by test body text. | |
61 | 70 | * Subsequent test input texts need not be preceded by a test body text; |
62 | in this case, the previously-defined test body text will be used again. | |
71 | in this case, the previously-appearing test body text will be used again. | |
63 | 72 | * Test body text must be followed by either test input text, expected |
64 | 73 | output text, or expected error text, with no intervening text. |
65 | 74 | * Either expected output or error text must follow either test body |
67 | 76 | |
68 | 77 | See the sections for these introducers, below, for more details. |
69 | 78 | |
70 | Bird-style embedded code is not considered part of a test by the Falderal | |
71 | Literate Test Format, but may be recognized as such by a formatting tool, | |
72 | for purposes of formatting. | |
73 | ||
79 | In the other, "freestyle" format, not all lines in a block require | |
80 | introducers. A freestyle format block is indentified as such if one or | |
81 | more of the final lines of the block begin with any of the following | |
82 | introducers: | |
83 | ||
84 | * `=> `: expected output text | |
85 | * `==> `: expected output text | |
86 | * `===> `: expected output text | |
87 | * `?> `: expected error text | |
88 | * `??> `: expected error text | |
89 | * `???> `: expected error text | |
90 | ||
91 | If a block is identified as a freestyle block, all lines preceding the | |
92 | final lines with one of these introducers, are interpreted as having | |
93 | no introducer at all (even if they begin with `| ` or some other sequence | |
94 | already mentioned) and are used as the test body block. | |
95 | ||
96 | Lines without introducers are called _intervening text_. | |
74 | 97 | Lines of intervening text are classified as either blank or non-blank. A |
75 | 98 | line is blank if it contains no characters, or if it contains only whitespace. |
76 | 99 | A group of non-blank lines is referred to as a _paragraph_. |
216 | 239 | test input text; either way it must then be followed immediately by either |
217 | 240 | and expected output section or expected error section. |
218 | 241 | |
219 | Valid examples: | |
242 | Valid examples in the "verbose" format: | |
220 | 243 | |
221 | 244 | | thing to test |
222 | 245 | = output to expect |
234 | 257 | |
235 | 258 | + different input to give the immediately previously defined test body |
236 | 259 | ? different error to expect |
260 | ||
261 | Valid examples in the "freestyle" format: | |
262 | ||
263 | thing to test | |
264 | => output to expect | |
265 | ||
266 | thing to test | |
267 | ==> output to expect | |
268 | ||
269 | thing to test | |
270 | ===> output to expect | |
271 | ||
272 | thing to test | |
273 | ?> error to expect | |
274 | ||
275 | thing to test | |
276 | ??> error to expect | |
277 | ||
278 | thing to test | |
279 | ???> error to expect | |
237 | 280 | |
238 | 281 | Invalid examples: |
239 | 282 |