git @ Cat's Eye Technologies Falderal / gh-pages index.html
gh-pages

Tree @gh-pages (Download .tar.gz)

index.html @gh-pages — raw · history · blame 

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
<!DOCTYPE html>
<html>

  <head>
    <meta charset='utf-8' />
    <meta http-equiv="X-UA-Compatible" content="chrome=1" />
    <meta name="description" content="Falderal : Markdown-embedded literate testing for programming languages" />

    <link rel="stylesheet" type="text/css" media="screen" href="stylesheets/stylesheet.css">

    <title>Falderal</title>
  </head>

  <body>

    <!-- HEADER -->
    <div id="header_wrap" class="outer">
        <header class="inner">
          <a id="forkme_banner" href="https://github.com/catseye/Falderal">View on GitHub</a>

          <h1 id="project_title">Falderal</h1>
          <h2 id="project_tagline">Markdown-embedded literate testing for programming languages</h2>

            <section id="downloads">
              <a class="zip_download_link" href="https://github.com/catseye/Falderal/zipball/master">Download this project as a .zip file</a>
              <a class="tar_download_link" href="https://github.com/catseye/Falderal/tarball/master">Download this project as a tar.gz file</a>
            </section>
        </header>
    </div>

    <!-- MAIN CONTENT -->
    <div id="main_content_wrap" class="outer">
      <section id="main_content" class="inner">
        <h1>
<a name="falderal" class="anchor" href="#falderal"><span class="octicon octicon-link"></span></a>Falderal</h1>

<p>Version 0.9-2014.0525 "Municipal Pier #2, May 25th 2014"</p>

<p>This is the reference distribution of Falderal, a file format for literate
test suites.  Falderal is particularly suited for:</p>

<ul>
<li>  documenting programming languages with prose and examples</li>
<li>  testing multiple implementations of a programming language</li>
<li>  performing Test-Driven Language Design (TDLD)</li>
<li>  being embedded in Markdown documents</li>
</ul><p>Falderal in three words: "Doctests for DSLs".</p>

<h2>
<a name="motivation" class="anchor" href="#motivation"><span class="octicon octicon-link"></span></a>Motivation</h2>

<p>Say you have a programming language.  You probably have a document describing
the language.  That document probably includes examples.  Those examples are
probably intended to produce some kind of output.  That output is probably
expected to be a certain, predicted thing, and if it's not that thing, the
implementation is probably considered incorrect.</p>

<p>So why not write those examples in a format that can be tested?</p>

<p>You could write a bunch of standalone test sources, and store the output you
expect from them in a bunch of other files, and write a shell script that runs
each program and <code>diff</code>s the output with the expected output.  But this is a
lot of clutter — finding a particular example might not be so easy.  Each
test source exists in a void, not necessarily logically grouped with other,
similar tests.  And any text you write describing a test needs to be in the
comment syntax of your programming language (if your programming language
supports comments) and is also detached from all the other test descriptions.</p>

<p>You could write doctests, but if your language isn't implemented in Python
it's awkward, and there can be awkward quoting issues with how you embed your
test sources inside that big Python string.</p>

<p>You could write unit tests in the unit test framework of your choice, but
if your programming language has more than one implementation one day (and
you should really consider that possibility) then you might not be able to
re-use it so easily for other implementations in other languages.</p>

<p>Or... you could write a Markdown document with beautiful yet precise prose
describing your wonderful language, alternating with example code (in the
form of embedded Falderal tests) clarifying each of the points you are
making; then you could use a Falderal-speaking tool to run each of these tests
against any implementation of your language which exists or will exist in
the future.</p>

<p><em>And</em> you could even write this document <em>before</em> you even start implementing
your language; then when it is all clear "on paper", you have a target at
which you can aim while writing your language.  As you implement more and more
of it, more and more tests in your test suite will pass.  This is the essence
of Test-Driven Language Design (TDLD).</p>

<h2>
<a name="features-of-the-format" class="anchor" href="#features-of-the-format"><span class="octicon octicon-link"></span></a>Features of the Format</h2>

<p>Falderal is just a file format; it does not specify exactly what tools must
do with the tests they extract.  However, it is generally assumed that most
tools will want to, at the very least,</p>

<ul>
<li>  Run tests from one or more documents.</li>
<li>  Report the results, with some given level of detail.</li>
</ul><p>Each Falderal test is for some abstract <em>functionality</em>, and each
functionality may have multiple concrete <em>implementations</em>.  Thus the same
tests can be run multiple times, once for each implementation of the
functionality they test.</p>

<p>Directives in the Falderal document may assign functionalities to tests,
and may define implementations for given functionalities.  Implementations
may be defined outside of any document, as well.  Falderal defines one kind
of implementation, implementation by Bourne shell command, but is not
inherently restricted from supporting other kinds of implementations.</p>

<h2>
<a name="inherent-limitations" class="anchor" href="#inherent-limitations"><span class="octicon octicon-link"></span></a>Inherent Limitations</h2>

<p>Being tests, rather than proofs, Falderal tests cannot tell you that your
implementation of a language is correct.  If one or more tests fail, that's
an indication that your implementation is not correct; but even if all tests
pass, you have no guarantee that the implementation doesn't do something
contrary to the spec in one of the infinite number of cases that you have not
enumerated.</p>

<p>There is also no way to test that certain programs represent infinite loops,
for reasons that should be obvious.</p>

<h2>
<a name="contents-of-this-distribution" class="anchor" href="#contents-of-this-distribution"><span class="octicon octicon-link"></span></a>Contents of this Distribution</h2>

<p>This distribution contains:</p>

<ul>
<li>  <code>doc</code> — contains documents about Falderal.  For the specification of
the file format, see <code>doc/Falderal_Literate_Test_Format.markdown</code>.
(Note that this specification should not be expected to remain stable
through the 0.x version series.)  There are other documents in there too.</li>
<li>  <code>bin/falderal</code> — the reference implementation of Falderal, written in
Python and sometimes referred to as "py-falderal".  It imports the
sources in <code>src/falderal</code>.  You don't need to install it; just add
the <code>bin</code> directory of this distribution to your <code>$PATH</code>.  This
implementation is (somewhat) documented in <code>doc/py-falderal.markdown</code>.</li>
<li>  <code>src</code> — source code for py-falderal.</li>
<li>  <code>impl/Test.Falderal</code> — a (lagging, and not conformant) implementation of
Falderal in Haskell.</li>
<li>  <code>tests</code> — a set of tests for Falderal itself.  (Note that these are not
written in Falderal, as that would just be too confusing.)</li>
<li>  <code>HISTORY.markdown</code> — changelog for releases of Falderal.</li>
<li>  <code>TODO.markdown</code> — areas where Falderal and its implementations could be
improved.</li>
</ul><h2>
<a name="development" class="anchor" href="#development"><span class="octicon octicon-link"></span></a>Development</h2>

<p>Falderal development is
<a href="https://bitbucket.org/catseye/falderal/">hosted on Bitbucket</a> with a
<a href="https://github.com/catseye/Falderal">git mirror of the repository on Github</a>.</p>

<p>Official release distfiles are available on the
<a href="http://catseye.tc/node/Falderal">Falderal project page</a> at
<a href="http://catseye.tc/">Cat's Eye Technologies</a>.</p>

<h2>
<a name="projects-using-falderal" class="anchor" href="#projects-using-falderal"><span class="octicon octicon-link"></span></a>Projects using Falderal</h2>

<p>(NOTE Actually, I'm sure this information can be extracted from Chrysoberyl
somehow, so in the future, just link to that here.)</p>

<p>Exanoke, Flobnar, Hev, Iphigeneia, Madison, Pail, Pixley, PL-{GOTO}.NET, Robin,
Quylthulg, Velo, and Xoomonk.</p>

<p>Xoomonk, Madison, Velo, and Exanoke are good examples of how a literate
test suite can be useful in both describing a programming language through
examples and testing that an implementation of the language does not violate
the language specification.</p>

<p>Xoomonk, Madison, Velo, and Exanoke are, in fact, exercises in Test-Driven
Language Design (TDLD), where the tests were written as part of designing the
language, before any attempt at implementation; the others are more like
traditional test suites, written after-the-fact.</p>
      </section>
    </div>

    <!-- FOOTER  -->
    <div id="footer_wrap" class="outer">
      <footer class="inner">
        <p class="copyright">Falderal maintained by <a href="https://github.com/catseye">catseye</a></p>
        <p>Published with <a href="http://pages.github.com">GitHub Pages</a></p>
      </footer>
    </div>

    

  </body>
</html>