git @ Cat's Eye Technologies Lexeduct / gh-pages index.html
gh-pages

Tree @gh-pages (Download .tar.gz)

index.html @gh-pages — raw · history · blame 

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
<!DOCTYPE html>
<html lang="en-us">
  <head>
    <meta charset="UTF-8">
    <title>Lexeduct by catseye</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" type="text/css" href="stylesheets/normalize.css" media="screen">
    <link href='http://fonts.googleapis.com/css?family=Open+Sans:400,700' rel='stylesheet' type='text/css'>
    <link rel="stylesheet" type="text/css" href="stylesheets/stylesheet.css" media="screen">
    <link rel="stylesheet" type="text/css" href="stylesheets/github-light.css" media="screen">
  </head>
  <body>
    <section class="page-header">
      <h1 class="project-name">Lexeduct</h1>
      <h2 class="project-tagline">Experimental framework for text-processing pipelines in JS (node or browser)</h2>
      <a href="https://github.com/catseye/Lexeduct" class="btn">View on GitHub</a>
      <a href="https://github.com/catseye/Lexeduct/zipball/master" class="btn">Download .zip</a>
      <a href="https://github.com/catseye/Lexeduct/tarball/master" class="btn">Download .tar.gz</a>
    </section>

    <section class="main-content">
      <h1>
<a id="lexeduct" class="anchor" href="#lexeduct" aria-hidden="true"><span class="octicon octicon-link"></span></a>Lexeduct</h1>

<p><strong>You can try Lexeduct live in your web browser here: <a href="http://catseye.github.io/Lexeduct/in-browser/">Lexeduct Online</a></strong></p>

<blockquote>
<p>"this is not a wheel I've re-invented before"</p>
</blockquote>

<p><strong>Lexeduct</strong> is an experimental framework for text-processing pipelines,
written in Javascript, usable both on the console under <a href="https://nodejs.org/">Node.js</a>, and
in a web browser.</p>

<p>It is currently a work in progress.  The framework and usage and everything
is subject to change without notice.  The version number is nominally 0.1-PRE;
there is not yet a released version.</p>

<p>Being a framework, Lexeduct inevitably handles some use cases well, and other
use cases poorly.  See the "Limitations" section below for more details.</p>

<p>The name "Lexeduct" is in analogy with "aqueduct": conduits for words intead
of water.</p>

<h2>
<a id="basic-usage" class="anchor" href="#basic-usage" aria-hidden="true"><span class="octicon octicon-link"></span></a>Basic Usage</h2>

<p>The main tool is <code>lexeduct.js</code>.  You can <code>cd</code> into the <code>src</code> directory and run
it as <code>./lexeduct.js</code>, or you can put the <code>src</code> directory on your executable
search path, for example like</p>

<pre><code>export PATH=$PATH:/path/to/lexeduct/src
</code></pre>

<p>and run it as <code>lexeduct.js</code> from anywhere on your system.  (YMMV on Windows.)</p>

<p>The basic usage is</p>

<pre><code>lexeduct.js {param=value|transformer-name}
</code></pre>

<p>So, for example,</p>

<pre><code>$ echo 'Hello!' | lexeduct.js upper
HELLO
</code></pre>

<p>Parameters can be given with the syntax <code>name=value</code> before the name of the
transformer they are to be applied to:</p>

<pre><code>$ echo 'Hello' | lexeduct.js chars=e remove-chars
Hllo
</code></pre>

<p>You can of course use shell pipelines to compose transformers:</p>

<pre><code>$ echo 'Hello!' | lexeduct.js upper | lexeduct.js chars=' ' insert-chars
H E L L O !
</code></pre>

<p><em>Or</em> you can name multiple transformers on <code>lexeduct.js</code>'s command line to
compose them:</p>

<pre><code>$ echo 'Hello!' | lexeduct.js upper chars=' ' insert-chars
H E L L O !
</code></pre>

<p>Multiple transformers are applied left-to-right.</p>

<pre><code>$ echo 'Hello!' | lexeduct.js chars=a insert-chars upper
HAEALALAOA!A

$ echo 'Hello!' | lexeduct.js upper chars=a insert-chars
HaEaLaLaOa!a
</code></pre>

<h2>
<a id="transformers" class="anchor" href="#transformers" aria-hidden="true"><span class="octicon octicon-link"></span></a>Transformers</h2>

<p>The idea is that this repository will eventually contain a giant catalogue
of possible text transformers that can be composed.  Or at least, more than
are presently included.</p>

<p>Each transformer is in a seperate Javascript file in the <code>src/transformers</code>
directory which exports, node-style, a single function called <code>makeTransformer</code>
which takes a configuration object and returns a transformer function.  The
transformer function takes two arguments: the current string to process, and
(optionally) an object which can be used to store ancillary state.  Every
transformer function should return either a string, or null (not yet supported),
or an array of strings (not yet supported.)</p>

<p>The module may also export a couple of other things, like an English description
of the transformer, and the possible configuration options.  For a reasonably
simple example, see the source of the <code>upper</code> transformer, in <a href="https://github.com/catseye/Lexeduct/blob/master/src/transformers/upper.js">upper.js</a>.</p>

<p>State deposited into the state object is shared by all transformers, so it's
a good idea to choose a key that you think will probably be unique.</p>

<h2>
<a id="in-browser-version" class="anchor" href="#in-browser-version" aria-hidden="true"><span class="octicon octicon-link"></span></a>In-Browser Version</h2>

<p>Run <code>./make.sh</code> from this directory (or the commands it contains) to generate
a Javascript file which contains all the available transformers in a format
suitable for loading in an HTML document.</p>

<p>Then open <code>demo/lexeduct.html</code> in your browser.  It provides a UI for composing
these transformers and applying them to text provided in a textarea.</p>

<h2>
<a id="acknowledgements" class="anchor" href="#acknowledgements" aria-hidden="true"><span class="octicon octicon-link"></span></a>Acknowledgements</h2>

<p>Lexeduct was partly inspired by, and is partly a product of parallel evolution
resembling, <a href="https://github.com/MichaelPaulukonis/">Michael Paulukonis</a>'s <a href="https://github.com/MichaelPaulukonis/text-munger">TextMunger</a>.  It is also indebted to
various and sundry discussion with him, and others on the
<a href="https://groups.google.com/forum/#!forum/generativetext">GenerativeText Forum</a>, particularly <a href="https://github.com/enkiv2/">John Ohno</a>.</p>

      <footer class="site-footer">
        <span class="site-footer-owner"><a href="https://github.com/catseye/Lexeduct">Lexeduct</a> is maintained by <a href="https://github.com/catseye">catseye</a>.</span>

        <span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub Pages</a> using the <a href="https://github.com/jasonlong/cayman-theme">Cayman theme</a> by <a href="https://twitter.com/jasonlong">Jason Long</a>.</span>
      </footer>

    </section>

  
  </body>
</html>