Given a script containing token descriptions (each a regular expression), tokn compiles an automaton which it can then use to efficiently convert a text file to a sequence of those tokens.
Required Ruby Version
None
Authors
Jeff Sember
Versions
- 0.1.1 April 15, 2013 (117 KB)
- 0.1.0 April 09, 2013 (30.5 KB)
- 0.0.9 April 04, 2013 (28 KB)
- 0.0.8 April 04, 2013 (28.5 KB)
- 0.0.6 March 16, 2013 (44.5 KB)