lexertl: The Modular Lexical Analyser Generator

Latest Version:

• 2010-09-04: lexertl.zip

Blog: here

What I'm Currently Working on

• Introduce template type as replacement for std::size_t in tables.
• Add a build() function to generator that outputs a char_state_machine directly.
• state_machine will always have a 256 entry first phase lookup.
• Use strong exception guarantee in rules class and anywhere else that it is reasonable to have it.
• Support memory allocator parameter in templates.
• Investigate auto minimisation of state machine.
• Allow user data in state machine.
• Recode char_state_machine to support re2c style generators.
• Write re2c style generator.
• Write a file_iterator.
• Preparing for boost review.

Outstanding Features

• Redo code generator (in the style of re2c with ^, $ and multi-state support).
• Investigate Ville Laurikari's tagged DFAs in more depth.
• Implement POSIX Extensions support?

Boost

boost::spirit has had lexertl integrated for several years now. After discussing with Hartmut, I have decided not to supply a zip of boost::lexer directly anymore, but instead link to the docs here. lexertl will continue to be developed and will get new features and fixes implemented first (the boost version will have bug fixes applied shortly afterwards!) lexertl will continue to only require standard C++ support. The codebase continues to build on VC++ 6 (let me know if your compiler can't build it).