TINY-LEX(MyLex)

Overview

MyLex is a tiny lexical analyzer implemented in C++ and it is my home work of compiler.

STEPS

1. parse .mylex file
2. generate a list of NFA by the regex strings
3. generate a large NFA by combining the NFAs
4. convert the NFA to a DFA
5. minimize the DFA
6. generate the c code from DFA

Some Important Algorithms

1. convert regex to postfix expression
2. convert postfix expression to NFA, reference
3. convert NFA to DFA
4. minimize DFA

Compile And Run

Environment for MyLex

1. OS: Linux , Unix or Mac OS
2. Compiler: g++ or clang
3. Library: Boost

Environment for c code generated by MyLex

1. OS: Linux , Unix or Mac OS
2. Compiler: gcc or clang, Must Use Compilers Which Support C99

Compiler MyLex

For Debug
  make DEBUG=1

Not For Debug
  make

How to use

mylex infile [outfile](default output to stdout)

Sample

make
./mylex sample/c_lex.mylex > c_lex.c
make c_lex

MyLex Syntax

• Sample: c_lex.mylex
• Sample Input: sample.c
• Sample Output: sample.out

File Format

%{
[declear]
%}
%%
[Entry]
%%
[Code]

Entry Format

[Regex] {
  [Handler] with the param (shm_token)
}

And there are some constrains:

• you must provide a main function in 'Code' section
• in 'main', you must invoke myylex(char* filename, void (*func)())
• you must pass a function pointer when you invoke myylex
• in myylex, you can use the 'Token List'

// To travel the token list
// You should define trav_func
void iter_list(void (*trav_func)(Token*));

// print a specific token (a pre-defined trav_func)
void print_token(Token* token);

// invoke in 'main'
void myylex(char* input, void (*func)());

// init a iter
#define INIT_ITER(iter) 
// get next token
#define ITER_NEXT(iter)  (iter = iter->next)
// if there is a next token
#define ITER_HASNEXT(iter) (iter != NULL && iter->next != NULL)
// if the iter is NULL
#define ITER_ISEND(iter) (iter == NULL)