- When a Lexer::typeIdentifier is found, it can be compared to a list of
other tokens, with the possibility of changing the type. This applies to
tokens that are longer than the four-character lookahead in the Lexer.
With this change, the Lexer can now identify all operators supported by Eval,
and therefore the Lexer can be used on all Eval input expressions. This is
because all the evaluator needs to know is the distinction between operators
and operands.
- Implemented Lexer::word, which is just like ::token, but does not
understand dates, durations or operators.
- Implemented Lexer::split, which uses Lexer::word.
- Added unit tests.
