Commit graph

144 commits

Author SHA1 Message Date
Paul Beckingham
9630b76990 Build: Resolve warnings from flod build 2015-03-29 18:21:33 -04:00
Paul Beckingham
289780c8cc Lexer: Added comments 2015-03-20 17:16:00 -04:00
Paul Beckingham
d202691638 Lexer
- Improved comment.
2015-03-07 10:09:58 -05:00
Paul Beckingham
e11b333a0b Lexer
- ::isPath was not observing the null character string terminator.
2015-03-05 21:22:40 -05:00
Paul Beckingham
53e9bd0cbd Lexer
- Words cannot contain single-char operators.
2015-03-02 00:03:01 -05:00
Paul Beckingham
237d932ff9 Lexer
- Improved ::isIdentifier, ::isUUID and ::isDOM.
2015-03-01 23:54:45 -05:00
Paul Beckingham
2af470bb90 Lexer
- Lexer::Type::identifier now includes DOM references.
2015-03-01 22:08:19 -05:00
Paul Beckingham
309b607672 Lexer
- Number digits followed by non-whitespace, non-operators are not numbers, ie
  "2nd" is not "2","nd".
2015-03-01 16:03:10 -05:00
Paul Beckingham
3f2d377fef Lexer
- Allow '=' in rc.<name>[:=]<value>, but not in non-rc Lexer::Type::pair
  combinations. That means 'name=value' is not a Lexer::Type::pair.
2015-02-28 12:05:24 -05:00
Paul Beckingham
e67c6c45cf Lexer
- Strings now retain their quotes, for compatibility with 2.4.1.
2015-02-24 23:01:12 -05:00
Paul Beckingham
8791c0a921 Lexer
- Migrated old noSpaces() function into Lexer::isOneWord.
2015-02-22 18:23:26 -05:00
Paul Beckingham
745aad0d27 Lexer
- Renamed Lexer2 to Lexer, it looks good enough to assume control.
2015-02-22 18:23:03 -05:00
Paul Beckingham
0cf18f3b16 Lexer2
- Integrated Lexer2 in place of Lexer. Tests fail.
2015-02-22 13:52:14 -05:00
Paul Beckingham
6626207ad1 TW-1522
- TW-1522 Date format doesn't like hyphens (thanks to Scott Carter).
2015-01-25 14:49:02 -05:00
Paul Beckingham
b7ad091d00 Updated copyright to 2015 2015-01-01 00:00:41 -05:00
Paul Beckingham
06319711f1 Quoting
- Removed automatic dequoting by the Lexer.
- Implemented Lexer::dequote for manual control.
- Variant dequotes string values when appropriate.
- Fixed some unit tests that became wrong.
2014-11-18 00:59:52 -05:00
Paul Beckingham
38359b779a Lexer
- Exposed more primitives as static methods.
2014-11-04 22:51:25 -05:00
Paul Beckingham
51f08496b5 Lexer
- When parsing '\o/' the state Lexer::typeIdentifierEscape had no exit but a
  successful outcome, and looped.
- Fixed test.
2014-10-16 21:22:34 -04:00
Paul Beckingham
aab23692f1 Lexer
- Added a new type Lexer::typeTag.
2014-09-07 01:17:48 -04:00
Paul Beckingham
32566c9844 Bug
- Segfault when 'project:android' is split into 'and' and 'roid' (thanks to
  Richard Boß).
2014-07-07 21:43:09 -04:00
Paul Beckingham
3716e61259 L10N
- Localized Lexer.
2014-07-05 11:11:19 -04:00
Paul Beckingham
9778100d29 Lexer
- When parsing two-character operators ('or') from a string ('ordinary'), the
  lack of boundary between the 'r' and the 'd' now prevents the operator 'or'
  from being recognized.
2014-07-03 16:26:17 -04:00
Paul Beckingham
0c0e36993d Date
- Added a new date parse method that indicates the length of the parsed item,
  and does not require Nibbler::depletion.
2014-06-29 22:16:44 -04:00
Paul Beckingham
be6ec99fcb Lexer
- Shut off the depletion requiremeent for lexed dates.
2014-06-29 11:36:51 -04:00
Paul Beckingham
5150e49f67 Lexer
- Legacy date parsing should not permis ISO and epoch inputs.
2014-06-29 09:55:31 -04:00
Paul Beckingham
65f979cb4f Lexer
- Refactored (step 1) the ISO and Legacy date/duration parsing for lexer state
  machine breakout.
2014-06-29 09:36:27 -04:00
Paul Beckingham
008ba6ecab Lexer
- Implmented boundary detection hints.
2014-06-18 17:45:25 -04:00
Paul Beckingham
a2936c045c Lexer
- Now recognized the exact inequality operator !==.
2014-06-14 16:03:07 -04:00
Paul Beckingham
7d4e166277 Lexer
- Implemented an overload of ::token_split that preserveѕ types.
2014-06-14 13:46:10 -04:00
Paul Beckingham
2554b29041 Lexer
- Needed a shift counter, rather than a read counter, as ::token was
  lexing '-10d' into '-' and '-10d', which when evaluated is '--10d',
  which yields 10d.
2014-06-10 15:42:21 -04:00
Paul Beckingham
11a4c58735 Lexer
- Factored out an invariant.
2014-06-09 18:02:34 -04:00
Paul Beckingham
4041639e36 Lexer
- Added downgrade from typeNumber to typeIdentifer if a number is followed by a
  letter, thus supporting "9th" as an identifier. This causes much breakage.
2014-06-08 14:15:07 -04:00
Paul Beckingham
870e45be0d Lexer
- Extended Lexer to consider forms like '123.abc' as identifiers. This is
  essential to recignizing and extending DOM support.
2014-06-02 22:19:38 -04:00
Paul Beckingham
6706d1b955 Date/ISO8601
- Changed the order of parsing from ISO, Date to Date, ISO. This favors the
  old dateformat mechanism more than the new one. One day this will flip.
2014-06-01 00:10:16 -04:00
Paul Beckingham
7598997e70 Lexer
- Implemented ::token_split, which performs a full lex, and doesn't require
  white space like ::word_split does.
- Added unit tests.
2014-05-31 13:51:10 -04:00
Paul Beckingham
0af9bbdc03 Lexer
- Renamed ::split to ::word_split, for clarity and because of the need for a
  full token split, coming next.
2014-05-31 13:48:52 -04:00
Paul Beckingham
592a3bb60f Lexer
- Lexer now makes a speculative legacy dateformat parse whenever it encounters
  a decimal digit.  This assumes that rc.dateformat begins with a numeric date
  element, which is a restriction, but not a big one.
2014-05-29 18:09:11 -04:00
Paul Beckingham
dead140c74 Code Cleanup
- Removed unnecessary includes.
- Added comment for Date integration.
2014-05-28 21:03:03 -04:00
Paul Beckingham
b06ac68248 Lexer
- When a Lexer::typeIdentifier is found, it can be compared to a list of
  other tokens, with the possibility of changing the type. This applies to
  tokens that are longer than the four-character lookahead in the Lexer.
  With this change, the Lexer can now identify all operators supported by Eval,
  and therefore the Lexer can be used on all Eval input expressions. This is
  because all the evaluator needs to know is the distinction between operators
  and operands.
2014-05-26 20:58:10 -04:00
Paul Beckingham
26d9be8164 Lexer
- Reduced Lexer::word to simply distinguish quoted and unquoted strings, that
  may contain escaped characters.
2014-04-24 09:06:04 -04:00
Paul Beckingham
611812007a Lexer
- Implemented Lexer::word, which is just like ::token, but does not
  understand dates, durations or operators.
- Implemented Lexer::split, which uses Lexer::word.
- Added unit tests.
2014-04-23 23:19:41 -04:00
Paul Beckingham
9bfe40fac7 Lexer, Duration
- Merged libexpr code.
2014-01-02 00:55:53 -05:00
Paul Beckingham
9bf1ec2f7c Code Cleanup
- Eliminated Lexer.
2011-07-26 00:37:49 -04:00
Paul Beckingham
ed8454c202 Expressions
- Implemented sequence --> infix converter.
- Added new Lexer code.
- Added Lexer unit tests.
2011-06-06 01:46:11 -04:00