regex - Handle number followed by letters in Flex

Question

In some Flex/Lex code we have,

DIGIT       [0-9]
LETTER      [A-Za-z]
ID          {LETTER}{LETTER}|{DIGIT})*
NUM         {DIGIT}+(.{DIGIT}+)?

So, 123 matches NUM and abc123 matches ID. In case of 123abc, this matches NUM followed by and ID. However, I want this to generate an error message instead of matching.

Is there a simple way to do this?

Answer 1

You can just add an additional pattern to detect an error.

The following does not use macros, because I personally find them distracting. But the idea is the same.

[[:alpha:]][[:alnum:]]*        { return IDENTIFIER; }
[[:digit:]]+"."([[:digit:]]+)? { return NUMBER; }
[[:digit:]]+"."([[:digit:]]+)?[[:alpha:]] { return BAD_NUMBER; }

The last pattern will only match if there is a letter immediately following a NUMBER, and will override the second pattern because of the longest-match rule.

By the way, a better pattern for a number is:

[[:digit:]]+("."[[:digit:]]*)?|"."[[:digit:]]+

That will match 23. and .56, which many people expect to be valid numbers.

---

You might also find this answer interesting, particularly the examples from other programming languages. Most languages (other than C & family) do allow 123abc to be scanned as two tokens, which usually leads to a syntax error anyway, and that is both the easiest and the most maintainable solution.