Regular Expressions

Last update: 2022-10-11 18:31:04 +0800 +0800
Available translations: 日本語 / 中文(简体) / 中文(繁體)

Regular expressions are not one language. POSIX BRE, POSIX ERE, and Perl-compatible engines overlap, but they differ in syntax, matching rules, and available features.

POSIX BRE #

Basic regular expressions are the historical syntax used by grep and sed.

Important rules:

  • grouping uses \( and \);
  • counted repetition uses \{m,n\};
  • +, ?, |, and () are literal characters unless the tool adds extensions;
  • backreferences use \1, \2, and so on.

BRE is portable, but it is verbose because many metacharacters need escaping.

POSIX ERE #

Extended regular expressions are closer to modern regex syntax:

  • () groups;
  • + means one or more;
  • ? means zero or one;
  • {m,n} counts repetitions;
  • | alternates.

grep -E and sed -E use this form. POSIX ERE is still limited, because it does not support lazy quantifiers, lookahead, or lookbehind.

Perl-Compatible Engines #

Perl-compatible regexes add features that POSIX REs do not have:

  • lazy quantifiers such as *? and +?;
  • lookahead and lookbehind;
  • noncapturing groups such as (?:...);
  • named captures in many engines;
  • richer character-class syntax.

That is why a non-greedy match often forces you to switch engines. POSIX BRE and ERE simply do not have an equivalent operator.

Matching Behavior #

POSIX engines are defined around leftmost-longest matching. That rule can produce results that differ from Perl-style engines even when the syntax looks similar.

This matters when alternation and repetition overlap, because the engine may prefer a longer match instead of the first syntactic alternative that succeeds.

Anchors also depend on the engine and mode. POSIX ^ and $ describe the start and end of a string, with tool-specific behavior for lines; PCRE-style engines often provide separate absolute-string anchors and multiline modes. Match a literal input boundary explicitly instead of assuming a command’s default.

Character classes are not fixed ASCII shorthands. POSIX bracket classes such as [[:alpha:]] follow the active locale, while \w, case folding, and Unicode properties vary among Perl-compatible implementations and flags. Set a known locale for byte-oriented automation or select an engine’s Unicode mode deliberately.

Backtracking engines can take exponential time on ambiguous nested repetition, such as (a+)+$ against a long nonmatching input. Bound input size, avoid overlapping alternatives and nested quantifiers, or use a linear-time engine when the pattern processes untrusted data. Lookaround and noncapturing groups improve expression structure but do not make engines mutually compatible.

Practical Translation #

The same intent often needs different syntax:

  • a capturing group: \(...\) in BRE, (...) in ERE;
  • one-or-more digits: \{1,\} in BRE, + in ERE;
  • non-greedy wildcard: unavailable in POSIX REs.

When portability matters, choose a specific engine and use the smallest feature set that fits the task. Translate from intended language and match semantics, not by mechanically adding or removing backslashes.

Comments

Read or comment on GitHub (sign-in required to comment)