trunk/doc/pcrecompat.3

.TH PCRECOMPAT 3
.SH NAME
PCRE - Perl-compatible regular expressions
.SH "DIFFERENCES BETWEEN PCRE AND PERL"
.rs
.sp
This document describes the differences in the ways that PCRE and Perl handle
regular expressions. The differences described here are with respect to Perl
versions 5.10 and above.
.P
1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details of what
it does have are given in the
.\" HTML <a href="pcre.html#utf8support">
.\" </a>
section on UTF-8 support
.\"
in the main
.\" HREF
\fBpcre\fP
.\"
page.
.P
2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl permits
them, but they do not mean what you might think. For example, (?!a){3} does
not assert that the next three characters are not "a". It just asserts that the
next character is not "a" three times.
.P
3. Capturing subpatterns that occur inside negative lookahead assertions are
counted, but their entries in the offsets vector are never set. Perl sets its
numerical variables from any such patterns that are matched before the
assertion fails to match something (thereby succeeding), but only if the
negative lookahead assertion contains just one branch.
.P
4. Though binary zero characters are supported in the subject string, they are
not allowed in a pattern string because it is passed as a normal C string,
terminated by zero. The escape sequence \e0 can be used in the pattern to
represent a binary zero.
.P
5. The following Perl escape sequences are not supported: \el, \eu, \eL,
\eU, and \eN. In fact these are implemented by Perl's general string-handling
and are not part of its pattern matching engine. If any of these are
encountered by PCRE, an error is generated.
.P
6. The Perl escape sequences \ep, \eP, and \eX are supported only if PCRE is
built with Unicode character property support. The properties that can be
tested with \ep and \eP are limited to the general category properties such as
Lu and Nd, script names such as Greek or Han, and the derived properties Any
and L&. PCRE does support the Cs (surrogate) property, which Perl does not; the
Perl documentation says "Because Perl hides the need for the user to understand
the internal representation of Unicode characters, there is no need to
implement the somewhat messy concept of surrogates."
.P
7. PCRE does support the \eQ...\eE escape for quoting substrings. Characters in
between are treated as literals. This is slightly different from Perl in that $
and @ are also handled as literals inside the quotes. In Perl, they cause
variable interpolation (but of course PCRE does not have variables). Note the
following examples:
.sp
    Pattern            PCRE matches      Perl matches
.sp
.\" JOIN
    \eQabc$xyz\eE        abc$xyz           abc followed by the
                                           contents of $xyz
    \eQabc\e$xyz\eE       abc\e$xyz          abc\e$xyz
    \eQabc\eE\e$\eQxyz\eE   abc$xyz           abc$xyz
.sp
The \eQ...\eE sequence is recognized both inside and outside character classes.
.P
8. Fairly obviously, PCRE does not support the (?{code}) and (??{code})
constructions. However, there is support for recursive patterns. This is not
available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE "callout"
feature allows an external function to be called during pattern matching. See
the
.\" HREF
\fBpcrecallout\fP
.\"
documentation for details.
.P
9. Subpatterns that are called recursively or as "subroutines" are always
treated as atomic groups in PCRE. This is like Python, but unlike Perl. There
is a discussion of an example that explains this in more detail in the
.\" HTML <a href="pcrepattern.html#recursiondifference">
.\" </a>
section on recursion differences from Perl
.\"
in the
.\" HREF
\fBpcrepattern\fP
.\"
page.
.P
10. There are some differences that are concerned with the settings of captured
strings when part of a pattern is repeated. For example, matching "aba" against
the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE it is set to "b".
.P
11. PCRE's handling of duplicate subpattern numbers and duplicate subpattern
names is not as general as Perl's. This is a consequence of the fact the PCRE
works internally just with numbers, using an external table to translate
between numbers and names. In particular, a pattern such as (?|(?<a>A)|(?<b)B),
where the two capturing parentheses have the same number but different names,
is not supported, and causes an error at compile time. If it were allowed, it
would not be possible to distinguish which parentheses matched, because both
names map to capturing subpattern number 1. To avoid this confusing situation,
an error is given at compile time.
.P
12. Perl recognizes comments in some places that PCRE doesn't, for example, 
between the ( and ? at the start of a subpattern.
.P
13. PCRE provides some extensions to the Perl regular expression facilities.
Perl 5.10 includes new features that are not in earlier versions of Perl, some
of which (such as named parentheses) have been in PCRE for some time. This list
is with respect to Perl 5.10:
.sp
(a) Although lookbehind assertions in PCRE must match fixed length strings,
each alternative branch of a lookbehind assertion can match a different length
of string. Perl requires them all to have the same length.
.sp
(b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $
meta-character matches only at the very end of the string.
.sp
(c) If PCRE_EXTRA is set, a backslash followed by a letter with no special
meaning is faulted. Otherwise, like Perl, the backslash is quietly ignored.
(Perl can be made to issue a warning.)
.sp
(d) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is
inverted, that is, by default they are not greedy, but if followed by a
question mark they are.
.sp
(e) PCRE_ANCHORED can be used at matching time to force a pattern to be tried
only at the first matching position in the subject string.
.sp
(f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, and
PCRE_NO_AUTO_CAPTURE options for \fBpcre_exec()\fP have no Perl equivalents.
.sp
(g) The \eR escape sequence can be restricted to match only CR, LF, or CRLF
by the PCRE_BSR_ANYCRLF option.
.sp
(h) The callout facility is PCRE-specific.
.sp
(i) The partial matching facility is PCRE-specific.
.sp
(j) Patterns compiled by PCRE can be saved and re-used at a later time, even on
different hosts that have the other endianness.
.sp
(k) The alternative matching function (\fBpcre_dfa_exec()\fP) matches in a
different way and is not Perl-compatible.
.sp
(l) PCRE recognizes some special sequences such as (*CR) at the start of
a pattern that set overall options that cannot be changed within the pattern.
.
.
.SH AUTHOR
.rs
.sp
.nf
Philip Hazel
University Computing Service
Cambridge CB2 3QH, England.
.fi
.
.
.SH REVISION
.rs
.sp
.nf
Last updated: 31 October 2010
Copyright (c) 1997-2010 University of Cambridge.
.fi
1	.TH PCRECOMPAT 3
2	.SH NAME
3	PCRE - Perl-compatible regular expressions
4	.SH "DIFFERENCES BETWEEN PCRE AND PERL"
5	.rs
6	.sp
7	This document describes the differences in the ways that PCRE and Perl handle
8	regular expressions. The differences described here are with respect to Perl
9	versions 5.10 and above.
10	.P
11	1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details of what
12	it does have are given in the
13	.\" HTML <a href="pcre.html#utf8support">
14	.\" </a>
15	section on UTF-8 support
16	.\"
17	in the main
18	.\" HREF
19	\fBpcre\fP
20	.\"
21	page.
22	.P
23	2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl permits
24	them, but they do not mean what you might think. For example, (?!a){3} does
25	not assert that the next three characters are not "a". It just asserts that the
26	next character is not "a" three times.
27	.P
28	3. Capturing subpatterns that occur inside negative lookahead assertions are
29	counted, but their entries in the offsets vector are never set. Perl sets its
30	numerical variables from any such patterns that are matched before the
31	assertion fails to match something (thereby succeeding), but only if the
32	negative lookahead assertion contains just one branch.
33	.P
34	4. Though binary zero characters are supported in the subject string, they are
35	not allowed in a pattern string because it is passed as a normal C string,
36	terminated by zero. The escape sequence \e0 can be used in the pattern to
37	represent a binary zero.
38	.P
39	5. The following Perl escape sequences are not supported: \el, \eu, \eL,
40	\eU, and \eN. In fact these are implemented by Perl's general string-handling
41	and are not part of its pattern matching engine. If any of these are
42	encountered by PCRE, an error is generated.
43	.P
44	6. The Perl escape sequences \ep, \eP, and \eX are supported only if PCRE is
45	built with Unicode character property support. The properties that can be
46	tested with \ep and \eP are limited to the general category properties such as
47	Lu and Nd, script names such as Greek or Han, and the derived properties Any
48	and L&. PCRE does support the Cs (surrogate) property, which Perl does not; the
49	Perl documentation says "Because Perl hides the need for the user to understand
50	the internal representation of Unicode characters, there is no need to
51	implement the somewhat messy concept of surrogates."
52	.P
53	7. PCRE does support the \eQ...\eE escape for quoting substrings. Characters in
54	between are treated as literals. This is slightly different from Perl in that $
55	and @ are also handled as literals inside the quotes. In Perl, they cause
56	variable interpolation (but of course PCRE does not have variables). Note the
57	following examples:
58	.sp
59	Pattern PCRE matches Perl matches
60	.sp
61	.\" JOIN
62	\eQabc$xyz\eE abc$xyz abc followed by the
63	contents of $xyz
64	\eQabc\e$xyz\eE abc\e$xyz abc\e$xyz
65	\eQabc\eE\e$\eQxyz\eE abc$xyz abc$xyz
66	.sp
67	The \eQ...\eE sequence is recognized both inside and outside character classes.
68	.P
69	8. Fairly obviously, PCRE does not support the (?{code}) and (??{code})
70	constructions. However, there is support for recursive patterns. This is not
71	available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE "callout"
72	feature allows an external function to be called during pattern matching. See
73	the
74	.\" HREF
75	\fBpcrecallout\fP
76	.\"
77	documentation for details.
78	.P
79	9. Subpatterns that are called recursively or as "subroutines" are always
80	treated as atomic groups in PCRE. This is like Python, but unlike Perl. There
81	is a discussion of an example that explains this in more detail in the
82	.\" HTML <a href="pcrepattern.html#recursiondifference">
83	.\" </a>
84	section on recursion differences from Perl
85	.\"
86	in the
87	.\" HREF
88	\fBpcrepattern\fP
89	.\"
90	page.
91	.P
92	10. There are some differences that are concerned with the settings of captured
93	strings when part of a pattern is repeated. For example, matching "aba" against
94	the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE it is set to "b".
95	.P
96	11. PCRE's handling of duplicate subpattern numbers and duplicate subpattern
97	names is not as general as Perl's. This is a consequence of the fact the PCRE
98	works internally just with numbers, using an external table to translate
99	between numbers and names. In particular, a pattern such as (?\|(?<a>A)\|(?<b)B),
100	where the two capturing parentheses have the same number but different names,
101	is not supported, and causes an error at compile time. If it were allowed, it
102	would not be possible to distinguish which parentheses matched, because both
103	names map to capturing subpattern number 1. To avoid this confusing situation,
104	an error is given at compile time.
105	.P
106	12. Perl recognizes comments in some places that PCRE doesn't, for example,
107	between the ( and ? at the start of a subpattern.
108	.P
109	13. PCRE provides some extensions to the Perl regular expression facilities.
110	Perl 5.10 includes new features that are not in earlier versions of Perl, some
111	of which (such as named parentheses) have been in PCRE for some time. This list
112	is with respect to Perl 5.10:
113	.sp
114	(a) Although lookbehind assertions in PCRE must match fixed length strings,
115	each alternative branch of a lookbehind assertion can match a different length
116	of string. Perl requires them all to have the same length.
117	.sp
118	(b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $
119	meta-character matches only at the very end of the string.
120	.sp
121	(c) If PCRE_EXTRA is set, a backslash followed by a letter with no special
122	meaning is faulted. Otherwise, like Perl, the backslash is quietly ignored.
123	(Perl can be made to issue a warning.)
124	.sp
125	(d) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is
126	inverted, that is, by default they are not greedy, but if followed by a
127	question mark they are.
128	.sp
129	(e) PCRE_ANCHORED can be used at matching time to force a pattern to be tried
130	only at the first matching position in the subject string.
131	.sp
132	(f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, and
133	PCRE_NO_AUTO_CAPTURE options for \fBpcre_exec()\fP have no Perl equivalents.
134	.sp
135	(g) The \eR escape sequence can be restricted to match only CR, LF, or CRLF
136	by the PCRE_BSR_ANYCRLF option.
137	.sp
138	(h) The callout facility is PCRE-specific.
139	.sp
140	(i) The partial matching facility is PCRE-specific.
141	.sp
142	(j) Patterns compiled by PCRE can be saved and re-used at a later time, even on
143	different hosts that have the other endianness.
144	.sp
145	(k) The alternative matching function (\fBpcre_dfa_exec()\fP) matches in a
146	different way and is not Perl-compatible.
147	.sp
148	(l) PCRE recognizes some special sequences such as (*CR) at the start of
149	a pattern that set overall options that cannot be changed within the pattern.
150	.
151	.
152	.SH AUTHOR
153	.rs
154	.sp
155	.nf
156	Philip Hazel
157	University Computing Service
158	Cambridge CB2 3QH, England.
159	.fi
160	.
161	.
162	.SH REVISION
163	.rs
164	.sp
165	.nf
166	Last updated: 31 October 2010
167	Copyright (c) 1997-2010 University of Cambridge.
168	.fi
Name	Value
svn:eol-style	native
svn:keywords	"Author Date Id Revision Url"