trunk/doc/pcrecompat.3

.TH PCRECOMPAT 3
.SH NAME
PCRE - Perl-compatible regular expressions
.SH "DIFFERENCES BETWEEN PCRE AND PERL"
.rs
.sp
This document describes the differences in the ways that PCRE and Perl handle
regular expressions. The differences described here are with respect to Perl
versions 5.10 and above.
.P
1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details of what
it does have are given in the
.\" HTML <a href="pcre.html#utf8support">
.\" </a>
section on UTF-8 support
.\"
in the main
.\" HREF
\fBpcre\fP
.\"
page.
.P
2. PCRE does not allow repeat quantifiers on assertions. Perl permits them, but
they do not mean what you might think. For example, (?!a){3} does not assert
that the next three characters are not "a". It just asserts that the next
character is not "a" three times.
.P
3. Capturing subpatterns that occur inside negative lookahead assertions are
counted, but their entries in the offsets vector are never set. Perl sets its
numerical variables from any such patterns that are matched before the
assertion fails to match something (thereby succeeding), but only if the
negative lookahead assertion contains just one branch.
.P
4. Though binary zero characters are supported in the subject string, they are
not allowed in a pattern string because it is passed as a normal C string,
terminated by zero. The escape sequence \e0 can be used in the pattern to
represent a binary zero.
.P
5. The following Perl escape sequences are not supported: \el, \eu, \eL,
\eU, and \eN. In fact these are implemented by Perl's general string-handling
and are not part of its pattern matching engine. If any of these are
encountered by PCRE, an error is generated.
.P
6. The Perl escape sequences \ep, \eP, and \eX are supported only if PCRE is
built with Unicode character property support. The properties that can be
tested with \ep and \eP are limited to the general category properties such as
Lu and Nd, script names such as Greek or Han, and the derived properties Any
and L&. PCRE does support the Cs (surrogate) property, which Perl does not; the
Perl documentation says "Because Perl hides the need for the user to understand
the internal representation of Unicode characters, there is no need to
implement the somewhat messy concept of surrogates."
.P
7. PCRE implements a simpler version of \eX than Perl, which changed to make
\eX match what Unicode calls an "extended grapheme cluster". This is more 
complicated than an extended Unicode sequence, which is what PCRE matches.
.P
8. PCRE does support the \eQ...\eE escape for quoting substrings. Characters in
between are treated as literals. This is slightly different from Perl in that $
and @ are also handled as literals inside the quotes. In Perl, they cause
variable interpolation (but of course PCRE does not have variables). Note the
following examples:
.sp
    Pattern            PCRE matches      Perl matches
.sp
.\" JOIN
    \eQabc$xyz\eE        abc$xyz           abc followed by the
                                           contents of $xyz
    \eQabc\e$xyz\eE       abc\e$xyz          abc\e$xyz
    \eQabc\eE\e$\eQxyz\eE   abc$xyz           abc$xyz
.sp
The \eQ...\eE sequence is recognized both inside and outside character classes.
.P
9. Fairly obviously, PCRE does not support the (?{code}) and (??{code})
constructions. However, there is support for recursive patterns. This is not
available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE "callout"
feature allows an external function to be called during pattern matching. See
the
.\" HREF
\fBpcrecallout\fP
.\"
documentation for details.
.P
10. Subpatterns that are called recursively or as "subroutines" are always
treated as atomic groups in PCRE. This is like Python, but unlike Perl. There
is a discussion of an example that explains this in more detail in the
.\" HTML <a href="pcrepattern.html#recursiondifference">
.\" </a>
section on recursion differences from Perl
.\"
in the
.\" HREF
\fBpcrepattern\fP
.\"
page.
.P
11. There are some differences that are concerned with the settings of captured
strings when part of a pattern is repeated. For example, matching "aba" against
the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE it is set to "b".
.P
12. PCRE's handling of duplicate subpattern numbers and duplicate subpattern
names is not as general as Perl's. This is a consequence of the fact the PCRE
works internally just with numbers, using an external table to translate
between numbers and names. In particular, a pattern such as (?|(?<a>A)|(?<b)B),
where the two capturing parentheses have the same number but different names,
is not supported, and causes an error at compile time. If it were allowed, it
would not be possible to distinguish which parentheses matched, because both
names map to capturing subpattern number 1. To avoid this confusing situation,
an error is given at compile time.
.P
13. Perl recognizes comments in some places that PCRE doesn't, for example,
between the ( and ? at the start of a subpattern.
.P
14. PCRE provides some extensions to the Perl regular expression facilities.
Perl 5.10 includes new features that are not in earlier versions of Perl, some
of which (such as named parentheses) have been in PCRE for some time. This list
is with respect to Perl 5.10:
.sp
(a) Although lookbehind assertions in PCRE must match fixed length strings,
each alternative branch of a lookbehind assertion can match a different length
of string. Perl requires them all to have the same length.
.sp
(b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $
meta-character matches only at the very end of the string.
.sp
(c) If PCRE_EXTRA is set, a backslash followed by a letter with no special
meaning is faulted. Otherwise, like Perl, the backslash is quietly ignored.
(Perl can be made to issue a warning.)
.sp
(d) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is
inverted, that is, by default they are not greedy, but if followed by a
question mark they are.
.sp
(e) PCRE_ANCHORED can be used at matching time to force a pattern to be tried
only at the first matching position in the subject string.
.sp
(f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, and
PCRE_NO_AUTO_CAPTURE options for \fBpcre_exec()\fP have no Perl equivalents.
.sp
(g) The \eR escape sequence can be restricted to match only CR, LF, or CRLF
by the PCRE_BSR_ANYCRLF option.
.sp
(h) The callout facility is PCRE-specific.
.sp
(i) The partial matching facility is PCRE-specific.
.sp
(j) Patterns compiled by PCRE can be saved and re-used at a later time, even on
different hosts that have the other endianness.
.sp
(k) The alternative matching function (\fBpcre_dfa_exec()\fP) matches in a
different way and is not Perl-compatible.
.sp
(l) PCRE recognizes some special sequences such as (*CR) at the start of
a pattern that set overall options that cannot be changed within the pattern.
.
.
.SH AUTHOR
.rs
.sp
.nf
Philip Hazel
University Computing Service
Cambridge CB2 3QH, England.
.fi
.
.
.SH REVISION
.rs
.sp
.nf
Last updated: 20 July 2011
Copyright (c) 1997-2011 University of Cambridge.
.fi
1	nigel	79	.TH PCRECOMPAT 3
2	nigel	63	.SH NAME
3			PCRE - Perl-compatible regular expressions
4	nigel	75	.SH "DIFFERENCES BETWEEN PCRE AND PERL"
5	nigel	63	.rs
6			.sp
7			This document describes the differences in the ways that PCRE and Perl handle
8	ph10	456	regular expressions. The differences described here are with respect to Perl
9	ph10	562	versions 5.10 and above.
10	nigel	75	.P
11	nigel	87	1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details of what
12			it does have are given in the
13	nigel	73	.\" HTML <a href="pcre.html#utf8support">
14			.\" </a>
15			section on UTF-8 support
16			.\"
17			in the main
18			.\" HREF
19	nigel	75	\fBpcre\fP
20	nigel	73	.\"
21			page.
22	nigel	75	.P
23	ph10	596	2. PCRE does not allow repeat quantifiers on assertions. Perl permits them, but
24			they do not mean what you might think. For example, (?!a){3} does not assert
25			that the next three characters are not "a". It just asserts that the next
26			character is not "a" three times.
27	nigel	75	.P
28	nigel	73	3. Capturing subpatterns that occur inside negative lookahead assertions are
29	nigel	63	counted, but their entries in the offsets vector are never set. Perl sets its
30			numerical variables from any such patterns that are matched before the
31			assertion fails to match something (thereby succeeding), but only if the
32			negative lookahead assertion contains just one branch.
33	nigel	75	.P
34	nigel	73	4. Though binary zero characters are supported in the subject string, they are
35	nigel	63	not allowed in a pattern string because it is passed as a normal C string,
36	nigel	75	terminated by zero. The escape sequence \e0 can be used in the pattern to
37	nigel	63	represent a binary zero.
38	nigel	75	.P
39			5. The following Perl escape sequences are not supported: \el, \eu, \eL,
40			\eU, and \eN. In fact these are implemented by Perl's general string-handling
41			and are not part of its pattern matching engine. If any of these are
42			encountered by PCRE, an error is generated.
43			.P
44			6. The Perl escape sequences \ep, \eP, and \eX are supported only if PCRE is
45			built with Unicode character property support. The properties that can be
46			tested with \ep and \eP are limited to the general category properties such as
47	nigel	87	Lu and Nd, script names such as Greek or Han, and the derived properties Any
48	ph10	450	and L&. PCRE does support the Cs (surrogate) property, which Perl does not; the
49			Perl documentation says "Because Perl hides the need for the user to understand
50			the internal representation of Unicode characters, there is no need to
51			implement the somewhat messy concept of surrogates."
52	nigel	75	.P
53	ph10	628	7. PCRE implements a simpler version of \eX than Perl, which changed to make
54			\eX match what Unicode calls an "extended grapheme cluster". This is more
55			complicated than an extended Unicode sequence, which is what PCRE matches.
56			.P
57			8. PCRE does support the \eQ...\eE escape for quoting substrings. Characters in
58	nigel	63	between are treated as literals. This is slightly different from Perl in that $
59			and @ are also handled as literals inside the quotes. In Perl, they cause
60			variable interpolation (but of course PCRE does not have variables). Note the
61			following examples:
62	nigel	75	.sp
63	nigel	63	Pattern PCRE matches Perl matches
64	nigel	75	.sp
65			.\" JOIN
66			\eQabc$xyz\eE abc$xyz abc followed by the
67	nigel	63	contents of $xyz
68	nigel	75	\eQabc\e$xyz\eE abc\e$xyz abc\e$xyz
69			\eQabc\eE\e$\eQxyz\eE abc$xyz abc$xyz
70			.sp
71			The \eQ...\eE sequence is recognized both inside and outside character classes.
72			.P
73	ph10	628	9. Fairly obviously, PCRE does not support the (?{code}) and (??{code})
74	nigel	93	constructions. However, there is support for recursive patterns. This is not
75	ph10	453	available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE "callout"
76	nigel	93	feature allows an external function to be called during pattern matching. See
77			the
78	nigel	75	.\" HREF
79			\fBpcrecallout\fP
80			.\"
81			documentation for details.
82			.P
83	ph10	628	10. Subpatterns that are called recursively or as "subroutines" are always
84	ph10	461	treated as atomic groups in PCRE. This is like Python, but unlike Perl. There
85	ph10	453	is a discussion of an example that explains this in more detail in the
86			.\" HTML <a href="pcrepattern.html#recursiondifference">
87			.\" </a>
88			section on recursion differences from Perl
89			.\"
90			in the
91			.\" HREF
92	ph10	456	\fBpcrepattern\fP
93	ph10	453	.\"
94			page.
95	nigel	93	.P
96	ph10	628	11. There are some differences that are concerned with the settings of captured
97	nigel	63	strings when part of a pattern is repeated. For example, matching "aba" against
98			the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE it is set to "b".
99	nigel	75	.P
100	ph10	628	12. PCRE's handling of duplicate subpattern numbers and duplicate subpattern
101	ph10	459	names is not as general as Perl's. This is a consequence of the fact the PCRE
102			works internally just with numbers, using an external table to translate
103			between numbers and names. In particular, a pattern such as (?\|(?<a>A)\|(?<b)B),
104			where the two capturing parentheses have the same number but different names,
105			is not supported, and causes an error at compile time. If it were allowed, it
106			would not be possible to distinguish which parentheses matched, because both
107			names map to capturing subpattern number 1. To avoid this confusing situation,
108			an error is given at compile time.
109	ph10	456	.P
110	ph10	628	13. Perl recognizes comments in some places that PCRE doesn't, for example,
111	ph10	562	between the ( and ? at the start of a subpattern.
112			.P
113	ph10	628	14. PCRE provides some extensions to the Perl regular expression facilities.
114	ph10	456	Perl 5.10 includes new features that are not in earlier versions of Perl, some
115			of which (such as named parentheses) have been in PCRE for some time. This list
116			is with respect to Perl 5.10:
117			.sp
118			(a) Although lookbehind assertions in PCRE must match fixed length strings,
119			each alternative branch of a lookbehind assertion can match a different length
120			of string. Perl requires them all to have the same length.
121			.sp
122	nigel	63	(b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $
123			meta-character matches only at the very end of the string.
124	nigel	75	.sp
125	nigel	63	(c) If PCRE_EXTRA is set, a backslash followed by a letter with no special
126	ph10	181	meaning is faulted. Otherwise, like Perl, the backslash is quietly ignored.
127			(Perl can be made to issue a warning.)
128	nigel	75	.sp
129	nigel	63	(d) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is
130			inverted, that is, by default they are not greedy, but if followed by a
131			question mark they are.
132	nigel	75	.sp
133			(e) PCRE_ANCHORED can be used at matching time to force a pattern to be tried
134			only at the first matching position in the subject string.
135			.sp
136	ph10	442	(f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, and
137			PCRE_NO_AUTO_CAPTURE options for \fBpcre_exec()\fP have no Perl equivalents.
138	nigel	75	.sp
139	ph10	231	(g) The \eR escape sequence can be restricted to match only CR, LF, or CRLF
140			by the PCRE_BSR_ANYCRLF option.
141	nigel	75	.sp
142	ph10	231	(h) The callout facility is PCRE-specific.
143	nigel	75	.sp
144	ph10	231	(i) The partial matching facility is PCRE-specific.
145			.sp
146			(j) Patterns compiled by PCRE can be saved and re-used at a later time, even on
147	nigel	75	different hosts that have the other endianness.
148	nigel	77	.sp
149	ph10	231	(k) The alternative matching function (\fBpcre_dfa_exec()\fP) matches in a
150	nigel	77	different way and is not Perl-compatible.
151	ph10	231	.sp
152			(l) PCRE recognizes some special sequences such as (*CR) at the start of
153			a pattern that set overall options that cannot be changed within the pattern.
154	ph10	99	.
155			.
156			.SH AUTHOR
157			.rs
158			.sp
159			.nf
160			Philip Hazel
161			University Computing Service
162			Cambridge CB2 3QH, England.
163			.fi
164			.
165			.
166			.SH REVISION
167			.rs
168			.sp
169			.nf
170	ph10	628	Last updated: 20 July 2011
171	ph10	596	Copyright (c) 1997-2011 University of Cambridge.
172	ph10	99	.fi
Name	Value
svn:eol-style	native
svn:keywords	"Author Date Id Revision Url"