trunk/doc/pcre.3

.TH PCRE 3 "13 May 2013" "PCRE 8.33"
.SH NAME
PCRE - Perl-compatible regular expressions
.SH INTRODUCTION
.rs
.sp
The PCRE library is a set of functions that implement regular expression
pattern matching using the same syntax and semantics as Perl, with just a few
differences. Some features that appeared in Python and PCRE before they
appeared in Perl are also available using the Python syntax, there is some
support for one or two .NET and Oniguruma syntax items, and there is an option
for requesting some minor changes that give better JavaScript compatibility.
.P
Starting with release 8.30, it is possible to compile two separate PCRE
libraries: the original, which supports 8-bit character strings (including
UTF-8 strings), and a second library that supports 16-bit character strings
(including UTF-16 strings). The build process allows either one or both to be
built. The majority of the work to make this possible was done by Zoltan
Herczeg.
.P
Starting with release 8.32 it is possible to compile a third separate PCRE
library that supports 32-bit character strings (including UTF-32 strings). The
build process allows any combination of the 8-, 16- and 32-bit libraries. The
work to make this possible was done by Christian Persch.
.P
The three libraries contain identical sets of functions, except that the names
in the 16-bit library start with \fBpcre16_\fP instead of \fBpcre_\fP, and the
names in the 32-bit library start with \fBpcre32_\fP instead of \fBpcre_\fP. To
avoid over-complication and reduce the documentation maintenance load, most of
the documentation describes the 8-bit library, with the differences for the
16-bit and 32-bit libraries described separately in the
.\" HREF
\fBpcre16\fP
and
.\" HREF
\fBpcre32\fP
.\"
pages. References to functions or structures of the form \fIpcre[16|32]_xxx\fP
should be read as meaning "\fIpcre_xxx\fP when using the 8-bit library,
\fIpcre16_xxx\fP when using the 16-bit library, or \fIpcre32_xxx\fP when using
the 32-bit library".
.P
The current implementation of PCRE corresponds approximately with Perl 5.12,
including support for UTF-8/16/32 encoded strings and Unicode general category
properties. However, UTF-8/16/32 and Unicode support has to be explicitly
enabled; it is not the default. The Unicode tables correspond to Unicode
release 6.2.0.
.P
In addition to the Perl-compatible matching function, PCRE contains an
alternative function that matches the same compiled patterns in a different
way. In certain circumstances, the alternative function has some advantages.
For a discussion of the two matching algorithms, see the
.\" HREF
\fBpcrematching\fP
.\"
page.
.P
PCRE is written in C and released as a C library. A number of people have
written wrappers and interfaces of various kinds. In particular, Google Inc.
have provided a comprehensive C++ wrapper for the 8-bit library. This is now
included as part of the PCRE distribution. The
.\" HREF
\fBpcrecpp\fP
.\"
page has details of this interface. Other people's contributions can be found
in the \fIContrib\fP directory at the primary FTP site, which is:
.sp
.\" HTML <a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre">
.\" </a>
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre
.\"
.P
Details of exactly which Perl regular expression features are and are not
supported by PCRE are given in separate documents. See the
.\" HREF
\fBpcrepattern\fP
.\"
and
.\" HREF
\fBpcrecompat\fP
.\"
pages. There is a syntax summary in the
.\" HREF
\fBpcresyntax\fP
.\"
page.
.P
Some features of PCRE can be included, excluded, or changed when the library is
built. The
.\" HREF
\fBpcre_config()\fP
.\"
function makes it possible for a client to discover which features are
available. The features themselves are described in the
.\" HREF
\fBpcrebuild\fP
.\"
page. Documentation about building PCRE for various operating systems can be
found in the 
.\" HTML <a href="README.txt">
.\" </a>
\fBREADME\fP 
.\"
and 
.\" HTML <a href="NON-AUTOTOOLS-BUILD.txt">
.\" </a>
\fBNON-AUTOTOOLS_BUILD\fP 
.\"
files in the source distribution.
.P
The libraries contains a number of undocumented internal functions and data
tables that are used by more than one of the exported external functions, but
which are not intended for use by external callers. Their names all begin with
"_pcre_" or "_pcre16_" or "_pcre32_", which hopefully will not provoke any name
clashes. In some environments, it is possible to control which external symbols
are exported when a shared library is built, and in these cases the
undocumented symbols are not exported.
.
.
.SH "SECURITY CONSIDERATIONS"
.rs
.sp
If you are using PCRE in a non-UTF application that permits users to supply
arbitrary patterns for compilation, you should be aware of a feature that
allows users to turn on UTF support from within a pattern, provided that PCRE
was built with UTF support. For example, an 8-bit pattern that begins with
"(*UTF8)" or "(*UTF)" turns on UTF-8 mode, which interprets patterns and
subjects as strings of UTF-8 characters instead of individual 8-bit characters.
This causes both the pattern and any data against which it is matched to be
checked for UTF-8 validity. If the data string is very long, such a check might
use sufficiently many resources as to cause your application to lose
performance.
.P
One way of guarding against this possibility is to use the
\fBpcre_fullinfo()\fP function to check the compiled pattern's options for UTF.
Alternatively, from release 8.33, you can set the PCRE_NEVER_UTF option at
compile time. This causes an compile time error if a pattern contains a
UTF-setting sequence.
.P
If your application is one that supports UTF, be aware that validity checking
can take time. If the same data string is to be matched many times, you can use
the PCRE_NO_UTF[8|16|32]_CHECK option for the second and subsequent matches to
save redundant checks.
.P
Another way that performance can be hit is by running a pattern that has a very
large search tree against a string that will never match. Nested unlimited
repeats in a pattern are a common example. PCRE provides some protection
against this: see the PCRE_EXTRA_MATCH_LIMIT feature in the
.\" HREF
\fBpcreapi\fP
.\"
page.
.
.
.SH "USER DOCUMENTATION"
.rs
.sp
The user documentation for PCRE comprises a number of different sections. In
the "man" format, each of these is a separate "man page". In the HTML format,
each is a separate page, linked from the index page. In the plain text format,
all the sections, except the \fBpcredemo\fP section, are concatenated, for ease
of searching. The sections are as follows:
.sp
  pcre              this document
  pcre-config       show PCRE installation configuration information
  pcre16            details of the 16-bit library
  pcre32            details of the 32-bit library
  pcreapi           details of PCRE's native C API
  pcrebuild         building PCRE
  pcrecallout       details of the callout feature
  pcrecompat        discussion of Perl compatibility
  pcrecpp           details of the C++ wrapper for the 8-bit library
  pcredemo          a demonstration C program that uses PCRE
  pcregrep          description of the \fBpcregrep\fP command (8-bit only)
  pcrejit           discussion of the just-in-time optimization support
  pcrelimits        details of size and other limits
  pcrematching      discussion of the two matching algorithms
  pcrepartial       details of the partial matching facility
.\" JOIN
  pcrepattern       syntax and semantics of supported
                      regular expressions
  pcreperform       discussion of performance issues
  pcreposix         the POSIX-compatible C API for the 8-bit library
  pcreprecompile    details of saving and re-using precompiled patterns
  pcresample        discussion of the pcredemo program
  pcrestack         discussion of stack usage
  pcresyntax        quick syntax reference
  pcretest          description of the \fBpcretest\fP testing command
  pcreunicode       discussion of Unicode and UTF-8/16/32 support
.sp
In addition, in the "man" and HTML formats, there is a short page for each
C library function, listing its arguments and results.
.
.
.SH AUTHOR
.rs
.sp
.nf
Philip Hazel
University Computing Service
Cambridge CB2 3QH, England.
.fi
.P
Putting an actual email address here seems to have been a spam magnet, so I've
taken it away. If you want to email me, use my two initials, followed by the
two digits 10, at the domain cam.ac.uk.
.
.
.SH REVISION
.rs
.sp
.nf
Last updated: 13 May 2013
Copyright (c) 1997-2013 University of Cambridge.
.fi
1	ph10	1332	.TH PCRE 3 "13 May 2013" "PCRE 8.33"
2	nigel	41	.SH NAME
3	nigel	63	PCRE - Perl-compatible regular expressions
4	nigel	75	.SH INTRODUCTION
5	nigel	63	.rs
6			.sp
7	nigel	41	The PCRE library is a set of functions that implement regular expression
8	nigel	63	pattern matching using the same syntax and semantics as Perl, with just a few
9	ph10	456	differences. Some features that appeared in Python and PCRE before they
10			appeared in Perl are also available using the Python syntax, there is some
11			support for one or two .NET and Oniguruma syntax items, and there is an option
12			for requesting some minor changes that give better JavaScript compatibility.
13	nigel	75	.P
14	ph10	903	Starting with release 8.30, it is possible to compile two separate PCRE
15	ph10	846	libraries: the original, which supports 8-bit character strings (including
16			UTF-8 strings), and a second library that supports 16-bit character strings
17			(including UTF-16 strings). The build process allows either one or both to be
18	ph10	903	built. The majority of the work to make this possible was done by Zoltan
19	ph10	846	Herczeg.
20			.P
21	chpe	1103	Starting with release 8.32 it is possible to compile a third separate PCRE
22	ph10	1332	library that supports 32-bit character strings (including UTF-32 strings). The
23			build process allows any combination of the 8-, 16- and 32-bit libraries. The
24			work to make this possible was done by Christian Persch.
25	chpe	1055	.P
26	ph10	1191	The three libraries contain identical sets of functions, except that the names
27			in the 16-bit library start with \fBpcre16_\fP instead of \fBpcre_\fP, and the
28			names in the 32-bit library start with \fBpcre32_\fP instead of \fBpcre_\fP. To
29			avoid over-complication and reduce the documentation maintenance load, most of
30			the documentation describes the 8-bit library, with the differences for the
31			16-bit and 32-bit libraries described separately in the
32	ph10	846	.\" HREF
33			\fBpcre16\fP
34	ph10	1191	and
35	chpe	1055	.\" HREF
36			\fBpcre32\fP
37	ph10	846	.\"
38	ph10	1191	pages. References to functions or structures of the form \fIpcre[16\|32]_xxx\fP
39			should be read as meaning "\fIpcre_xxx\fP when using the 8-bit library,
40			\fIpcre16_xxx\fP when using the 16-bit library, or \fIpcre32_xxx\fP when using
41			the 32-bit library".
42	ph10	846	.P
43	ph10	572	The current implementation of PCRE corresponds approximately with Perl 5.12,
44	ph10	1191	including support for UTF-8/16/32 encoded strings and Unicode general category
45			properties. However, UTF-8/16/32 and Unicode support has to be explicitly
46			enabled; it is not the default. The Unicode tables correspond to Unicode
47			release 6.2.0.
48	nigel	93	.P
49			In addition to the Perl-compatible matching function, PCRE contains an
50	ph10	456	alternative function that matches the same compiled patterns in a different
51			way. In certain circumstances, the alternative function has some advantages.
52			For a discussion of the two matching algorithms, see the
53	nigel	77	.\" HREF
54			\fBpcrematching\fP
55			.\"
56			page.
57			.P
58	nigel	75	PCRE is written in C and released as a C library. A number of people have
59	nigel	77	written wrappers and interfaces of various kinds. In particular, Google Inc.
60	ph10	846	have provided a comprehensive C++ wrapper for the 8-bit library. This is now
61			included as part of the PCRE distribution. The
62	nigel	77	.\" HREF
63			\fBpcrecpp\fP
64			.\"
65			page has details of this interface. Other people's contributions can be found
66	ph10	583	in the \fIContrib\fP directory at the primary FTP site, which is:
67	nigel	75	.sp
68	nigel	63	.\" HTML <a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre">
69			.\" </a>
70			ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre
71	ph10	1332	.\"
72	nigel	75	.P
73	nigel	63	Details of exactly which Perl regular expression features are and are not
74			supported by PCRE are given in separate documents. See the
75			.\" HREF
76	ph10	583	\fBpcrepattern\fP
77	nigel	63	.\"
78			and
79			.\" HREF
80	ph10	583	\fBpcrecompat\fP
81	nigel	63	.\"
82	ph10	208	pages. There is a syntax summary in the
83			.\" HREF
84	ph10	583	\fBpcresyntax\fP
85	ph10	208	.\"
86			page.
87	nigel	75	.P
88	nigel	63	Some features of PCRE can be included, excluded, or changed when the library is
89			built. The
90			.\" HREF
91	ph10	583	\fBpcre_config()\fP
92	nigel	63	.\"
93			function makes it possible for a client to discover which features are
94	nigel	75	available. The features themselves are described in the
95			.\" HREF
96			\fBpcrebuild\fP
97			.\"
98			page. Documentation about building PCRE for various operating systems can be
99	ph10	1332	found in the
100			.\" HTML <a href="README.txt">
101			.\" </a>
102			\fBREADME\fP
103			.\"
104			and
105			.\" HTML <a href="NON-AUTOTOOLS-BUILD.txt">
106			.\" </a>
107			\fBNON-AUTOTOOLS_BUILD\fP
108			.\"
109			files in the source distribution.
110	nigel	77	.P
111	ph10	846	The libraries contains a number of undocumented internal functions and data
112	nigel	77	tables that are used by more than one of the exported external functions, but
113			which are not intended for use by external callers. Their names all begin with
114	chpe	1055	"_pcre_" or "_pcre16_" or "_pcre32_", which hopefully will not provoke any name
115			clashes. In some environments, it is possible to control which external symbols
116	ph10	1191	are exported when a shared library is built, and in these cases the
117			undocumented symbols are not exported.
118	nigel	75	.
119			.
120	ph10	1193	.SH "SECURITY CONSIDERATIONS"
121			.rs
122			.sp
123			If you are using PCRE in a non-UTF application that permits users to supply
124			arbitrary patterns for compilation, you should be aware of a feature that
125			allows users to turn on UTF support from within a pattern, provided that PCRE
126			was built with UTF support. For example, an 8-bit pattern that begins with
127	ph10	1221	"(UTF8)" or "(UTF)" turns on UTF-8 mode, which interprets patterns and
128	ph10	1219	subjects as strings of UTF-8 characters instead of individual 8-bit characters.
129			This causes both the pattern and any data against which it is matched to be
130			checked for UTF-8 validity. If the data string is very long, such a check might
131			use sufficiently many resources as to cause your application to lose
132			performance.
133	ph10	1193	.P
134	ph10	1314	One way of guarding against this possibility is to use the
135	ph10	1193	\fBpcre_fullinfo()\fP function to check the compiled pattern's options for UTF.
136	ph10	1314	Alternatively, from release 8.33, you can set the PCRE_NEVER_UTF option at
137			compile time. This causes an compile time error if a pattern contains a
138			UTF-setting sequence.
139	ph10	1193	.P
140			If your application is one that supports UTF, be aware that validity checking
141			can take time. If the same data string is to be matched many times, you can use
142	ph10	1221	the PCRE_NO_UTF[8\|16\|32]_CHECK option for the second and subsequent matches to
143	ph10	1193	save redundant checks.
144			.P
145	ph10	1221	Another way that performance can be hit is by running a pattern that has a very
146			large search tree against a string that will never match. Nested unlimited
147			repeats in a pattern are a common example. PCRE provides some protection
148	ph10	1193	against this: see the PCRE_EXTRA_MATCH_LIMIT feature in the
149			.\" HREF
150			\fBpcreapi\fP
151			.\"
152			page.
153			.
154			.
155	nigel	75	.SH "USER DOCUMENTATION"
156	nigel	63	.rs
157			.sp
158	nigel	75	The user documentation for PCRE comprises a number of different sections. In
159			the "man" format, each of these is a separate "man page". In the HTML format,
160			each is a separate page, linked from the index page. In the plain text format,
161	ph10	429	all the sections, except the \fBpcredemo\fP section, are concatenated, for ease
162			of searching. The sections are as follows:
163	nigel	75	.sp
164	nigel	63	pcre this document
165	ph10	1332	pcre-config show PCRE installation configuration information
166	ph10	903	pcre16 details of the 16-bit library
167	chpe	1055	pcre32 details of the 32-bit library
168	nigel	77	pcreapi details of PCRE's native C API
169	ph10	1332	pcrebuild building PCRE
170	nigel	63	pcrecallout details of the callout feature
171			pcrecompat discussion of Perl compatibility
172	ph10	846	pcrecpp details of the C++ wrapper for the 8-bit library
173	ph10	429	pcredemo a demonstration C program that uses PCRE
174	ph10	846	pcregrep description of the \fBpcregrep\fP command (8-bit only)
175	ph10	691	pcrejit discussion of the just-in-time optimization support
176			pcrelimits details of size and other limits
177	nigel	77	pcrematching discussion of the two matching algorithms
178	nigel	75	pcrepartial details of the partial matching facility
179			.\" JOIN
180	nigel	63	pcrepattern syntax and semantics of supported
181			regular expressions
182			pcreperform discussion of performance issues
183	ph10	846	pcreposix the POSIX-compatible C API for the 8-bit library
184	nigel	75	pcreprecompile details of saving and re-using precompiled patterns
185	ph10	429	pcresample discussion of the pcredemo program
186	nigel	91	pcrestack discussion of stack usage
187	ph10	456	pcresyntax quick syntax reference
188	nigel	75	pcretest description of the \fBpcretest\fP testing command
189	chpe	1055	pcreunicode discussion of Unicode and UTF-8/16/32 support
190	nigel	75	.sp
191	nigel	63	In addition, in the "man" and HTML formats, there is a short page for each
192	ph10	1191	C library function, listing its arguments and results.
193	nigel	75	.
194			.
195	nigel	41	.SH AUTHOR
196	nigel	63	.rs
197			.sp
198	ph10	99	.nf
199	nigel	77	Philip Hazel
200	ph10	99	University Computing Service
201	nigel	93	Cambridge CB2 3QH, England.
202	ph10	99	.fi
203	nigel	77	.P
204			Putting an actual email address here seems to have been a spam magnet, so I've
205	ph10	153	taken it away. If you want to email me, use my two initials, followed by the
206			two digits 10, at the domain cam.ac.uk.
207	ph10	99	.
208			.
209			.SH REVISION
210			.rs
211	nigel	75	.sp
212	ph10	99	.nf
213	ph10	1332	Last updated: 13 May 2013
214	ph10	1314	Copyright (c) 1997-2013 University of Cambridge.
215	ph10	99	.fi
Name	Value
svn:eol-style	native
svn:keywords	"Author Date Id Revision Url"