/[pcre]/code/trunk/doc/pcre.txt
ViewVC logotype

Contents of /code/trunk/doc/pcre.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 461 - (hide annotations) (download)
Mon Oct 5 10:59:35 2009 UTC (4 years, 9 months ago) by ph10
File MIME type: text/plain
File size: 335891 byte(s)
Tidy up, remove trailing spaces, etc. for 8.00-RC1.

1 nigel 75 -----------------------------------------------------------------------------
2 nigel 63 This file contains a concatenation of the PCRE man pages, converted to plain
3     text format for ease of searching with a text editor, or for use on systems
4     that do not have a man page processor. The small individual files that give
5 ph10 461 synopses of each function in the library have not been included. Neither has
6 ph10 429 the pcredemo program. There are separate text files for the pcregrep and
7     pcretest commands.
8 nigel 63 -----------------------------------------------------------------------------
9    
10 nigel 41
11 nigel 79 PCRE(3) PCRE(3)
12 nigel 41
13 nigel 79
14 nigel 73 NAME
15     PCRE - Perl-compatible regular expressions
16    
17 nigel 77
18 nigel 75 INTRODUCTION
19 nigel 41
20 nigel 73 The PCRE library is a set of functions that implement regular expres-
21     sion pattern matching using the same syntax and semantics as Perl, with
22 ph10 461 just a few differences. Some features that appeared in Python and PCRE
23     before they appeared in Perl are also available using the Python syn-
24     tax, there is some support for one or two .NET and Oniguruma syntax
25     items, and there is an option for requesting some minor changes that
26     give better JavaScript compatibility.
27 nigel 63
28 ph10 461 The current implementation of PCRE corresponds approximately with Perl
29     5.10, including support for UTF-8 encoded strings and Unicode general
30     category properties. However, UTF-8 and Unicode support has to be
31     explicitly enabled; it is not the default. The Unicode tables corre-
32     spond to Unicode release 5.1.
33 nigel 77
34 nigel 93 In addition to the Perl-compatible matching function, PCRE contains an
35 ph10 461 alternative function that matches the same compiled patterns in a dif-
36     ferent way. In certain circumstances, the alternative function has some
37     advantages. For a discussion of the two matching algorithms, see the
38     pcrematching page.
39 nigel 93
40     PCRE is written in C and released as a C library. A number of people
41     have written wrappers and interfaces of various kinds. In particular,
42     Google Inc. have provided a comprehensive C++ wrapper. This is now
43 nigel 77 included as part of the PCRE distribution. The pcrecpp page has details
44 nigel 93 of this interface. Other people's contributions can be found in the
45 nigel 77 Contrib directory at the primary FTP site, which is:
46 nigel 63
47 nigel 73 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre
48 nigel 63
49 nigel 93 Details of exactly which Perl regular expression features are and are
50 nigel 73 not supported by PCRE are given in separate documents. See the pcrepat-
51 ph10 208 tern and pcrecompat pages. There is a syntax summary in the pcresyntax
52     page.
53 nigel 63
54 ph10 208 Some features of PCRE can be included, excluded, or changed when the
55     library is built. The pcre_config() function makes it possible for a
56     client to discover which features are available. The features them-
57     selves are described in the pcrebuild page. Documentation about build-
58 ph10 461 ing PCRE for various operating systems can be found in the README and
59     NON-UNIX-USE files in the source distribution.
60 nigel 63
61 ph10 208 The library contains a number of undocumented internal functions and
62     data tables that are used by more than one of the exported external
63     functions, but which are not intended for use by external callers.
64     Their names all begin with "_pcre_", which hopefully will not provoke
65 nigel 83 any name clashes. In some environments, it is possible to control which
66 ph10 208 external symbols are exported when a shared library is built, and in
67 nigel 83 these cases the undocumented symbols are not exported.
68 nigel 63
69 nigel 77
70 nigel 63 USER DOCUMENTATION
71    
72 ph10 208 The user documentation for PCRE comprises a number of different sec-
73     tions. In the "man" format, each of these is a separate "man page". In
74     the HTML format, each is a separate page, linked from the index page.
75 ph10 429 In the plain text format, all the sections, except the pcredemo sec-
76     tion, are concatenated, for ease of searching. The sections are as fol-
77     lows:
78 nigel 63
79 nigel 73 pcre this document
80 ph10 153 pcre-config show PCRE installation configuration information
81 nigel 77 pcreapi details of PCRE's native C API
82 nigel 73 pcrebuild options for building PCRE
83     pcrecallout details of the callout feature
84     pcrecompat discussion of Perl compatibility
85 nigel 77 pcrecpp details of the C++ wrapper
86 ph10 429 pcredemo a demonstration C program that uses PCRE
87 nigel 73 pcregrep description of the pcregrep command
88 nigel 77 pcrematching discussion of the two matching algorithms
89 nigel 75 pcrepartial details of the partial matching facility
90 nigel 73 pcrepattern syntax and semantics of supported
91     regular expressions
92     pcreperform discussion of performance issues
93 nigel 77 pcreposix the POSIX-compatible C API
94 nigel 75 pcreprecompile details of saving and re-using precompiled patterns
95 ph10 429 pcresample discussion of the pcredemo program
96 nigel 91 pcrestack discussion of stack usage
97 ph10 461 pcresyntax quick syntax reference
98 nigel 75 pcretest description of the pcretest testing command
99 nigel 63
100 ph10 429 In addition, in the "man" and HTML formats, there is a short page for
101 nigel 77 each C library function, listing its arguments and results.
102 nigel 63
103    
104     LIMITATIONS
105    
106 ph10 429 There are some size limitations in PCRE but it is hoped that they will
107 nigel 73 never in practice be relevant.
108 nigel 63
109 ph10 429 The maximum length of a compiled pattern is 65539 (sic) bytes if PCRE
110 nigel 73 is compiled with the default internal linkage size of 2. If you want to
111 ph10 429 process regular expressions that are truly enormous, you can compile
112     PCRE with an internal linkage size of 3 or 4 (see the README file in
113     the source distribution and the pcrebuild documentation for details).
114     In these cases the limit is substantially larger. However, the speed
115 nigel 93 of execution is slower.
116 nigel 63
117 ph10 208 All values in repeating quantifiers must be less than 65536.
118 nigel 63
119 nigel 93 There is no limit to the number of parenthesized subpatterns, but there
120     can be no more than 65535 capturing subpatterns.
121 nigel 63
122 nigel 93 The maximum length of name for a named subpattern is 32 characters, and
123     the maximum number of named subpatterns is 10000.
124 nigel 91
125 ph10 429 The maximum length of a subject string is the largest positive number
126     that an integer variable can hold. However, when using the traditional
127 nigel 77 matching function, PCRE uses recursion to handle subpatterns and indef-
128 ph10 429 inite repetition. This means that the available stack space may limit
129 nigel 77 the size of a subject string that can be processed by certain patterns.
130 nigel 91 For a discussion of stack issues, see the pcrestack documentation.
131 nigel 63
132    
133 nigel 75 UTF-8 AND UNICODE PROPERTY SUPPORT
134 nigel 63
135 ph10 429 From release 3.3, PCRE has had some support for character strings
136     encoded in the UTF-8 format. For release 4.0 this was greatly extended
137     to cover most common requirements, and in release 5.0 additional sup-
138 nigel 75 port for Unicode general category properties was added.
139 nigel 63
140 ph10 429 In order process UTF-8 strings, you must build PCRE to include UTF-8
141     support in the code, and, in addition, you must call pcre_compile()
142     with the PCRE_UTF8 option flag, or the pattern must start with the
143     sequence (*UTF8). When either of these is the case, both the pattern
144     and any subject strings that are matched against it are treated as
145 ph10 461 UTF-8 strings instead of strings of 1-byte characters.
146 nigel 63
147 ph10 429 If you compile PCRE with UTF-8 support, but do not use it at run time,
148     the library will be a bit bigger, but the additional run time overhead
149 nigel 93 is limited to testing the PCRE_UTF8 flag occasionally, so should not be
150     very big.
151 nigel 63
152 nigel 75 If PCRE is built with Unicode character property support (which implies
153 ph10 429 UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup-
154 nigel 75 ported. The available properties that can be tested are limited to the
155 ph10 429 general category properties such as Lu for an upper case letter or Nd
156     for a decimal number, the Unicode script names such as Arabic or Han,
157     and the derived properties Any and L&. A full list is given in the
158 nigel 87 pcrepattern documentation. Only the short names for properties are sup-
159 ph10 429 ported. For example, \p{L} matches a letter. Its Perl synonym, \p{Let-
160     ter}, is not supported. Furthermore, in Perl, many properties may
161     optionally be prefixed by "Is", for compatibility with Perl 5.6. PCRE
162 nigel 87 does not support this.
163 nigel 75
164 ph10 211 Validity of UTF-8 strings
165 nigel 63
166 ph10 429 When you set the PCRE_UTF8 flag, the strings passed as patterns and
167 ph10 211 subjects are (by default) checked for validity on entry to the relevant
168 ph10 429 functions. From release 7.3 of PCRE, the check is according the rules
169     of RFC 3629, which are themselves derived from the Unicode specifica-
170     tion. Earlier releases of PCRE followed the rules of RFC 2279, which
171     allows the full range of 31-bit values (0 to 0x7FFFFFFF). The current
172 ph10 211 check allows only values in the range U+0 to U+10FFFF, excluding U+D800
173     to U+DFFF.
174 nigel 63
175 ph10 429 The excluded code points are the "Low Surrogate Area" of Unicode, of
176     which the Unicode Standard says this: "The Low Surrogate Area does not
177     contain any character assignments, consequently no character code
178 ph10 211 charts or namelists are provided for this area. Surrogates are reserved
179 ph10 429 for use with UTF-16 and then must be used in pairs." The code points
180     that are encoded by UTF-16 pairs are available as independent code
181     points in the UTF-8 encoding. (In other words, the whole surrogate
182 ph10 211 thing is a fudge for UTF-16 which unfortunately messes up UTF-8.)
183    
184 ph10 429 If an invalid UTF-8 string is passed to PCRE, an error return
185 ph10 211 (PCRE_ERROR_BADUTF8) is given. In some situations, you may already know
186     that your strings are valid, and therefore want to skip these checks in
187     order to improve performance. If you set the PCRE_NO_UTF8_CHECK flag at
188 ph10 429 compile time or at run time, PCRE assumes that the pattern or subject
189     it is given (respectively) contains only valid UTF-8 codes. In this
190 ph10 211 case, it does not diagnose an invalid UTF-8 string.
191    
192 ph10 429 If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set,
193     what happens depends on why the string is invalid. If the string con-
194 ph10 211 forms to the "old" definition of UTF-8 (RFC 2279), it is processed as a
195 ph10 429 string of characters in the range 0 to 0x7FFFFFFF. In other words,
196 ph10 211 apart from the initial validity test, PCRE (when in UTF-8 mode) handles
197 ph10 429 strings according to the more liberal rules of RFC 2279. However, if
198     the string does not even conform to RFC 2279, the result is undefined.
199 ph10 211 Your program may crash.
200    
201 ph10 429 If you want to process strings of values in the full range 0 to
202     0x7FFFFFFF, encoded in a UTF-8-like manner as per the old RFC, you can
203 ph10 211 set PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in
204     this situation, you will have to apply your own validity check.
205    
206     General comments about UTF-8 mode
207    
208 ph10 429 1. An unbraced hexadecimal escape sequence (such as \xb3) matches a
209 nigel 87 two-byte UTF-8 character if the value is greater than 127.
210 nigel 63
211 ph10 429 2. Octal numbers up to \777 are recognized, and match two-byte UTF-8
212 nigel 91 characters for values greater than \177.
213    
214 ph10 429 3. Repeat quantifiers apply to complete UTF-8 characters, not to indi-
215 nigel 73 vidual bytes, for example: \x{100}{3}.
216 nigel 63
217 ph10 429 4. The dot metacharacter matches one UTF-8 character instead of a sin-
218 nigel 75 gle byte.
219 nigel 63
220 ph10 429 5. The escape sequence \C can be used to match a single byte in UTF-8
221     mode, but its use can lead to some strange effects. This facility is
222 nigel 77 not available in the alternative matching function, pcre_dfa_exec().
223 nigel 63
224 ph10 429 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly
225     test characters of any code value, but the characters that PCRE recog-
226     nizes as digits, spaces, or word characters remain the same set as
227 nigel 75 before, all with values less than 256. This remains true even when PCRE
228 ph10 429 includes Unicode property support, because to do otherwise would slow
229     down PCRE in many common cases. If you really want to test for a wider
230     sense of, say, "digit", you must use Unicode property tests such as
231     \p{Nd}. Note that this also applies to \b, because it is defined in
232 ph10 406 terms of \w and \W.
233 nigel 63
234 ph10 429 7. Similarly, characters that match the POSIX named character classes
235 nigel 75 are all low-valued characters.
236 nigel 63
237 ph10 429 8. However, the Perl 5.10 horizontal and vertical whitespace matching
238 ph10 182 escapes (\h, \H, \v, and \V) do match all the appropriate Unicode char-
239     acters.
240    
241 ph10 429 9. Case-insensitive matching applies only to characters whose values
242     are less than 128, unless PCRE is built with Unicode property support.
243     Even when Unicode property support is available, PCRE still uses its
244     own character tables when checking the case of low-valued characters,
245     so as not to degrade performance. The Unicode property information is
246 nigel 87 used only for characters with higher values. Even when Unicode property
247     support is available, PCRE supports case-insensitive matching only when
248 ph10 429 there is a one-to-one mapping between a letter's cases. There are a
249     small number of many-to-one mappings in Unicode; these are not sup-
250 nigel 87 ported by PCRE.
251 nigel 63
252    
253     AUTHOR
254    
255 nigel 77 Philip Hazel
256 ph10 99 University Computing Service
257 nigel 93 Cambridge CB2 3QH, England.
258 nigel 63
259 ph10 429 Putting an actual email address here seems to have been a spam magnet,
260     so I've taken it away. If you want to email me, use my two initials,
261 ph10 153 followed by the two digits 10, at the domain cam.ac.uk.
262 nigel 77
263 nigel 63
264 ph10 99 REVISION
265 nigel 63
266 ph10 461 Last updated: 28 September 2009
267 ph10 406 Copyright (c) 1997-2009 University of Cambridge.
268 ph10 99 ------------------------------------------------------------------------------
269 ph10 461
270    
271 nigel 79 PCREBUILD(3) PCREBUILD(3)
272 nigel 63
273 nigel 79
274 nigel 73 NAME
275     PCRE - Perl-compatible regular expressions
276    
277 nigel 77
278 nigel 63 PCRE BUILD-TIME OPTIONS
279    
280 nigel 73 This document describes the optional features of PCRE that can be
281 ph10 261 selected when the library is compiled. It assumes use of the configure
282     script, where the optional features are selected or deselected by pro-
283     viding options to configure before running the make command. However,
284     the same options can be selected in both Unix-like and non-Unix-like
285 ph10 453 environments using the GUI facility of cmake-gui if you are using CMake
286     instead of configure to build PCRE.
287 nigel 63
288 ph10 453 There is a lot more information about building PCRE in non-Unix-like
289     environments in the file called NON_UNIX_USE, which is part of the PCRE
290     distribution. You should consult this file as well as the README file
291     if you are building in a non-Unix-like environment.
292    
293 ph10 261 The complete list of options for configure (which includes the standard
294 ph10 453 ones such as the selection of the installation directory) can be
295 ph10 261 obtained by running
296    
297 nigel 73 ./configure --help
298 nigel 63
299 ph10 453 The following sections include descriptions of options whose names
300 ph10 128 begin with --enable or --disable. These settings specify changes to the
301 ph10 453 defaults for the configure command. Because of the way that configure
302     works, --enable and --disable always come in pairs, so the complemen-
303     tary option always exists as well, but as it specifies the default, it
304 ph10 128 is not described.
305 nigel 63
306    
307 nigel 83 C++ SUPPORT
308    
309     By default, the configure script will search for a C++ compiler and C++
310     header files. If it finds them, it automatically builds the C++ wrapper
311     library for PCRE. You can disable this by adding
312    
313     --disable-cpp
314    
315     to the configure command.
316    
317    
318 nigel 63 UTF-8 SUPPORT
319    
320 ph10 392 To build PCRE with support for UTF-8 Unicode character strings, add
321 nigel 63
322 nigel 73 --enable-utf8
323 nigel 63
324 ph10 453 to the configure command. Of itself, this does not make PCRE treat
325     strings as UTF-8. As well as compiling PCRE with this option, you also
326     have have to set the PCRE_UTF8 option when you call the pcre_compile()
327 ph10 461 or pcre_compile2() functions.
328 nigel 63
329 ph10 453 If you set --enable-utf8 when compiling in an EBCDIC environment, PCRE
330 ph10 392 expects its input to be either ASCII or UTF-8 (depending on the runtime
331 ph10 453 option). It is not possible to support both EBCDIC and UTF-8 codes in
332     the same version of the library. Consequently, --enable-utf8 and
333 ph10 392 --enable-ebcdic are mutually exclusive.
334 nigel 63
335 ph10 392
336 nigel 75 UNICODE CHARACTER PROPERTY SUPPORT
337    
338 ph10 453 UTF-8 support allows PCRE to process character values greater than 255
339     in the strings that it handles. On its own, however, it does not pro-
340 nigel 75 vide any facilities for accessing the properties of such characters. If
341 ph10 453 you want to be able to use the pattern escapes \P, \p, and \X, which
342 nigel 75 refer to Unicode character properties, you must add
343    
344     --enable-unicode-properties
345    
346 ph10 453 to the configure command. This implies UTF-8 support, even if you have
347 nigel 75 not explicitly requested it.
348    
349 ph10 453 Including Unicode property support adds around 30K of tables to the
350     PCRE library. Only the general category properties such as Lu and Nd
351 ph10 128 are supported. Details are given in the pcrepattern documentation.
352 nigel 75
353    
354 nigel 63 CODE VALUE OF NEWLINE
355    
356 ph10 453 By default, PCRE interprets the linefeed (LF) character as indicating
357     the end of a line. This is the normal newline character on Unix-like
358     systems. You can compile PCRE to use carriage return (CR) instead, by
359 ph10 392 adding
360 nigel 63
361 nigel 73 --enable-newline-is-cr
362 nigel 63
363 ph10 453 to the configure command. There is also a --enable-newline-is-lf
364 nigel 91 option, which explicitly specifies linefeed as the newline character.
365 nigel 63
366 nigel 91 Alternatively, you can specify that line endings are to be indicated by
367     the two character sequence CRLF. If you want this, add
368 nigel 63
369 nigel 91 --enable-newline-is-crlf
370    
371 nigel 93 to the configure command. There is a fourth option, specified by
372 nigel 91
373 ph10 150 --enable-newline-is-anycrlf
374    
375 ph10 453 which causes PCRE to recognize any of the three sequences CR, LF, or
376 ph10 150 CRLF as indicating a line ending. Finally, a fifth option, specified by
377    
378 nigel 93 --enable-newline-is-any
379 nigel 91
380 ph10 150 causes PCRE to recognize any Unicode newline sequence.
381 nigel 93
382 ph10 453 Whatever line ending convention is selected when PCRE is built can be
383     overridden when the library functions are called. At build time it is
384 nigel 93 conventional to use the standard for your operating system.
385    
386    
387 ph10 231 WHAT \R MATCHES
388    
389 ph10 453 By default, the sequence \R in a pattern matches any Unicode newline
390     sequence, whatever has been selected as the line ending sequence. If
391 ph10 231 you specify
392    
393     --enable-bsr-anycrlf
394    
395 ph10 453 the default is changed so that \R matches only CR, LF, or CRLF. What-
396     ever is selected when PCRE is built can be overridden when the library
397 ph10 231 functions are called.
398    
399    
400 nigel 63 BUILDING SHARED AND STATIC LIBRARIES
401    
402 ph10 453 The PCRE building process uses libtool to build both shared and static
403     Unix libraries by default. You can suppress one of these by adding one
404 nigel 73 of
405 nigel 63
406 nigel 73 --disable-shared
407     --disable-static
408 nigel 63
409 nigel 73 to the configure command, as required.
410 nigel 63
411    
412     POSIX MALLOC USAGE
413    
414 nigel 75 When PCRE is called through the POSIX interface (see the pcreposix doc-
415 ph10 453 umentation), additional working storage is required for holding the
416     pointers to capturing substrings, because PCRE requires three integers
417     per substring, whereas the POSIX interface provides only two. If the
418 nigel 73 number of expected substrings is small, the wrapper function uses space
419     on the stack, because this is faster than using malloc() for each call.
420     The default threshold above which the stack is no longer used is 10; it
421     can be changed by adding a setting such as
422 nigel 63
423 nigel 73 --with-posix-malloc-threshold=20
424 nigel 63
425 nigel 73 to the configure command.
426 nigel 63
427    
428     HANDLING VERY LARGE PATTERNS
429    
430 ph10 453 Within a compiled pattern, offset values are used to point from one
431     part to another (for example, from an opening parenthesis to an alter-
432     nation metacharacter). By default, two-byte values are used for these
433     offsets, leading to a maximum size for a compiled pattern of around
434     64K. This is sufficient to handle all but the most gigantic patterns.
435 ph10 461 Nevertheless, some people do want to process truyl enormous patterns,
436     so it is possible to compile PCRE to use three-byte or four-byte off-
437     sets by adding a setting such as
438 nigel 63
439 nigel 73 --with-link-size=3
440 nigel 63
441 ph10 453 to the configure command. The value given must be 2, 3, or 4. Using
442     longer offsets slows down the operation of PCRE because it has to load
443 nigel 73 additional bytes when handling them.
444 nigel 63
445    
446 nigel 73 AVOIDING EXCESSIVE STACK USAGE
447    
448 nigel 77 When matching with the pcre_exec() function, PCRE implements backtrack-
449 ph10 453 ing by making recursive calls to an internal function called match().
450     In environments where the size of the stack is limited, this can se-
451     verely limit PCRE's operation. (The Unix environment does not usually
452 nigel 91 suffer from this problem, but it may sometimes be necessary to increase
453 ph10 453 the maximum stack size. There is a discussion in the pcrestack docu-
454     mentation.) An alternative approach to recursion that uses memory from
455     the heap to remember data, instead of using recursive function calls,
456     has been implemented to work round the problem of limited stack size.
457 nigel 91 If you want to build a version of PCRE that works this way, add
458 nigel 73
459     --disable-stack-for-recursion
460    
461 ph10 453 to the configure command. With this configuration, PCRE will use the
462     pcre_stack_malloc and pcre_stack_free variables to call memory manage-
463     ment functions. By default these point to malloc() and free(), but you
464 ph10 461 can replace the pointers so that your own functions are used instead.
465 nigel 73
466 ph10 453 Separate functions are provided rather than using pcre_malloc and
467     pcre_free because the usage is very predictable: the block sizes
468     requested are always the same, and the blocks are always freed in
469     reverse order. A calling program might be able to implement optimized
470     functions that perform better than malloc() and free(). PCRE runs
471 ph10 182 noticeably more slowly when built in this way. This option affects only
472 ph10 461 the pcre_exec() function; it is not relevant for pcre_dfa_exec().
473 nigel 73
474 ph10 182
475 nigel 91 LIMITING PCRE RESOURCE USAGE
476    
477 ph10 461 Internally, PCRE has a function called match(), which it calls repeat-
478     edly (sometimes recursively) when matching a pattern with the
479     pcre_exec() function. By controlling the maximum number of times this
480     function may be called during a single matching operation, a limit can
481     be placed on the resources used by a single call to pcre_exec(). The
482     limit can be changed at run time, as described in the pcreapi documen-
483     tation. The default is 10 million, but this can be changed by adding a
484 nigel 91 setting such as
485    
486     --with-match-limit=500000
487    
488 ph10 461 to the configure command. This setting has no effect on the
489 nigel 91 pcre_dfa_exec() matching function.
490    
491 ph10 461 In some environments it is desirable to limit the depth of recursive
492 nigel 91 calls of match() more strictly than the total number of calls, in order
493 ph10 461 to restrict the maximum amount of stack (or heap, if --disable-stack-
494 nigel 91 for-recursion is specified) that is used. A second limit controls this;
495 ph10 461 it defaults to the value that is set for --with-match-limit, which
496     imposes no additional constraints. However, you can set a lower limit
497 nigel 91 by adding, for example,
498    
499     --with-match-limit-recursion=10000
500    
501 ph10 461 to the configure command. This value can also be overridden at run
502 nigel 91 time.
503    
504    
505 ph10 128 CREATING CHARACTER TABLES AT BUILD TIME
506    
507 ph10 461 PCRE uses fixed tables for processing characters whose code values are
508     less than 256. By default, PCRE is built with a set of tables that are
509     distributed in the file pcre_chartables.c.dist. These tables are for
510 ph10 128 ASCII codes only. If you add
511    
512     --enable-rebuild-chartables
513    
514 ph10 461 to the configure command, the distributed tables are no longer used.
515     Instead, a program called dftables is compiled and run. This outputs
516 ph10 128 the source for new set of tables, created in the default locale of your
517     C runtime system. (This method of replacing the tables does not work if
518 ph10 461 you are cross compiling, because dftables is run on the local host. If
519     you need to create alternative tables when cross compiling, you will
520 ph10 128 have to do so "by hand".)
521    
522    
523 nigel 73 USING EBCDIC CODE
524    
525 ph10 461 PCRE assumes by default that it will run in an environment where the
526     character code is ASCII (or Unicode, which is a superset of ASCII).
527     This is the case for most computer operating systems. PCRE can, how-
528 ph10 197 ever, be compiled to run in an EBCDIC environment by adding
529 nigel 73
530     --enable-ebcdic
531    
532 ph10 128 to the configure command. This setting implies --enable-rebuild-charta-
533 ph10 461 bles. You should only use it if you know that you are in an EBCDIC
534     environment (for example, an IBM mainframe operating system). The
535 ph10 392 --enable-ebcdic option is incompatible with --enable-utf8.
536 nigel 73
537 nigel 93
538 ph10 286 PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT
539    
540     By default, pcregrep reads all files as plain text. You can build it so
541     that it recognizes files whose names end in .gz or .bz2, and reads them
542     with libz or libbz2, respectively, by adding one or both of
543    
544     --enable-pcregrep-libz
545     --enable-pcregrep-libbz2
546    
547     to the configure command. These options naturally require that the rel-
548 ph10 461 evant libraries are installed on your system. Configuration will fail
549 ph10 286 if they are not.
550    
551    
552 ph10 289 PCRETEST OPTION FOR LIBREADLINE SUPPORT
553    
554     If you add
555    
556     --enable-pcretest-libreadline
557    
558 ph10 461 to the configure command, pcretest is linked with the libreadline
559     library, and when its input is from a terminal, it reads it using the
560 ph10 289 readline() function. This provides line-editing and history facilities.
561 ph10 461 Note that libreadline is GPL-licensed, so if you distribute a binary of
562 ph10 289 pcretest linked in this way, there may be licensing issues.
563    
564 ph10 461 Setting this option causes the -lreadline option to be added to the
565     pcretest build. In many operating environments with a sytem-installed
566 ph10 345 libreadline this is sufficient. However, in some environments (e.g. if
567 ph10 461 an unmodified distribution version of readline is in use), some extra
568     configuration may be necessary. The INSTALL file for libreadline says
569 ph10 345 this:
570 ph10 289
571 ph10 345 "Readline uses the termcap functions, but does not link with the
572     termcap or curses library itself, allowing applications which link
573     with readline the to choose an appropriate library."
574    
575 ph10 461 If your environment has not been set up so that an appropriate library
576 ph10 345 is automatically included, you may need to add something like
577    
578     LIBS="-ncurses"
579    
580     immediately before the configure command.
581    
582    
583 nigel 93 SEE ALSO
584    
585     pcreapi(3), pcre_config(3).
586    
587 nigel 63
588 ph10 99 AUTHOR
589 nigel 63
590 ph10 99 Philip Hazel
591     University Computing Service
592     Cambridge CB2 3QH, England.
593    
594    
595     REVISION
596    
597 ph10 461 Last updated: 29 September 2009
598 ph10 392 Copyright (c) 1997-2009 University of Cambridge.
599 ph10 99 ------------------------------------------------------------------------------
600 ph10 461
601    
602 nigel 79 PCREMATCHING(3) PCREMATCHING(3)
603 nigel 63
604 nigel 79
605 nigel 77 NAME
606     PCRE - Perl-compatible regular expressions
607 nigel 73
608 nigel 77
609     PCRE MATCHING ALGORITHMS
610    
611     This document describes the two different algorithms that are available
612     in PCRE for matching a compiled regular expression against a given sub-
613     ject string. The "standard" algorithm is the one provided by the
614     pcre_exec() function. This works in the same was as Perl's matching
615     function, and provides a Perl-compatible matching operation.
616    
617     An alternative algorithm is provided by the pcre_dfa_exec() function;
618     this operates in a different way, and is not Perl-compatible. It has
619     advantages and disadvantages compared with the standard algorithm, and
620     these are described below.
621    
622     When there is only one possible way in which a given subject string can
623     match a pattern, the two algorithms give the same answer. A difference
624     arises, however, when there are multiple possibilities. For example, if
625     the pattern
626    
627     ^<.*>
628    
629     is matched against the string
630    
631     <something> <something else> <something further>
632    
633     there are three possible answers. The standard algorithm finds only one
634 nigel 93 of them, whereas the alternative algorithm finds all three.
635 nigel 77
636    
637     REGULAR EXPRESSIONS AS TREES
638    
639     The set of strings that are matched by a regular expression can be rep-
640     resented as a tree structure. An unlimited repetition in the pattern
641     makes the tree of infinite size, but it is still a tree. Matching the
642     pattern to a given subject string (from a given starting point) can be
643 nigel 91 thought of as a search of the tree. There are two ways to search a
644     tree: depth-first and breadth-first, and these correspond to the two
645     matching algorithms provided by PCRE.
646 nigel 77
647    
648     THE STANDARD MATCHING ALGORITHM
649    
650 ph10 148 In the terminology of Jeffrey Friedl's book "Mastering Regular Expres-
651     sions", the standard algorithm is an "NFA algorithm". It conducts a
652 nigel 77 depth-first search of the pattern tree. That is, it proceeds along a
653     single path through the tree, checking that the subject matches what is
654     required. When there is a mismatch, the algorithm tries any alterna-
655     tives at the current point, and if they all fail, it backs up to the
656     previous branch point in the tree, and tries the next alternative
657     branch at that level. This often involves backing up (moving to the
658     left) in the subject string as well. The order in which repetition
659     branches are tried is controlled by the greedy or ungreedy nature of
660     the quantifier.
661    
662     If a leaf node is reached, a matching string has been found, and at
663     that point the algorithm stops. Thus, if there is more than one possi-
664     ble match, this algorithm returns the first one that it finds. Whether
665     this is the shortest, the longest, or some intermediate length depends
666     on the way the greedy and ungreedy repetition quantifiers are specified
667     in the pattern.
668    
669     Because it ends up with a single path through the tree, it is rela-
670     tively straightforward for this algorithm to keep track of the sub-
671     strings that are matched by portions of the pattern in parentheses.
672     This provides support for capturing parentheses and back references.
673    
674    
675 nigel 93 THE ALTERNATIVE MATCHING ALGORITHM
676 nigel 77
677 nigel 93 This algorithm conducts a breadth-first search of the tree. Starting
678     from the first matching point in the subject, it scans the subject
679     string from left to right, once, character by character, and as it does
680     this, it remembers all the paths through the tree that represent valid
681     matches. In Friedl's terminology, this is a kind of "DFA algorithm",
682     though it is not implemented as a traditional finite state machine (it
683     keeps multiple states active simultaneously).
684 nigel 77
685 ph10 461 Although the general principle of this matching algorithm is that it
686     scans the subject string only once, without backtracking, there is one
687     exception: when a lookaround assertion is encountered, the characters
688     following or preceding the current point have to be independently
689     inspected.
690    
691 nigel 93 The scan continues until either the end of the subject is reached, or
692     there are no more unterminated paths. At this point, terminated paths
693     represent the different matching possibilities (if there are none, the
694     match has failed). Thus, if there is more than one possible match,
695 nigel 77 this algorithm finds all of them, and in particular, it finds the long-
696 ph10 461 est. There is an option to stop the algorithm after the first match
697     (which is necessarily the shortest) is found.
698 nigel 77
699     Note that all the matches that are found start at the same point in the
700     subject. If the pattern
701    
702     cat(er(pillar)?)
703    
704 nigel 93 is matched against the string "the caterpillar catchment", the result
705     will be the three strings "cat", "cater", and "caterpillar" that start
706 nigel 77 at the fourth character of the subject. The algorithm does not automat-
707     ically move on to find matches that start at later positions.
708    
709     There are a number of features of PCRE regular expressions that are not
710 nigel 93 supported by the alternative matching algorithm. They are as follows:
711 nigel 77
712 ph10 461 1. Because the algorithm finds all possible matches, the greedy or
713     ungreedy nature of repetition quantifiers is not relevant. Greedy and
714 nigel 93 ungreedy quantifiers are treated in exactly the same way. However, pos-
715 ph10 461 sessive quantifiers can make a difference when what follows could also
716 nigel 93 match what is quantified, for example in a pattern like this:
717 nigel 77
718 nigel 93 ^a++\w!
719    
720 ph10 461 This pattern matches "aaab!" but not "aaa!", which would be matched by
721     a non-possessive quantifier. Similarly, if an atomic group is present,
722     it is matched as if it were a standalone pattern at the current point,
723     and the longest match is then "locked in" for the rest of the overall
724 nigel 93 pattern.
725    
726 nigel 77 2. When dealing with multiple paths through the tree simultaneously, it
727 ph10 461 is not straightforward to keep track of captured substrings for the
728     different matching possibilities, and PCRE's implementation of this
729 nigel 77 algorithm does not attempt to do this. This means that no captured sub-
730     strings are available.
731    
732 ph10 461 3. Because no substrings are captured, back references within the pat-
733 nigel 77 tern are not supported, and cause errors if encountered.
734    
735 ph10 461 4. For the same reason, conditional expressions that use a backrefer-
736     ence as the condition or test for a specific group recursion are not
737 nigel 93 supported.
738 nigel 77
739 ph10 461 5. Because many paths through the tree may be active, the \K escape
740 ph10 172 sequence, which resets the start of the match when encountered (but may
741 ph10 461 be on some paths and not on others), is not supported. It causes an
742 ph10 172 error if encountered.
743    
744 ph10 461 6. Callouts are supported, but the value of the capture_top field is
745 nigel 77 always 1, and the value of the capture_last field is always -1.
746    
747 ph10 461 7. The \C escape sequence, which (in the standard algorithm) matches a
748     single byte, even in UTF-8 mode, is not supported because the alterna-
749     tive algorithm moves through the subject string one character at a
750 nigel 93 time, for all active paths through the tree.
751 nigel 77
752 ph10 461 8. Except for (*FAIL), the backtracking control verbs such as (*PRUNE)
753     are not supported. (*FAIL) is supported, and behaves like a failing
754 ph10 345 negative assertion.
755 nigel 77
756 ph10 211
757 nigel 93 ADVANTAGES OF THE ALTERNATIVE ALGORITHM
758 nigel 77
759 ph10 461 Using the alternative matching algorithm provides the following advan-
760 nigel 93 tages:
761 nigel 77
762     1. All possible matches (at a single point in the subject) are automat-
763 ph10 461 ically found, and in particular, the longest match is found. To find
764 nigel 77 more than one match using the standard algorithm, you have to do kludgy
765     things with callouts.
766    
767 ph10 461 2. Because the alternative algorithm scans the subject string just
768     once, and never needs to backtrack, it is possible to pass very long
769     subject strings to the matching function in several pieces, checking
770     for partial matching each time. The pcrepartial documentation gives
771     details of partial matching.
772 nigel 77
773    
774 nigel 93 DISADVANTAGES OF THE ALTERNATIVE ALGORITHM
775 nigel 77
776 nigel 93 The alternative algorithm suffers from a number of disadvantages:
777 nigel 77
778 ph10 453 1. It is substantially slower than the standard algorithm. This is
779     partly because it has to search for all possible matches, but is also
780 nigel 77 because it is less susceptible to optimization.
781    
782     2. Capturing parentheses and back references are not supported.
783    
784 nigel 93 3. Although atomic groups are supported, their use does not provide the
785     performance advantage that it does for the standard algorithm.
786 nigel 77
787    
788 ph10 99 AUTHOR
789 nigel 77
790 ph10 99 Philip Hazel
791     University Computing Service
792     Cambridge CB2 3QH, England.
793    
794    
795     REVISION
796    
797 ph10 461 Last updated: 29 September 2009
798 ph10 429 Copyright (c) 1997-2009 University of Cambridge.
799 ph10 99 ------------------------------------------------------------------------------
800 ph10 461
801    
802 nigel 79 PCREAPI(3) PCREAPI(3)
803 nigel 77
804 nigel 79
805 nigel 73 NAME
806     PCRE - Perl-compatible regular expressions
807    
808 nigel 77
809 nigel 75 PCRE NATIVE API
810 nigel 63
811 nigel 73 #include <pcre.h>
812 nigel 41
813 nigel 73 pcre *pcre_compile(const char *pattern, int options,
814     const char **errptr, int *erroffset,
815     const unsigned char *tableptr);
816 nigel 41
817 nigel 77 pcre *pcre_compile2(const char *pattern, int options,
818     int *errorcodeptr,
819     const char **errptr, int *erroffset,
820     const unsigned char *tableptr);
821    
822 nigel 73 pcre_extra *pcre_study(const pcre *code, int options,
823     const char **errptr);
824 nigel 41
825 nigel 73 int pcre_exec(const pcre *code, const pcre_extra *extra,
826     const char *subject, int length, int startoffset,
827     int options, int *ovector, int ovecsize);
828 nigel 41
829 nigel 77 int pcre_dfa_exec(const pcre *code, const pcre_extra *extra,
830     const char *subject, int length, int startoffset,
831     int options, int *ovector, int ovecsize,
832     int *workspace, int wscount);
833    
834 nigel 73 int pcre_copy_named_substring(const pcre *code,
835     const char *subject, int *ovector,
836     int stringcount, const char *stringname,
837     char *buffer, int buffersize);
838 nigel 63
839 nigel 73 int pcre_copy_substring(const char *subject, int *ovector,
840     int stringcount, int stringnumber, char *buffer,
841     int buffersize);
842 nigel 41
843 nigel 73 int pcre_get_named_substring(const pcre *code,
844     const char *subject, int *ovector,
845     int stringcount, const char *stringname,
846     const char **stringptr);
847 nigel 63
848 nigel 73 int pcre_get_stringnumber(const pcre *code,
849     const char *name);
850 nigel 63
851 nigel 91 int pcre_get_stringtable_entries(const pcre *code,
852     const char *name, char **first, char **last);
853    
854 nigel 73 int pcre_get_substring(const char *subject, int *ovector,
855     int stringcount, int stringnumber,
856     const char **stringptr);
857 nigel 41
858 nigel 73 int pcre_get_substring_list(const char *subject,
859     int *ovector, int stringcount, const char ***listptr);
860 nigel 41
861 nigel 73 void pcre_free_substring(const char *stringptr);
862 nigel 49
863 nigel 73 void pcre_free_substring_list(const char **stringptr);
864 nigel 49
865 nigel 73 const unsigned char *pcre_maketables(void);
866 nigel 41
867 nigel 73 int pcre_fullinfo(const pcre *code, const pcre_extra *extra,
868     int what, void *where);
869 nigel 43
870 nigel 73 int pcre_info(const pcre *code, int *optptr, int *firstcharptr);
871 nigel 63
872 nigel 77 int pcre_refcount(pcre *code, int adjust);
873    
874 nigel 73 int pcre_config(int what, void *where);
875 nigel 41
876 nigel 73 char *pcre_version(void);
877 nigel 63
878 nigel 73 void *(*pcre_malloc)(size_t);
879 nigel 41
880 nigel 73 void (*pcre_free)(void *);
881 nigel 41
882 nigel 73 void *(*pcre_stack_malloc)(size_t);
883 nigel 41
884 nigel 73 void (*pcre_stack_free)(void *);
885 nigel 41
886 nigel 73 int (*pcre_callout)(pcre_callout_block *);
887 nigel 41
888 nigel 73
889 nigel 75 PCRE API OVERVIEW
890 nigel 41
891 nigel 73 PCRE has its own native API, which is described in this document. There
892 nigel 93 are also some wrapper functions that correspond to the POSIX regular
893 nigel 77 expression API. These are described in the pcreposix documentation.
894     Both of these APIs define a set of C function calls. A C++ wrapper is
895     distributed with PCRE. It is documented in the pcrecpp page.
896 nigel 43
897 nigel 77 The native API C function prototypes are defined in the header file
898     pcre.h, and on Unix systems the library itself is called libpcre. It
899 nigel 75 can normally be accessed by adding -lpcre to the command for linking an
900     application that uses PCRE. The header file defines the macros
901     PCRE_MAJOR and PCRE_MINOR to contain the major and minor release num-
902     bers for the library. Applications can use these to include support
903     for different releases of PCRE.
904 nigel 41
905 nigel 77 The functions pcre_compile(), pcre_compile2(), pcre_study(), and
906     pcre_exec() are used for compiling and matching regular expressions in
907     a Perl-compatible manner. A sample program that demonstrates the sim-
908     plest way of using them is provided in the file called pcredemo.c in
909 ph10 429 the PCRE source distribution. A listing of this program is given in the
910     pcredemo documentation, and the pcresample documentation describes how
911     to compile and run it.
912 nigel 49
913 nigel 77 A second matching function, pcre_dfa_exec(), which is not Perl-compati-
914 ph10 429 ble, is also provided. This uses a different algorithm for the match-
915     ing. The alternative algorithm finds all possible matches (at a given
916 ph10 453 point in the subject), and scans the subject just once (unless there
917     are lookbehind assertions). However, this algorithm does not return
918     captured substrings. A description of the two matching algorithms and
919     their advantages and disadvantages is given in the pcrematching docu-
920     mentation.
921 nigel 63
922 ph10 453 In addition to the main compiling and matching functions, there are
923 nigel 77 convenience functions for extracting captured substrings from a subject
924     string that is matched by pcre_exec(). They are:
925    
926 nigel 73 pcre_copy_substring()
927     pcre_copy_named_substring()
928     pcre_get_substring()
929     pcre_get_named_substring()
930     pcre_get_substring_list()
931 nigel 75 pcre_get_stringnumber()
932 nigel 91 pcre_get_stringtable_entries()
933 nigel 63
934 nigel 73 pcre_free_substring() and pcre_free_substring_list() are also provided,
935     to free the memory used for extracted strings.
936 nigel 41
937 ph10 453 The function pcre_maketables() is used to build a set of character
938     tables in the current locale for passing to pcre_compile(),
939     pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is
940     provided for specialist use. Most commonly, no special tables are
941     passed, in which case internal tables that are generated when PCRE is
942 nigel 77 built are used.
943 nigel 49
944 ph10 453 The function pcre_fullinfo() is used to find out information about a
945     compiled pattern; pcre_info() is an obsolete version that returns only
946     some of the available information, but is retained for backwards com-
947     patibility. The function pcre_version() returns a pointer to a string
948 nigel 73 containing the version of PCRE and its date of release.
949 nigel 41
950 ph10 453 The function pcre_refcount() maintains a reference count in a data
951     block containing a compiled pattern. This is provided for the benefit
952 nigel 77 of object-oriented applications.
953    
954 ph10 453 The global variables pcre_malloc and pcre_free initially contain the
955     entry points of the standard malloc() and free() functions, respec-
956 nigel 73 tively. PCRE calls the memory management functions via these variables,
957 ph10 453 so a calling program can replace them if it wishes to intercept the
958 nigel 73 calls. This should be done before calling any PCRE functions.
959 nigel 41
960 ph10 453 The global variables pcre_stack_malloc and pcre_stack_free are also
961     indirections to memory management functions. These special functions
962     are used only when PCRE is compiled to use the heap for remembering
963 nigel 77 data, instead of recursive function calls, when running the pcre_exec()
964 ph10 453 function. See the pcrebuild documentation for details of how to do
965     this. It is a non-standard way of building PCRE, for use in environ-
966     ments that have limited stacks. Because of the greater use of memory
967     management, it runs more slowly. Separate functions are provided so
968     that special-purpose external code can be used for this case. When
969     used, these functions are always called in a stack-like manner (last
970     obtained, first freed), and always for memory blocks of the same size.
971     There is a discussion about PCRE's stack usage in the pcrestack docu-
972 nigel 91 mentation.
973 nigel 41
974 nigel 73 The global variable pcre_callout initially contains NULL. It can be set
975 ph10 453 by the caller to a "callout" function, which PCRE will then call at
976     specified points during a matching operation. Details are given in the
977 nigel 73 pcrecallout documentation.
978 nigel 41
979 nigel 73
980 nigel 91 NEWLINES
981    
982 ph10 453 PCRE supports five different conventions for indicating line breaks in
983     strings: a single CR (carriage return) character, a single LF (line-
984 ph10 150 feed) character, the two-character sequence CRLF, any of the three pre-
985 ph10 453 ceding, or any Unicode newline sequence. The Unicode newline sequences
986     are the three just mentioned, plus the single characters VT (vertical
987     tab, U+000B), FF (formfeed, U+000C), NEL (next line, U+0085), LS (line
988 ph10 150 separator, U+2028), and PS (paragraph separator, U+2029).
989 nigel 93
990 ph10 453 Each of the first three conventions is used by at least one operating
991     system as its standard newline sequence. When PCRE is built, a default
992     can be specified. The default default is LF, which is the Unix stan-
993     dard. When PCRE is run, the default can be overridden, either when a
994 nigel 93 pattern is compiled, or when it is matched.
995    
996 ph10 227 At compile time, the newline convention can be specified by the options
997 ph10 453 argument of pcre_compile(), or it can be specified by special text at
998 ph10 227 the start of the pattern itself; this overrides any other settings. See
999     the pcrepattern page for details of the special character sequences.
1000    
1001 nigel 91 In the PCRE documentation the word "newline" is used to mean "the char-
1002 ph10 453 acter or pair of characters that indicate a line break". The choice of
1003     newline convention affects the handling of the dot, circumflex, and
1004 nigel 93 dollar metacharacters, the handling of #-comments in /x mode, and, when
1005 ph10 453 CRLF is a recognized line ending sequence, the match position advance-
1006 ph10 227 ment for a non-anchored pattern. There is more detail about this in the
1007 ph10 231 section on pcre_exec() options below.
1008 nigel 91
1009 ph10 453 The choice of newline convention does not affect the interpretation of
1010     the \n or \r escape sequences, nor does it affect what \R matches,
1011 ph10 231 which is controlled in a similar way, but by separate options.
1012 nigel 91
1013 ph10 231
1014 nigel 63 MULTITHREADING
1015    
1016 ph10 453 The PCRE functions can be used in multi-threading applications, with
1017 nigel 73 the proviso that the memory management functions pointed to by
1018     pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the
1019     callout function pointed to by pcre_callout, are shared by all threads.
1020 nigel 41
1021 ph10 453 The compiled form of a regular expression is not altered during match-
1022 nigel 73 ing, so the same compiled pattern can safely be used by several threads
1023     at once.
1024 nigel 41
1025    
1026 nigel 75 SAVING PRECOMPILED PATTERNS FOR LATER USE
1027    
1028     The compiled form of a regular expression can be saved and re-used at a
1029 ph10 453 later time, possibly by a different program, and even on a host other
1030     than the one on which it was compiled. Details are given in the
1031     pcreprecompile documentation. However, compiling a regular expression
1032     with one version of PCRE for use with a different version is not guar-
1033 ph10 155 anteed to work and may cause crashes.
1034 nigel 75
1035    
1036 nigel 63 CHECKING BUILD-TIME OPTIONS
1037 nigel 41
1038 nigel 73 int pcre_config(int what, void *where);
1039 nigel 63
1040 ph10 453 The function pcre_config() makes it possible for a PCRE client to dis-
1041 nigel 73 cover which optional features have been compiled into the PCRE library.
1042 ph10 453 The pcrebuild documentation has more details about these optional fea-
1043 nigel 73 tures.
1044 nigel 63
1045 ph10 453 The first argument for pcre_config() is an integer, specifying which
1046 nigel 73 information is required; the second argument is a pointer to a variable
1047 ph10 453 into which the information is placed. The following information is
1048 nigel 73 available:
1049 nigel 63
1050 nigel 73 PCRE_CONFIG_UTF8
1051 nigel 63
1052 ph10 453 The output is an integer that is set to one if UTF-8 support is avail-
1053 nigel 73 able; otherwise it is set to zero.
1054 nigel 63
1055 nigel 75 PCRE_CONFIG_UNICODE_PROPERTIES
1056    
1057 ph10 453 The output is an integer that is set to one if support for Unicode
1058 nigel 75 character properties is available; otherwise it is set to zero.
1059    
1060 nigel 73 PCRE_CONFIG_NEWLINE
1061 nigel 63
1062 ph10 453 The output is an integer whose value specifies the default character
1063     sequence that is recognized as meaning "newline". The four values that
1064 ph10 150 are supported are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF,
1065 ph10 453 and -1 for ANY. Though they are derived from ASCII, the same values
1066 ph10 392 are returned in EBCDIC environments. The default should normally corre-
1067     spond to the standard sequence for your operating system.
1068 nigel 63
1069 ph10 231 PCRE_CONFIG_BSR
1070    
1071     The output is an integer whose value indicates what character sequences
1072 ph10 453 the \R escape sequence matches by default. A value of 0 means that \R
1073     matches any Unicode line ending sequence; a value of 1 means that \R
1074 ph10 231 matches only CR, LF, or CRLF. The default can be overridden when a pat-
1075     tern is compiled or matched.
1076    
1077 nigel 73 PCRE_CONFIG_LINK_SIZE
1078 nigel 63
1079 ph10 453 The output is an integer that contains the number of bytes used for
1080 nigel 73 internal linkage in compiled regular expressions. The value is 2, 3, or
1081 ph10 453 4. Larger values allow larger regular expressions to be compiled, at
1082     the expense of slower matching. The default value of 2 is sufficient
1083     for all but the most massive patterns, since it allows the compiled
1084 nigel 73 pattern to be up to 64K in size.
1085 nigel 63
1086 nigel 73 PCRE_CONFIG_POSIX_MALLOC_THRESHOLD
1087 nigel 63
1088 ph10 453 The output is an integer that contains the threshold above which the
1089     POSIX interface uses malloc() for output vectors. Further details are
1090 nigel 73 given in the pcreposix documentation.
1091 nigel 63
1092 nigel 73 PCRE_CONFIG_MATCH_LIMIT
1093 nigel 63
1094 ph10 453 The output is a long integer that gives the default limit for the num-
1095     ber of internal matching function calls in a pcre_exec() execution.
1096 ph10 392 Further details are given with pcre_exec() below.
1097 nigel 63
1098 nigel 87 PCRE_CONFIG_MATCH_LIMIT_RECURSION
1099    
1100 ph10 392 The output is a long integer that gives the default limit for the depth
1101 ph10 453 of recursion when calling the internal matching function in a
1102     pcre_exec() execution. Further details are given with pcre_exec()
1103 ph10 392 below.
1104 nigel 87
1105 nigel 73 PCRE_CONFIG_STACKRECURSE
1106 nigel 63
1107 ph10 453 The output is an integer that is set to one if internal recursion when
1108 nigel 77 running pcre_exec() is implemented by recursive function calls that use
1109 ph10 453 the stack to remember their state. This is the usual way that PCRE is
1110 nigel 77 compiled. The output is zero if PCRE was compiled to use blocks of data
1111 ph10 453 on the heap instead of recursive function calls. In this case,
1112     pcre_stack_malloc and pcre_stack_free are called to manage memory
1113 nigel 77 blocks on the heap, thus avoiding the use of the stack.
1114 nigel 73
1115    
1116 nigel 41 COMPILING A PATTERN
1117 nigel 63
1118 nigel 73 pcre *pcre_compile(const char *pattern, int options,
1119     const char **errptr, int *erroffset,
1120     const unsigned char *tableptr);
1121 nigel 63
1122 nigel 77 pcre *pcre_compile2(const char *pattern, int options,
1123     int *errorcodeptr,
1124     const char **errptr, int *erroffset,
1125     const unsigned char *tableptr);
1126 nigel 41
1127 nigel 77 Either of the functions pcre_compile() or pcre_compile2() can be called
1128     to compile a pattern into an internal form. The only difference between
1129 ph10 453 the two interfaces is that pcre_compile2() has an additional argument,
1130 ph10 461 errorcodeptr, via which a numerical error code can be returned. To
1131     avoid too much repetition, we refer just to pcre_compile() below, but
1132     the information applies equally to pcre_compile2().
1133 nigel 77
1134     The pattern is a C string terminated by a binary zero, and is passed in
1135 ph10 453 the pattern argument. A pointer to a single block of memory that is
1136     obtained via pcre_malloc is returned. This contains the compiled code
1137 nigel 77 and related data. The pcre type is defined for the returned block; this
1138     is a typedef for a structure whose contents are not externally defined.
1139 nigel 91 It is up to the caller to free the memory (via pcre_free) when it is no
1140     longer required.
1141 nigel 77
1142 ph10 453 Although the compiled code of a PCRE regex is relocatable, that is, it
1143 nigel 73 does not depend on memory location, the complete pcre data block is not
1144 ph10 453 fully relocatable, because it may contain a copy of the tableptr argu-
1145 nigel 75 ment, which is an address (see below).
1146 nigel 41
1147 nigel 93 The options argument contains various bit settings that affect the com-
1148 ph10 453 pilation. It should be zero if no options are required. The available
1149     options are described below. Some of them (in particular, those that
1150 ph10 461 are compatible with Perl, but some others as well) can also be set and
1151 ph10 453 unset from within the pattern (see the detailed description in the
1152     pcrepattern documentation). For those options that can be different in
1153     different parts of the pattern, the contents of the options argument
1154 ph10 461 specifies their settings at the start of compilation and execution. The
1155     PCRE_ANCHORED, PCRE_BSR_xxx, and PCRE_NEWLINE_xxx options can be set at
1156     the time of matching as well as at compile time.
1157 nigel 41
1158 nigel 73 If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise,
1159 ph10 453 if compilation of a pattern fails, pcre_compile() returns NULL, and
1160 nigel 73 sets the variable pointed to by errptr to point to a textual error mes-
1161 nigel 87 sage. This is a static string that is part of the library. You must not
1162 ph10 454 try to free it. The byte offset from the start of the pattern to the
1163 ph10 461 character that was being processed when the error was discovered is
1164 ph10 454 placed in the variable pointed to by erroffset, which must not be NULL.
1165     If it is, an immediate error is given. Some errors are not detected
1166     until checks are carried out when the whole pattern has been scanned;
1167     in this case the offset is set to the end of the pattern.
1168 nigel 53
1169 ph10 453 If pcre_compile2() is used instead of pcre_compile(), and the error-
1170     codeptr argument is not NULL, a non-zero error code number is returned
1171     via this argument in the event of an error. This is in addition to the
1172 nigel 77 textual error message. Error codes and messages are listed below.
1173    
1174 ph10 453 If the final argument, tableptr, is NULL, PCRE uses a default set of
1175     character tables that are built when PCRE is compiled, using the
1176     default C locale. Otherwise, tableptr must be an address that is the
1177     result of a call to pcre_maketables(). This value is stored with the
1178     compiled pattern, and used again by pcre_exec(), unless another table
1179 nigel 75 pointer is passed to it. For more discussion, see the section on locale
1180     support below.
1181 nigel 53
1182 ph10 453 This code fragment shows a typical straightforward call to pcre_com-
1183 nigel 73 pile():
1184 nigel 41
1185 nigel 73 pcre *re;
1186     const char *error;
1187     int erroffset;
1188     re = pcre_compile(
1189     "^A.*Z", /* the pattern */
1190     0, /* default options */
1191     &error, /* for error message */
1192     &erroffset, /* for error offset */
1193     NULL); /* use default character tables */
1194 nigel 41
1195 ph10 453 The following names for option bits are defined in the pcre.h header
1196 nigel 75 file:
1197 nigel 41
1198 nigel 73 PCRE_ANCHORED
1199 nigel 41
1200 nigel 73 If this bit is set, the pattern is forced to be "anchored", that is, it
1201 ph10 453 is constrained to match only at the first matching point in the string
1202     that is being searched (the "subject string"). This effect can also be
1203     achieved by appropriate constructs in the pattern itself, which is the
1204 nigel 73 only way to do it in Perl.
1205 nigel 41
1206 nigel 75 PCRE_AUTO_CALLOUT
1207    
1208     If this bit is set, pcre_compile() automatically inserts callout items,
1209 ph10 453 all with number 255, before each pattern item. For discussion of the
1210 nigel 75 callout facility, see the pcrecallout documentation.
1211    
1212 ph10 231 PCRE_BSR_ANYCRLF
1213     PCRE_BSR_UNICODE
1214    
1215     These options (which are mutually exclusive) control what the \R escape
1216 ph10 453 sequence matches. The choice is either to match only CR, LF, or CRLF,
1217 ph10 231 or to match any Unicode newline sequence. The default is specified when
1218     PCRE is built. It can be overridden from within the pattern, or by set-
1219     ting an option when a compiled pattern is matched.
1220    
1221 nigel 73 PCRE_CASELESS
1222 nigel 41
1223 ph10 453 If this bit is set, letters in the pattern match both upper and lower
1224     case letters. It is equivalent to Perl's /i option, and it can be
1225     changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE
1226     always understands the concept of case for characters whose values are
1227     less than 128, so caseless matching is always possible. For characters
1228     with higher values, the concept of case is supported if PCRE is com-
1229     piled with Unicode property support, but not otherwise. If you want to
1230     use caseless matching for characters 128 and above, you must ensure
1231     that PCRE is compiled with Unicode property support as well as with
1232 nigel 77 UTF-8 support.
1233 nigel 41
1234 nigel 73 PCRE_DOLLAR_ENDONLY
1235 nigel 41
1236 ph10 453 If this bit is set, a dollar metacharacter in the pattern matches only
1237     at the end of the subject string. Without this option, a dollar also
1238     matches immediately before a newline at the end of the string (but not
1239     before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored
1240     if PCRE_MULTILINE is set. There is no equivalent to this option in
1241 nigel 91 Perl, and no way to set it within a pattern.
1242 nigel 41
1243 nigel 73 PCRE_DOTALL
1244 nigel 41
1245 nigel 73 If this bit is set, a dot metacharater in the pattern matches all char-
1246 ph10 453 acters, including those that indicate newline. Without it, a dot does
1247     not match when the current position is at a newline. This option is
1248     equivalent to Perl's /s option, and it can be changed within a pattern
1249     by a (?s) option setting. A negative class such as [^a] always matches
1250 nigel 93 newline characters, independent of the setting of this option.
1251 nigel 63
1252 nigel 91 PCRE_DUPNAMES
1253    
1254 ph10 453 If this bit is set, names used to identify capturing subpatterns need
1255 nigel 91 not be unique. This can be helpful for certain types of pattern when it
1256 ph10 453 is known that only one instance of the named subpattern can ever be
1257     matched. There are more details of named subpatterns below; see also
1258 nigel 91 the pcrepattern documentation.
1259    
1260 nigel 73 PCRE_EXTENDED
1261 nigel 41
1262 ph10 453 If this bit is set, whitespace data characters in the pattern are
1263 nigel 77 totally ignored except when escaped or inside a character class. White-
1264     space does not include the VT character (code 11). In addition, charac-
1265     ters between an unescaped # outside a character class and the next new-
1266 ph10 453 line, inclusive, are also ignored. This is equivalent to Perl's /x
1267     option, and it can be changed within a pattern by a (?x) option set-
1268 nigel 91 ting.
1269 nigel 41
1270 ph10 453 This option makes it possible to include comments inside complicated
1271     patterns. Note, however, that this applies only to data characters.
1272     Whitespace characters may never appear within special character
1273     sequences in a pattern, for example within the sequence (?( which
1274 nigel 73 introduces a conditional subpattern.
1275 nigel 41
1276 nigel 73 PCRE_EXTRA
1277 nigel 41
1278 ph10 453 This option was invented in order to turn on additional functionality
1279     of PCRE that is incompatible with Perl, but it is currently of very
1280     little use. When set, any backslash in a pattern that is followed by a
1281     letter that has no special meaning causes an error, thus reserving
1282     these combinations for future expansion. By default, as in Perl, a
1283     backslash followed by a letter with no special meaning is treated as a
1284     literal. (Perl can, however, be persuaded to give a warning for this.)
1285     There are at present no other features controlled by this option. It
1286 nigel 91 can also be set by a (?X) option setting within a pattern.
1287 nigel 41
1288 nigel 77 PCRE_FIRSTLINE
1289    
1290 ph10 453 If this option is set, an unanchored pattern is required to match
1291     before or at the first newline in the subject string, though the
1292 nigel 91 matched text may continue over the newline.
1293 nigel 77
1294 ph10 345 PCRE_JAVASCRIPT_COMPAT
1295    
1296     If this option is set, PCRE's behaviour is changed in some ways so that
1297 ph10 453 it is compatible with JavaScript rather than Perl. The changes are as
1298 ph10 345 follows:
1299    
1300 ph10 453 (1) A lone closing square bracket in a pattern causes a compile-time
1301     error, because this is illegal in JavaScript (by default it is treated
1302 ph10 345 as a data character). Thus, the pattern AB]CD becomes illegal when this
1303     option is set.
1304    
1305 ph10 453 (2) At run time, a back reference to an unset subpattern group matches
1306     an empty string (by default this causes the current matching alterna-
1307     tive to fail). A pattern such as (\1)(a) succeeds when this option is
1308     set (assuming it can find an "a" in the subject), whereas it fails by
1309 ph10 345 default, for Perl compatibility.
1310    
1311 nigel 73 PCRE_MULTILINE
1312 nigel 41
1313 ph10 453 By default, PCRE treats the subject string as consisting of a single
1314     line of characters (even if it actually contains newlines). The "start
1315     of line" metacharacter (^) matches only at the start of the string,
1316     while the "end of line" metacharacter ($) matches only at the end of
1317 nigel 75 the string, or before a terminating newline (unless PCRE_DOLLAR_ENDONLY
1318     is set). This is the same as Perl.
1319 nigel 63
1320 ph10 453 When PCRE_MULTILINE it is set, the "start of line" and "end of line"
1321     constructs match immediately following or immediately before internal
1322     newlines in the subject string, respectively, as well as at the very
1323     start and end. This is equivalent to Perl's /m option, and it can be
1324 nigel 91 changed within a pattern by a (?m) option setting. If there are no new-
1325 ph10 453 lines in a subject string, or no occurrences of ^ or $ in a pattern,
1326 nigel 73 setting PCRE_MULTILINE has no effect.
1327 nigel 63
1328 nigel 91 PCRE_NEWLINE_CR
1329     PCRE_NEWLINE_LF
1330     PCRE_NEWLINE_CRLF
1331 ph10 150 PCRE_NEWLINE_ANYCRLF
1332 nigel 93 PCRE_NEWLINE_ANY
1333 nigel 91
1334 ph10 453 These options override the default newline definition that was chosen
1335     when PCRE was built. Setting the first or the second specifies that a
1336     newline is indicated by a single character (CR or LF, respectively).
1337     Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the
1338     two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies
1339 ph10 150 that any of the three preceding sequences should be recognized. Setting
1340 ph10 453 PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be
1341 ph10 150 recognized. The Unicode newline sequences are the three just mentioned,
1342 ph10 453 plus the single characters VT (vertical tab, U+000B), FF (formfeed,
1343     U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS
1344     (paragraph separator, U+2029). The last two are recognized only in
1345 ph10 150 UTF-8 mode.
1346 nigel 91
1347 ph10 453 The newline setting in the options word uses three bits that are
1348 ph10 150 treated as a number, giving eight possibilities. Currently only six are
1349 ph10 453 used (default plus the five values above). This means that if you set
1350     more than one newline option, the combination may or may not be sensi-
1351 ph10 150 ble. For example, PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to
1352 ph10 453 PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and
1353 ph10 150 cause an error.
1354 nigel 91
1355 ph10 453 The only time that a line break is specially recognized when compiling
1356     a pattern is if PCRE_EXTENDED is set, and an unescaped # outside a
1357     character class is encountered. This indicates a comment that lasts
1358     until after the next line break sequence. In other circumstances, line
1359     break sequences are treated as literal data, except that in
1360 nigel 93 PCRE_EXTENDED mode, both CR and LF are treated as whitespace characters
1361     and are therefore ignored.
1362    
1363     The newline option that is set at compile time becomes the default that
1364 ph10 392 is used for pcre_exec() and pcre_dfa_exec(), but it can be overridden.
1365 nigel 93
1366 nigel 73 PCRE_NO_AUTO_CAPTURE
1367 nigel 41
1368 nigel 73 If this option is set, it disables the use of numbered capturing paren-
1369 ph10 453 theses in the pattern. Any opening parenthesis that is not followed by
1370     ? behaves as if it were followed by ?: but named parentheses can still
1371     be used for capturing (and they acquire numbers in the usual way).
1372 nigel 73 There is no equivalent of this option in Perl.
1373 nigel 41
1374 nigel 73 PCRE_UNGREEDY
1375 nigel 41
1376 ph10 453 This option inverts the "greediness" of the quantifiers so that they
1377     are not greedy by default, but become greedy if followed by "?". It is
1378     not compatible with Perl. It can also be set by a (?U) option setting
1379 nigel 73 within the pattern.
1380 nigel 41
1381 nigel 73 PCRE_UTF8
1382 nigel 49
1383 ph10 453 This option causes PCRE to regard both the pattern and the subject as
1384     strings of UTF-8 characters instead of single-byte character strings.
1385     However, it is available only when PCRE is built to include UTF-8 sup-
1386     port. If not, the use of this option provokes an error. Details of how
1387     this option changes the behaviour of PCRE are given in the section on
1388 nigel 75 UTF-8 support in the main pcre page.
1389 nigel 71
1390 nigel 73 PCRE_NO_UTF8_CHECK
1391 nigel 71
1392 nigel 73 When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is
1393 ph10 453 automatically checked. There is a discussion about the validity of
1394     UTF-8 strings in the main pcre page. If an invalid UTF-8 sequence of
1395     bytes is found, pcre_compile() returns an error. If you already know
1396 ph10 211 that your pattern is valid, and you want to skip this check for perfor-
1397 ph10 453 mance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is
1398     set, the effect of passing an invalid UTF-8 string as a pattern is
1399     undefined. It may cause your program to crash. Note that this option
1400     can also be passed to pcre_exec() and pcre_dfa_exec(), to suppress the
1401 ph10 211 UTF-8 validity checking of subject strings.
1402 nigel 71
1403 nigel 73
1404 nigel 77 COMPILATION ERROR CODES
1405    
1406 ph10 453 The following table lists the error codes than may be returned by
1407     pcre_compile2(), along with the error messages that may be returned by
1408     both compiling functions. As PCRE has developed, some error codes have
1409 nigel 93 fallen out of use. To avoid confusion, they have not been re-used.
1410 nigel 77
1411     0 no error
1412     1 \ at end of pattern
1413     2 \c at end of pattern
1414     3 unrecognized character follows \
1415     4 numbers out of order in {} quantifier
1416     5 number too big in {} quantifier
1417     6 missing terminating ] for character class
1418     7 invalid escape sequence in character class
1419     8 range out of order in character class
1420     9 nothing to repeat
1421 nigel 93 10 [this code is not in use]
1422 nigel 77 11 internal error: unexpected repeat
1423 ph10 292 12 unrecognized character after (? or (?-
1424 nigel 77 13 POSIX named classes are supported only within a class
1425     14 missing )
1426     15 reference to non-existent subpattern
1427     16 erroffset passed as NULL
1428     17 unknown option bit(s) set
1429     18 missing ) after comment
1430 nigel 93 19 [this code is not in use]
1431 ph10 292 20 regular expression is too large
1432 nigel 77 21 failed to get memory
1433     22 unmatched parentheses
1434     23 internal error: code overflow
1435     24 unrecognized character after (?<
1436     25 lookbehind assertion is not fixed length
1437 nigel 91 26 malformed number or name after (?(
1438 nigel 77 27 conditional group contains more than two branches
1439     28 assertion expected after (?(
1440 ph10 182 29 (?R or (?[+-]digits must be followed by )
1441 nigel 77 30 unknown POSIX class name
1442     31 POSIX collating elements are not supported
1443     32 this version of PCRE is not compiled with PCRE_UTF8 support
1444 nigel 93 33 [this code is not in use]
1445 nigel 77 34 character value in \x{...} sequence is too large
1446     35 invalid condition (?(0)
1447     36 \C not allowed in lookbehind assertion
1448     37 PCRE does not support \L, \l, \N, \U, or \u
1449     38 number after (?C is > 255
1450     39 closing ) for (?C expected
1451     40 recursive call could loop indefinitely
1452     41 unrecognized character after (?P
1453 nigel 93 42 syntax error in subpattern name (missing terminator)
1454 nigel 91 43 two named subpatterns have the same name
1455 nigel 77 44 invalid UTF-8 string
1456     45 support for \P, \p, and \X has not been compiled
1457     46 malformed \P or \p sequence
1458     47 unknown property name after \P or \p
1459 nigel 91 48 subpattern name is too long (maximum 32 characters)
1460 ph10 292 49 too many named subpatterns (maximum 10000)
1461 ph10 202 50 [this code is not in use]
1462 nigel 91 51 octal value is greater than \377 (not in UTF-8 mode)
1463 nigel 93 52 internal error: overran compiling workspace
1464 ph10 453 53 internal error: previously-checked referenced subpattern not
1465 nigel 93 found
1466     54 DEFINE group contains more than one branch
1467     55 repeating a DEFINE group is not allowed
1468 ph10 231 56 inconsistent NEWLINE options
1469 ph10 345 57 \g is not followed by a braced, angle-bracketed, or quoted
1470     name/number or by a plain number
1471     58 a numbered reference must not be zero
1472 ph10 292 59 (*VERB) with an argument is not supported
1473     60 (*VERB) not recognized
1474     61 number is too big
1475     62 subpattern name expected
1476     63 digit expected after (?+
1477 ph10 345 64 ] is an invalid data character in JavaScript compatibility mode
1478 nigel 77
1479 ph10 453 The numbers 32 and 10000 in errors 48 and 49 are defaults; different
1480 ph10 292 values may be used if the limits were changed when PCRE was built.
1481 nigel 77
1482 ph10 292
1483 nigel 63 STUDYING A PATTERN
1484 nigel 49
1485 nigel 77 pcre_extra *pcre_study(const pcre *code, int options
1486 nigel 73 const char **errptr);
1487 nigel 63
1488 ph10 453 If a compiled pattern is going to be used several times, it is worth
1489 nigel 75 spending more time analyzing it in order to speed up the time taken for
1490 ph10 453 matching. The function pcre_study() takes a pointer to a compiled pat-
1491 nigel 75 tern as its first argument. If studying the pattern produces additional
1492 ph10 453 information that will help speed up matching, pcre_study() returns a
1493     pointer to a pcre_extra block, in which the study_data field points to
1494 nigel 75 the results of the study.
1495 nigel 41
1496 nigel 75 The returned value from pcre_study() can be passed directly to
1497 ph10 461 pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con-
1498     tains other fields that can be set by the caller before the block is
1499     passed; these are described below in the section on matching a pattern.
1500 nigel 63
1501 ph10 461 If studying the pattern does not produce any useful information,
1502 nigel 75 pcre_study() returns NULL. In that circumstance, if the calling program
1503 ph10 461 wants to pass any of the other fields to pcre_exec() or
1504     pcre_dfa_exec(), it must set up its own pcre_extra block.
1505 nigel 41
1506 ph10 453 The second argument of pcre_study() contains option bits. At present,
1507 nigel 75 no options are defined, and this argument should always be zero.
1508    
1509 ph10 453 The third argument for pcre_study() is a pointer for an error message.
1510     If studying succeeds (even if no data is returned), the variable it
1511     points to is set to NULL. Otherwise it is set to point to a textual
1512 nigel 87 error message. This is a static string that is part of the library. You
1513 ph10 453 must not try to free it. You should test the error pointer for NULL
1514 nigel 87 after calling pcre_study(), to be sure that it has run successfully.
1515 nigel 41
1516 nigel 73 This is a typical call to pcre_study():
1517 nigel 53
1518 nigel 73 pcre_extra *pe;
1519     pe = pcre_study(
1520     re, /* result of pcre_compile() */
1521     0, /* no options exist */
1522     &error); /* set to NULL or points to a message */
1523 nigel 53
1524 ph10 461 Studying a pattern does two things: first, a lower bound for the length
1525     of subject string that is needed to match the pattern is computed. This
1526     does not mean that there are any strings of that length that match, but
1527     it does guarantee that no shorter strings match. The value is used by
1528     pcre_exec() and pcre_dfa_exec() to avoid wasting time by trying to
1529     match strings that are shorter than the lower bound. You can find out
1530     the value in a calling program via the pcre_fullinfo() function.
1531 nigel 41
1532 ph10 461 Studying a pattern is also useful for non-anchored patterns that do not
1533     have a single fixed starting character. A bitmap of possible starting
1534     bytes is created. This speeds up finding a position in the subject at
1535     which to start matching.
1536 nigel 41
1537 ph10 461
1538 nigel 63 LOCALE SUPPORT
1539 nigel 41
1540 ph10 461 PCRE handles caseless matching, and determines whether characters are
1541     letters, digits, or whatever, by reference to a set of tables, indexed
1542     by character value. When running in UTF-8 mode, this applies only to
1543     characters with codes less than 128. Higher-valued codes never match
1544     escapes such as \w or \d, but can be tested with \p if PCRE is built
1545     with Unicode character property support. The use of locales with Uni-
1546     code is discouraged. If you are handling characters with codes greater
1547     than 128, you should either use UTF-8 and Unicode, or use locales, but
1548 ph10 142 not try to mix the two.
1549 nigel 41
1550 ph10 461 PCRE contains an internal set of tables that are used when the final
1551     argument of pcre_compile() is NULL. These are sufficient for many
1552 ph10 142 applications. Normally, the internal tables recognize only ASCII char-
1553     acters. However, when PCRE is built, it is possible to cause the inter-
1554     nal tables to be rebuilt in the default "C" locale of the local system,
1555     which may cause them to be different.
1556 nigel 41
1557 ph10 461 The internal tables can always be overridden by tables supplied by the
1558 ph10 142 application that calls PCRE. These may be created in a different locale
1559 ph10 461 from the default. As more and more applications change to using Uni-
1560 ph10 142 code, the need for this locale support is expected to die away.
1561    
1562 ph10 461 External tables are built by calling the pcre_maketables() function,
1563     which has no arguments, in the relevant locale. The result can then be
1564     passed to pcre_compile() or pcre_exec() as often as necessary. For
1565     example, to build and use tables that are appropriate for the French
1566     locale (where accented characters with values greater than 128 are
1567 nigel 75 treated as letters), the following code could be used:
1568    
1569     setlocale(LC_CTYPE, "fr_FR");
1570 nigel 73 tables = pcre_maketables();
1571     re = pcre_compile(..., tables);
1572 nigel 41
1573 ph10 461 The locale name "fr_FR" is used on Linux and other Unix-like systems;
1574 ph10 142 if you are using Windows, the name for the French locale is "french".
1575    
1576 ph10 461 When pcre_maketables() runs, the tables are built in memory that is
1577     obtained via pcre_malloc. It is the caller's responsibility to ensure
1578     that the memory containing the tables remains available for as long as
1579 nigel 75 it is needed.
1580 nigel 41
1581 nigel 75 The pointer that is passed to pcre_compile() is saved with the compiled
1582 ph10 461 pattern, and the same tables are used via this pointer by pcre_study()
1583 nigel 75 and normally also by pcre_exec(). Thus, by default, for any single pat-
1584     tern, compilation, studying and matching all happen in the same locale,
1585     but different patterns can be compiled in different locales.
1586 nigel 41
1587 ph10 461 It is possible to pass a table pointer or NULL (indicating the use of
1588     the internal tables) to pcre_exec(). Although not intended for this
1589     purpose, this facility could be used to match a pattern in a different
1590 nigel 75 locale from the one in which it was compiled. Passing table pointers at
1591     run time is discussed below in the section on matching a pattern.
1592    
1593    
1594 nigel 63 INFORMATION ABOUT A PATTERN
1595 nigel 41
1596 nigel 73 int pcre_fullinfo(const pcre *code, const pcre_extra *extra,
1597     int what, void *where);
1598 nigel 63
1599 ph10 461 The pcre_fullinfo() function returns information about a compiled pat-
1600 nigel 73 tern. It replaces the obsolete pcre_info() function, which is neverthe-
1601     less retained for backwards compability (and is documented below).
1602 nigel 43
1603 ph10 461 The first argument for pcre_fullinfo() is a pointer to the compiled
1604     pattern. The second argument is the result of pcre_study(), or NULL if
1605     the pattern was not studied. The third argument specifies which piece
1606     of information is required, and the fourth argument is a pointer to a
1607     variable to receive the data. The yield of the function is zero for
1608 nigel 73 success, or one of the following negative numbers:
1609 nigel 41
1610 nigel 73 PCRE_ERROR_NULL the argument code was NULL
1611     the argument where was NULL
1612     PCRE_ERROR_BADMAGIC the "magic number" was not found
1613     PCRE_ERROR_BADOPTION the value of what was invalid
1614 nigel 53
1615 ph10 461 The "magic number" is placed at the start of each compiled pattern as
1616     an simple check against passing an arbitrary memory pointer. Here is a
1617     typical call of pcre_fullinfo(), to obtain the length of the compiled
1618 nigel 75 pattern:
1619 nigel 53
1620 nigel 73 int rc;
1621 nigel 91 size_t length;
1622 nigel 73 rc = pcre_fullinfo(
1623     re, /* result of pcre_compile() */
1624     pe, /* result of pcre_study(), or NULL */
1625     PCRE_INFO_SIZE, /* what is required */
1626     &length); /* where to put the data */
1627 nigel 43
1628 ph10 461 The possible values for the third argument are defined in pcre.h, and
1629 nigel 73 are as follows:
1630 nigel 43
1631 nigel 73 PCRE_INFO_BACKREFMAX
1632 nigel 41
1633 ph10 461 Return the number of the highest back reference in the pattern. The
1634     fourth argument should point to an int variable. Zero is returned if
1635 nigel 73 there are no back references.
1636 nigel 43
1637 nigel 73 PCRE_INFO_CAPTURECOUNT
1638 nigel 43
1639 ph10 461 Return the number of capturing subpatterns in the pattern. The fourth
1640 nigel 73 argument should point to an int variable.
1641 nigel 43
1642 nigel 77 PCRE_INFO_DEFAULT_TABLES
1643 nigel 75
1644 ph10 461 Return a pointer to the internal default character tables within PCRE.
1645     The fourth argument should point to an unsigned char * variable. This
1646 nigel 75 information call is provided for internal use by the pcre_study() func-
1647 ph10 461 tion. External callers can cause PCRE to use its internal tables by
1648 nigel 75 passing a NULL table pointer.
1649    
1650 nigel 73 PCRE_INFO_FIRSTBYTE
1651 nigel 43
1652 ph10 461 Return information about the first byte of any matched string, for a
1653     non-anchored pattern. The fourth argument should point to an int vari-
1654     able. (This option used to be called PCRE_INFO_FIRSTCHAR; the old name
1655 nigel 91 is still recognized for backwards compatibility.)
1656 nigel 41
1657 ph10 461 If there is a fixed first byte, for example, from a pattern such as
1658 nigel 93 (cat|cow|coyote), its value is returned. Otherwise, if either
1659 nigel 41
1660 ph10 461 (a) the pattern was compiled with the PCRE_MULTILINE option, and every
1661 nigel 73 branch starts with "^", or
1662 nigel 43
1663 nigel 73 (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not
1664     set (if it were set, the pattern would be anchored),
1665 nigel 41
1666 ph10 461 -1 is returned, indicating that the pattern matches only at the start
1667     of a subject string or after any newline within the string. Otherwise
1668 nigel 73 -2 is returned. For anchored patterns, -2 is returned.
1669 nigel 41
1670 nigel 73 PCRE_INFO_FIRSTTABLE
1671 nigel 41
1672 ph10 461 If the pattern was studied, and this resulted in the construction of a
1673 nigel 73 256-bit table indicating a fixed set of bytes for the first byte in any
1674 ph10 461 matching string, a pointer to the table is returned. Otherwise NULL is
1675     returned. The fourth argument should point to an unsigned char * vari-
1676 nigel 73 able.
1677 nigel 43
1678 ph10 227 PCRE_INFO_HASCRORLF
1679    
1680 ph10 461 Return 1 if the pattern contains any explicit matches for CR or LF
1681     characters, otherwise 0. The fourth argument should point to an int
1682     variable. An explicit match is either a literal CR or LF character, or
1683 ph10 243 \r or \n.
1684 ph10 227
1685 ph10 172 PCRE_INFO_JCHANGED
1686    
1687 ph10 461 Return 1 if the (?J) or (?-J) option setting is used in the pattern,
1688     otherwise 0. The fourth argument should point to an int variable. (?J)
1689 ph10 286 and (?-J) set and unset the local PCRE_DUPNAMES option, respectively.
1690 ph10 172
1691 nigel 73 PCRE_INFO_LASTLITERAL
1692 nigel 43
1693 ph10 461 Return the value of the rightmost literal byte that must exist in any
1694     matched string, other than at its start, if such a byte has been
1695 nigel 73 recorded. The fourth argument should point to an int variable. If there
1696 ph10 461 is no such byte, -1 is returned. For anchored patterns, a last literal
1697     byte is recorded only if it follows something of variable length. For
1698 nigel 73 example, for the pattern /^a\d+z\d+/ the returned value is "z", but for
1699     /^a\dz\d/ the returned value is -1.
1700 nigel 63
1701 ph10 461 PCRE_INFO_MINLENGTH
1702    
1703     If the pattern was studied and a minimum length for matching subject
1704     strings was computed, its value is returned. Otherwise the returned
1705     value is -1. The value is a number of characters, not bytes (this may
1706     be relevant in UTF-8 mode). The fourth argument should point to an int
1707     variable. A non-negative value is a lower bound to the length of any
1708     matching string. There may not be any strings of that length that do
1709     actually match, but every string that does match is at least that long.
1710    
1711 nigel 73 PCRE_INFO_NAMECOUNT
1712     PCRE_INFO_NAMEENTRYSIZE
1713     PCRE_INFO_NAMETABLE
1714 nigel 63
1715 ph10 461 PCRE supports the use of named as well as numbered capturing parenthe-
1716     ses. The names are just an additional way of identifying the parenthe-
1717 nigel 91 ses, which still acquire numbers. Several convenience functions such as
1718 ph10 461 pcre_get_named_substring() are provided for extracting captured sub-
1719     strings by name. It is also possible to extract the data directly, by
1720     first converting the name to a number in order to access the correct
1721 nigel 91 pointers in the output vector (described with pcre_exec() below). To do
1722 ph10 461 the conversion, you need to use the name-to-number map, which is
1723 nigel 91 described by these three values.
1724 nigel 63
1725 nigel 73 The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT
1726     gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size
1727 ph10 461 of each entry; both of these return an int value. The entry size
1728     depends on the length of the longest name. PCRE_INFO_NAMETABLE returns
1729     a pointer to the first entry of the table (a pointer to char). The
1730 nigel 73 first two bytes of each entry are the number of the capturing parenthe-
1731 ph10 461 sis, most significant byte first. The rest of the entry is the corre-
1732     sponding name, zero terminated.
1733 nigel 63
1734 ph10 461 The names are in alphabetical order. Duplicate names may appear if (?|
1735     is used to create multiple groups with the same number, as described in
1736     the section on duplicate subpattern numbers in the pcrepattern page.
1737     Duplicate names for subpatterns with different numbers are permitted
1738     only if PCRE_DUPNAMES is set. In all cases of duplicate names, they
1739     appear in the table in the order in which they were found in the pat-
1740     tern. In the absence of (?| this is the order of increasing number;
1741     when (?| is used this is not necessarily the case because later subpat-
1742     terns may have lower numbers.
1743    
1744     As a simple example of the name/number table, consider the following
1745     pattern (assume PCRE_EXTENDED is set, so white space - including new-
1746     lines - is ignored):
1747    
1748 nigel 93 (?<date> (?<year>(\d\d)?\d\d) -
1749     (?<month>\d\d) - (?<day>\d\d) )
1750 nigel 63
1751 ph10 461 There are four named subpatterns, so the table has four entries, and
1752     each entry in the table is eight bytes long. The table is as follows,
1753 nigel 75 with non-printing bytes shows in hexadecimal, and undefined bytes shown
1754     as ??:
1755 nigel 63
1756 nigel 73 00 01 d a t e 00 ??
1757     00 05 d a y 00 ?? ??
1758     00 04 m o n t h 00
1759     00 02 y e a r 00 ??
1760 nigel 63
1761 ph10 461 When writing code to extract data from named subpatterns using the
1762     name-to-number map, remember that the length of the entries is likely
1763 nigel 91 to be different for each compiled pattern.
1764 nigel 63
1765 ph10 172 PCRE_INFO_OKPARTIAL
1766    
1767 ph10 461 Return 1 if the pattern can be used for partial matching with
1768     pcre_exec(), otherwise 0. The fourth argument should point to an int
1769     variable. From release 8.00, this always returns 1, because the
1770     restrictions that previously applied to partial matching have been
1771     lifted. The pcrepartial documentation gives details of partial match-
1772 ph10 453 ing.
1773 ph10 172
1774 nigel 73 PCRE_INFO_OPTIONS
1775 nigel 63
1776 ph10 461 Return a copy of the options with which the pattern was compiled. The
1777     fourth argument should point to an unsigned long int variable. These
1778 nigel 73 option bits are those specified in the call to pcre_compile(), modified
1779 ph10 197 by any top-level option settings at the start of the pattern itself. In
1780 ph10 461 other words, they are the options that will be in force when matching
1781     starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with
1782     the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE,
1783 ph10 197 and PCRE_EXTENDED.
1784 nigel 63
1785 ph10 461 A pattern is automatically anchored by PCRE if all of its top-level
1786 nigel 73 alternatives begin with one of the following:
1787 nigel 63
1788 nigel 73 ^ unless PCRE_MULTILINE is set
1789     \A always
1790     \G always
1791     .* if PCRE_DOTALL is set and there are no back
1792     references to the subpattern in which .* appears
1793 nigel 63
1794 nigel 73 For such patterns, the PCRE_ANCHORED bit is set in the options returned
1795     by pcre_fullinfo().
1796 nigel 63
1797 nigel 73 PCRE_INFO_SIZE
1798 nigel 63
1799 ph10 461 Return the size of the compiled pattern, that is, the value that was
1800 nigel 73 passed as the argument to pcre_malloc() when PCRE was getting memory in
1801     which to place the compiled data. The fourth argument should point to a
1802     size_t variable.
1803 nigel 63
1804 nigel 73 PCRE_INFO_STUDYSIZE
1805 nigel 63
1806 nigel 75 Return the size of the data block pointed to by the study_data field in
1807 ph10 461 a pcre_extra block. That is, it is the value that was passed to
1808 nigel 73 pcre_malloc() when PCRE was getting memory into which to place the data
1809 ph10 461 created by pcre_study(). If pcre_extra is NULL, or there is no study
1810     data, zero is returned. The fourth argument should point to a size_t
1811 nigel 73 variable.
1812 nigel 63
1813 nigel 73
1814 nigel 63 OBSOLETE INFO FUNCTION
1815    
1816 nigel 73 int pcre_info(const pcre *code, int *optptr, int *firstcharptr);
1817 nigel 63
1818 ph10 416 The pcre_info() function is now obsolete because its interface is too
1819     restrictive to return all the available data about a compiled pattern.
1820     New programs should use pcre_fullinfo() instead. The yield of
1821     pcre_info() is the number of capturing subpatterns, or one of the fol-
1822 nigel 73 lowing negative numbers:
1823 nigel 43
1824 nigel 73 PCRE_ERROR_NULL the argument code was NULL
1825     PCRE_ERROR_BADMAGIC the "magic number" was not found
1826 nigel 43
1827 ph10 416 If the optptr argument is not NULL, a copy of the options with which
1828     the pattern was compiled is placed in the integer it points to (see
1829 nigel 73 PCRE_INFO_OPTIONS above).
1830 nigel 43
1831 ph10 416 If the pattern is not anchored and the firstcharptr argument is not
1832     NULL, it is used to pass back information about the first character of
1833 nigel 73 any matched string (see PCRE_INFO_FIRSTBYTE above).
1834 nigel 43
1835    
1836 nigel 77 REFERENCE COUNTS
1837 nigel 53
1838 nigel 77 int pcre_refcount(pcre *code, int adjust);
1839    
1840 ph10 416 The pcre_refcount() function is used to maintain a reference count in
1841 nigel 77 the data block that contains a compiled pattern. It is provided for the
1842 ph10 416 benefit of applications that operate in an object-oriented manner,
1843 nigel 77 where different parts of the application may be using the same compiled
1844     pattern, but you want to free the block when they are all done.
1845    
1846     When a pattern is compiled, the reference count field is initialized to
1847 ph10 416 zero. It is changed only by calling this function, whose action is to
1848     add the adjust value (which may be positive or negative) to it. The
1849 nigel 77 yield of the function is the new value. However, the value of the count
1850 ph10 416 is constrained to lie between 0 and 65535, inclusive. If the new value
1851 nigel 77 is outside these limits, it is forced to the appropriate limit value.
1852    
1853 ph10 416 Except when it is zero, the reference count is not correctly preserved
1854     if a pattern is compiled on one host and then transferred to a host
1855 nigel 77 whose byte-order is different. (This seems a highly unlikely scenario.)
1856    
1857    
1858     MATCHING A PATTERN: THE TRADITIONAL FUNCTION
1859    
1860 nigel 73 int pcre_exec(const pcre *code, const pcre_extra *extra,
1861     const char *subject, int length, int startoffset,
1862     int options, int *ovector, int ovecsize);
1863 nigel 53
1864 ph10 416 The function pcre_exec() is called to match a subject string against a
1865     compiled pattern, which is passed in the code argument. If the pattern
1866 ph10 461 was studied, the result of the study should be passed in the extra
1867 ph10 416 argument. This function is the main matching facility of the library,
1868 nigel 77 and it operates in a Perl-like manner. For specialist use there is also
1869 ph10 416 an alternative matching function, which is described below in the sec-
1870 nigel 77 tion about the pcre_dfa_exec() function.
1871 nigel 41
1872 ph10 416 In most applications, the pattern will have been compiled (and option-
1873     ally studied) in the same process that calls pcre_exec(). However, it
1874 nigel 75 is possible to save compiled patterns and study data, and then use them
1875 ph10 416 later in different processes, possibly even on different hosts. For a
1876 nigel 75 discussion about this, see the pcreprecompile documentation.
1877    
1878 nigel 73 Here is an example of a simple call to pcre_exec():
1879 nigel 53
1880 nigel 73 int rc;
1881     int ovector[30];
1882     rc = pcre_exec(
1883     re, /* result of pcre_compile() */
1884     NULL, /* we didn't study the pattern */
1885     "some string", /* the subject string */
1886     11, /* the length of the subject string */
1887     0, /* start at offset 0 in the subject */
1888     0, /* default options */
1889 nigel 75 ovector, /* vector of integers for substring information */
1890 nigel 77 30); /* number of elements (NOT size in bytes) */
1891 nigel 53
1892 nigel 75 Extra data for pcre_exec()
1893 nigel 63
1894 ph10 416 If the extra argument is not NULL, it must point to a pcre_extra data
1895     block. The pcre_study() function returns such a block (when it doesn't
1896     return NULL), but you can also create one for yourself, and pass addi-
1897     tional information in it. The pcre_extra block contains the following
1898 nigel 87 fields (not necessarily in this order):
1899 nigel 75
1900 nigel 73 unsigned long int flags;
1901     void *study_data;
1902     unsigned long int match_limit;
1903 nigel 87 unsigned long int match_limit_recursion;
1904 nigel 73 void *callout_data;
1905 nigel 75 const unsigned char *tables;
1906 nigel 63
1907 ph10 416 The flags field is a bitmap that specifies which of the other fields
1908 nigel 73 are set. The flag bits are:
1909 nigel 63
1910 nigel 73 PCRE_EXTRA_STUDY_DATA
1911     PCRE_EXTRA_MATCH_LIMIT
1912 nigel 87 PCRE_EXTRA_MATCH_LIMIT_RECURSION
1913 nigel 73 PCRE_EXTRA_CALLOUT_DATA
1914 nigel 75 PCRE_EXTRA_TABLES
1915 nigel 63
1916 ph10 416 Other flag bits should be set to zero. The study_data field is set in
1917     the pcre_extra block that is returned by pcre_study(), together with
1918 nigel 75 the appropriate flag bit. You should not set this yourself, but you may
1919 ph10 416 add to the block by setting the other fields and their corresponding
1920 nigel 75 flag bits.
1921 nigel 63
1922 nigel 73 The match_limit field provides a means of preventing PCRE from using up
1923 ph10 416 a vast amount of resources when running patterns that are not going to
1924     match, but which have a very large number of possibilities in their
1925 ph10 461 search trees. The classic example is a pattern that uses nested unlim-
1926     ited repeats.
1927 nigel 63
1928 ph10 416 Internally, PCRE uses a function called match() which it calls repeat-
1929     edly (sometimes recursively). The limit set by match_limit is imposed
1930     on the number of times this function is called during a match, which
1931     has the effect of limiting the amount of backtracking that can take
1932 nigel 87 place. For patterns that are not anchored, the count restarts from zero
1933     for each position in the subject string.
1934 nigel 75
1935 ph10 416 The default value for the limit can be set when PCRE is built; the
1936     default default is 10 million, which handles all but the most extreme
1937     cases. You can override the default by suppling pcre_exec() with a
1938     pcre_extra block in which match_limit is set, and
1939     PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is
1940 nigel 73 exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT.
1941 nigel 63
1942 ph10 416 The match_limit_recursion field is similar to match_limit, but instead
1943 nigel 87 of limiting the total number of times that match() is called, it limits
1944 ph10 416 the depth of recursion. The recursion depth is a smaller number than
1945     the total number of calls, because not all calls to match() are recur-
1946 nigel 87 sive. This limit is of use only if it is set smaller than match_limit.
1947    
1948 ph10 416 Limiting the recursion depth limits the amount of stack that can be
1949 nigel 87 used, or, when PCRE has been compiled to use memory on the heap instead
1950     of the stack, the amount of heap memory that can be used.
1951    
1952 ph10 416 The default value for match_limit_recursion can be set when PCRE is
1953     built; the default default is the same value as the default for
1954     match_limit. You can override the default by suppling pcre_exec() with
1955     a pcre_extra block in which match_limit_recursion is set, and
1956     PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the
1957 nigel 87 limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT.
1958    
1959 ph10 453 The callout_data field is used in conjunction with the "callout" fea-
1960     ture, and is described in the pcrecallout documentation.
1961 nigel 63
1962 ph10 416 The tables field is used to pass a character tables pointer to
1963     pcre_exec(); this overrides the value that is stored with the compiled
1964     pattern. A non-NULL value is stored with the compiled pattern only if
1965     custom tables were supplied to pcre_compile() via its tableptr argu-
1966 nigel 75 ment. If NULL is passed to pcre_exec() using this mechanism, it forces
1967 ph10 416 PCRE's internal tables to be used. This facility is helpful when re-
1968     using patterns that have been saved after compiling with an external
1969     set of tables, because the external tables might be at a different
1970     address when pcre_exec() is called. See the pcreprecompile documenta-
1971 nigel 75 tion for a discussion of saving compiled patterns for later use.
1972 nigel 41
1973 nigel 75 Option bits for pcre_exec()
1974 nigel 71
1975 ph10 416 The unused bits of the options argument for pcre_exec() must be zero.
1976     The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx,
1977 ph10 453 PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
1978     PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and
1979     PCRE_PARTIAL_HARD.
1980 nigel 41
1981 nigel 75 PCRE_ANCHORED
1982 nigel 41
1983 ph10 453 The PCRE_ANCHORED option limits pcre_exec() to matching at the first
1984     matching position. If a pattern was compiled with PCRE_ANCHORED, or
1985     turned out to be anchored by virtue of its contents, it cannot be made
1986 nigel 75 unachored at matching time.
1987    
1988 ph10 231 PCRE_BSR_ANYCRLF
1989     PCRE_BSR_UNICODE
1990    
1991     These options (which are mutually exclusive) control what the \R escape
1992 ph10 453 sequence matches. The choice is either to match only CR, LF, or CRLF,
1993     or to match any Unicode newline sequence. These options override the
1994 ph10 231 choice that was made or defaulted when the pattern was compiled.
1995    
1996 nigel 91 PCRE_NEWLINE_CR
1997     PCRE_NEWLINE_LF
1998     PCRE_NEWLINE_CRLF
1999 ph10 150 PCRE_NEWLINE_ANYCRLF
2000 nigel 93 PCRE_NEWLINE_ANY
2001 nigel 91
2002 ph10 453 These options override the newline definition that was chosen or
2003     defaulted when the pattern was compiled. For details, see the descrip-
2004     tion of pcre_compile() above. During matching, the newline choice
2005     affects the behaviour of the dot, circumflex, and dollar metacharac-
2006     ters. It may also alter the way the match position is advanced after a
2007 ph10 227 match failure for an unanchored pattern.
2008 nigel 91
2009 ph10 453 When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is
2010     set, and a match attempt for an unanchored pattern fails when the cur-
2011     rent position is at a CRLF sequence, and the pattern contains no
2012     explicit matches for CR or LF characters, the match position is
2013 ph10 227 advanced by two characters instead of one, in other words, to after the
2014     CRLF.
2015    
2016     The above rule is a compromise that makes the most common cases work as
2017 ph10 453 expected. For example, if the pattern is .+A (and the PCRE_DOTALL
2018 ph10 227 option is not set), it does not match the string "\r\nA" because, after
2019 ph10 453 failing at the start, it skips both the CR and the LF before retrying.
2020     However, the pattern [\r\n]A does match that string, because it con-
2021 ph10 227 tains an explicit CR or LF reference, and so advances only by one char-
2022 ph10 231 acter after the first failure.
2023 ph10 227
2024 ph10 231 An explicit match for CR of LF is either a literal appearance of one of
2025 ph10 453 those characters, or one of the \r or \n escape sequences. Implicit
2026     matches such as [^X] do not count, nor does \s (which includes CR and
2027 ph10 231 LF in the characters that it matches).
2028    
2029 ph10 453 Notwithstanding the above, anomalous effects may still occur when CRLF
2030 ph10 227 is a valid newline sequence and explicit \r or \n escapes appear in the
2031     pattern.
2032    
2033 nigel 73 PCRE_NOTBOL
2034 nigel 41
2035 nigel 75 This option specifies that first character of the subject string is not
2036 ph10 453 the beginning of a line, so the circumflex metacharacter should not
2037     match before it. Setting this without PCRE_MULTILINE (at compile time)
2038     causes circumflex never to match. This option affects only the behav-
2039 nigel 77 iour of the circumflex metacharacter. It does not affect \A.
2040 nigel 41
2041 nigel 73 PCRE_NOTEOL
2042 nigel 41
2043 nigel 75 This option specifies that the end of the subject string is not the end
2044 ph10 453 of a line, so the dollar metacharacter should not match it nor (except
2045     in multiline mode) a newline immediately before it. Setting this with-
2046 nigel 75 out PCRE_MULTILINE (at compile time) causes dollar never to match. This
2047 ph10 453 option affects only the behaviour of the dollar metacharacter. It does
2048 nigel 75 not affect \Z or \z.
2049 nigel 41
2050 nigel 73 PCRE_NOTEMPTY
2051 nigel 41
2052 nigel 73 An empty string is not considered to be a valid match if this option is
2053 ph10 453 set. If there are alternatives in the pattern, they are tried. If all
2054     the alternatives match the empty string, the entire match fails. For
2055 nigel 73 example, if the pattern
2056 nigel 41
2057 nigel 73 a?b?
2058 nigel 41
2059 ph10 453 is applied to a string not beginning with "a" or "b", it matches an
2060     empty string at the start of the subject. With PCRE_NOTEMPTY set, this
2061 nigel 73 match is not valid, so PCRE searches further into the string for occur-
2062     rences of "a" or "b".
2063 nigel 41
2064 ph10 453 PCRE_NOTEMPTY_ATSTART
2065 nigel 41
2066 ph10 453 This is like PCRE_NOTEMPTY, except that an empty string match that is
2067     not at the start of the subject is permitted. If the pattern is
2068     anchored, such a match can occur only if the pattern contains \K.
2069    
2070     Perl has no direct equivalent of PCRE_NOTEMPTY or
2071     PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern
2072     match of the empty string within its split() function, and when using
2073     the /g modifier. It is possible to emulate Perl's behaviour after
2074     matching a null string by first trying the match again at the same off-
2075     set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that
2076     fails, by advancing the starting offset (see below) and trying an ordi-
2077     nary match again. There is some code that demonstrates how to do this
2078     in the pcredemo sample program.
2079    
2080 ph10 392 PCRE_NO_START_OPTIMIZE
2081    
2082 ph10 416 There are a number of optimizations that pcre_exec() uses at the start
2083     of a match, in order to speed up the process. For example, if it is
2084     known that a match must start with a specific character, it searches
2085 ph10 392 the subject for that character, and fails immediately if it cannot find
2086 ph10 416 it, without actually running the main matching function. When callouts
2087     are in use, these optimizations can cause them to be skipped. This
2088     option disables the "start-up" optimizations, causing performance to
2089 ph10 392 suffer, but ensuring that the callouts do occur.
2090    
2091 nigel 75 PCRE_NO_UTF8_CHECK
2092    
2093     When PCRE_UTF8 is set at compile time, the validity of the subject as a
2094 ph10 416 UTF-8 string is automatically checked when pcre_exec() is subsequently
2095     called. The value of startoffset is also checked to ensure that it
2096     points to the start of a UTF-8 character. There is a discussion about
2097     the validity of UTF-8 strings in the section on UTF-8 support in the
2098     main pcre page. If an invalid UTF-8 sequence of bytes is found,
2099     pcre_exec() returns the error PCRE_ERROR_BADUTF8. If startoffset con-
2100 ph10 211 tains an invalid value, PCRE_ERROR_BADUTF8_OFFSET is returned.
2101 nigel 75
2102 ph10 416 If you already know that your subject is valid, and you want to skip
2103     these checks for performance reasons, you can set the
2104     PCRE_NO_UTF8_CHECK option when calling pcre_exec(). You might want to
2105     do this for the second and subsequent calls to pcre_exec() if you are
2106     making repeated calls to find all the matches in a single subject
2107     string. However, you should be sure that the value of startoffset
2108     points to the start of a UTF-8 character. When PCRE_NO_UTF8_CHECK is
2109     set, the effect of passing an invalid UTF-8 string as a subject, or a
2110     value of startoffset that does not point to the start of a UTF-8 char-
2111 nigel 75 acter, is undefined. Your program may crash.
2112    
2113 ph10 429 PCRE_PARTIAL_HARD
2114     PCRE_PARTIAL_SOFT
2115 nigel 75
2116 ph10 429 These options turn on the partial matching feature. For backwards com-
2117     patibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial
2118     match occurs if the end of the subject string is reached successfully,
2119     but there are not enough subject characters to complete the match. If
2120     this happens when PCRE_PARTIAL_HARD is set, pcre_exec() immediately
2121     returns PCRE_ERROR_PARTIAL. Otherwise, if PCRE_PARTIAL_SOFT is set,
2122     matching continues by testing any other alternatives. Only if they all
2123     fail is PCRE_ERROR_PARTIAL returned (instead of PCRE_ERROR_NOMATCH).
2124 ph10 453 The portion of the string that was inspected when the partial match was
2125     found is set as the first matching string. There is a more detailed
2126     discussion in the pcrepartial documentation.
2127 nigel 75
2128     The string to be matched by pcre_exec()
2129    
2130 ph10 429 The subject string is passed to pcre_exec() as a pointer in subject, a
2131 ph10 371 length (in bytes) in length, and a starting byte offset in startoffset.
2132     In UTF-8 mode, the byte offset must point to the start of a UTF-8 char-
2133 ph10 429 acter. Unlike the pattern string, the subject may contain binary zero
2134     bytes. When the starting offset is zero, the search for a match starts
2135     at the beginning of the subject, and this is by far the most common
2136 ph10 371 case.
2137 nigel 63
2138 ph10 429 A non-zero starting offset is useful when searching for another match
2139     in the same subject by calling pcre_exec() again after a previous suc-
2140     cess. Setting startoffset differs from just passing over a shortened
2141     string and setting PCRE_NOTBOL in the case of a pattern that begins
2142 nigel 73 with any kind of lookbehind. For example, consider the pattern
2143 nigel 41
2144 nigel 73 \Biss\B
2145 nigel 41
2146 ph10 429 which finds occurrences of "iss" in the middle of words. (\B matches
2147     only if the current position in the subject is not a word boundary.)
2148     When applied to the string "Mississipi" the first call to pcre_exec()
2149     finds the first occurrence. If pcre_exec() is called again with just
2150     the remainder of the subject, namely "issipi", it does not match,
2151 nigel 73 because \B is always false at the start of the subject, which is deemed
2152 ph10 429 to be a word boundary. However, if pcre_exec() is passed the entire
2153 nigel 75 string again, but with startoffset set to 4, it finds the second occur-
2154 ph10 429 rence of "iss" because it is able to look behind the starting point to
2155 nigel 75 discover that it is preceded by a letter.
2156 nigel 41
2157 ph10 429 If a non-zero starting offset is passed when the pattern is anchored,
2158 nigel 75 one attempt to match at the given offset is made. This can only succeed
2159 ph10 429 if the pattern does not require the match to be at the start of the
2160 nigel 75 subject.
2161 nigel 41
2162 nigel 75 How pcre_exec() returns captured substrings
2163    
2164 ph10 429 In general, a pattern matches a certain portion of the subject, and in
2165     addition, further substrings from the subject may be picked out by
2166     parts of the pattern. Following the usage in Jeffrey Friedl's book,
2167     this is called "capturing" in what follows, and the phrase "capturing
2168     subpattern" is used for a fragment of a pattern that picks out a sub-
2169     string. PCRE supports several other kinds of parenthesized subpattern
2170 nigel 73 that do not cause substrings to be captured.
2171 nigel 65
2172 ph10 371 Captured substrings are returned to the caller via a vector of integers
2173 ph10 429 whose address is passed in ovector. The number of elements in the vec-
2174     tor is passed in ovecsize, which must be a non-negative number. Note:
2175 ph10 371 this argument is NOT the size of ovector in bytes.
2176 nigel 41
2177 ph10 429 The first two-thirds of the vector is used to pass back captured sub-
2178     strings, each substring using a pair of integers. The remaining third
2179     of the vector is used as workspace by pcre_exec() while matching cap-
2180     turing subpatterns, and is not available for passing back information.
2181     The number passed in ovecsize should always be a multiple of three. If
2182 nigel 75 it is not, it is rounded down.
2183    
2184 ph10 429 When a match is successful, information about captured substrings is
2185     returned in pairs of integers, starting at the beginning of ovector,
2186     and continuing up to two-thirds of its length at the most. The first
2187     element of each pair is set to the byte offset of the first character
2188     in a substring, and the second is set to the byte offset of the first
2189     character after the end of a substring. Note: these values are always
2190 ph10 371 byte offsets, even in UTF-8 mode. They are not character counts.
2191 nigel 41
2192 ph10 429 The first pair of integers, ovector[0] and ovector[1], identify the
2193     portion of the subject string matched by the entire pattern. The next
2194     pair is used for the first capturing subpattern, and so on. The value
2195 ph10 371 returned by pcre_exec() is one more than the highest numbered pair that
2196 ph10 429 has been set. For example, if two substrings have been captured, the
2197     returned value is 3. If there are no capturing subpatterns, the return
2198 ph10 371 value from a successful match is 1, indicating that just the first pair
2199     of offsets has been set.
2200    
2201 nigel 73 If a capturing subpattern is matched repeatedly, it is the last portion
2202 nigel 75 of the string that it matched that is returned.
2203 nigel 41
2204 ph10 429 If the vector is too small to hold all the captured substring offsets,
2205 nigel 75 it is used as far as possible (up to two-thirds of its length), and the
2206 ph10 429 function returns a value of zero. If the substring offsets are not of
2207     interest, pcre_exec() may be called with ovector passed as NULL and
2208     ovecsize as zero. However, if the pattern contains back references and
2209     the ovector is not big enough to remember the related substrings, PCRE
2210     has to get additional memory for use during matching. Thus it is usu-
2211 ph10 371 ally advisable to supply an ovector.
2212 nigel 41
2213 ph10 461 The pcre_fullinfo() function can be used to find out how many capturing
2214 ph10 429 subpatterns there are in a compiled pattern. The smallest size for
2215     ovector that will allow for n captured substrings, in addition to the
2216 nigel 91 offsets of the substring matched by the whole pattern, is (n+1)*3.
2217 nigel 41
2218 ph10 429 It is possible for capturing subpattern number n+1 to match some part
2219 nigel 91 of the subject when subpattern n has not been used at all. For example,
2220 ph10 429 if the string "abc" is matched against the pattern (a|(z))(bc) the
2221 nigel 91 return from the function is 4, and subpatterns 1 and 3 are matched, but
2222 ph10 429 2 is not. When this happens, both values in the offset pairs corre-
2223 nigel 91 sponding to unused subpatterns are set to -1.
2224 nigel 75
2225 ph10 429 Offset values that correspond to unused subpatterns at the end of the
2226     expression are also set to -1. For example, if the string "abc" is
2227     matched against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not
2228     matched. The return from the function is 2, because the highest used
2229 nigel 91 capturing subpattern number is 1. However, you can refer to the offsets
2230 ph10 429 for the second and third capturing subpatterns if you wish (assuming
2231 nigel 91 the vector is large enough, of course).
2232    
2233 ph10 429 Some convenience functions are provided for extracting the captured
2234 nigel 91 substrings as separate strings. These are described below.
2235    
2236     Error return values from pcre_exec()
2237    
2238 ph10 429 If pcre_exec() fails, it returns a negative number. The following are
2239 nigel 73 defined in the header file:
2240 nigel 41
2241 nigel 73 PCRE_ERROR_NOMATCH (-1)
2242 nigel 41
2243 nigel 73 The subject string did not match the pattern.
2244 nigel 41
2245 nigel 73 PCRE_ERROR_NULL (-2)
2246 nigel 41
2247 ph10 429 Either code or subject was passed as NULL, or ovector was NULL and
2248 nigel 73 ovecsize was not zero.
2249 nigel 41
2250 nigel 73 PCRE_ERROR_BADOPTION (-3)
2251 nigel 41
2252 nigel 73 An unrecognized bit was set in the options argument.
2253 nigel 41
2254 nigel 73 PCRE_ERROR_BADMAGIC (-4)
2255 nigel 41
2256 ph10 429 PCRE stores a 4-byte "magic number" at the start of the compiled code,
2257 nigel 75 to catch the case when it is passed a junk pointer and to detect when a
2258     pattern that was compiled in an environment of one endianness is run in
2259 ph10 429 an environment with the other endianness. This is the error that PCRE
2260 nigel 75 gives when the magic number is not present.
2261 nigel 41
2262 nigel 93 PCRE_ERROR_UNKNOWN_OPCODE (-5)
2263 nigel 41
2264 nigel 73 While running the pattern match, an unknown item was encountered in the
2265 ph10 429 compiled pattern. This error could be caused by a bug in PCRE or by
2266 nigel 73 overwriting of the compiled pattern.
2267 nigel 41
2268 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2269 nigel 41
2270 ph10 429 If a pattern contains back references, but the ovector that is passed
2271 nigel 73 to pcre_exec() is not big enough to remember the referenced substrings,
2272 ph10 429 PCRE gets a block of memory at the start of matching to use for this
2273     purpose. If the call via pcre_malloc() fails, this error is given. The
2274 nigel 75 memory is automatically freed at the end of matching.
2275 nigel 41
2276 nigel 73 PCRE_ERROR_NOSUBSTRING (-7)
2277 nigel 53
2278 ph10 429 This error is used by the pcre_copy_substring(), pcre_get_substring(),
2279 nigel 73 and pcre_get_substring_list() functions (see below). It is never
2280     returned by pcre_exec().
2281 nigel 63
2282 nigel 73 PCRE_ERROR_MATCHLIMIT (-8)
2283 nigel 63
2284 ph10 429 The backtracking limit, as specified by the match_limit field in a
2285     pcre_extra structure (or defaulted) was reached. See the description
2286 nigel 87 above.
2287    
2288 nigel 73 PCRE_ERROR_CALLOUT (-9)
2289 nigel 63
2290 nigel 73 This error is never generated by pcre_exec() itself. It is provided for
2291 ph10 429 use by callout functions that want to yield a distinctive error code.
2292 nigel 73 See the pcrecallout documentation for details.
2293 nigel 71
2294 nigel 73 PCRE_ERROR_BADUTF8 (-10)
2295 nigel 71
2296 ph10 429 A string that contains an invalid UTF-8 byte sequence was passed as a
2297 nigel 73 subject.
2298    
2299     PCRE_ERROR_BADUTF8_OFFSET (-11)
2300    
2301     The UTF-8 byte sequence that was passed as a subject was valid, but the
2302 ph10 429 value of startoffset did not point to the beginning of a UTF-8 charac-
2303 nigel 73 ter.
2304    
2305 nigel 77 PCRE_ERROR_PARTIAL (-12)
2306 nigel 73
2307 ph10 429 The subject string did not match, but it did match partially. See the
2308 nigel 75 pcrepartial documentation for details of partial matching.
2309    
2310 nigel 77 PCRE_ERROR_BADPARTIAL (-13)
2311 nigel 75
2312 ph10 429 This code is no longer in use. It was formerly returned when the
2313     PCRE_PARTIAL option was used with a compiled pattern containing items
2314     that were not supported for partial matching. From release 8.00
2315     onwards, there are no restrictions on partial matching.
2316 nigel 75
2317 nigel 77 PCRE_ERROR_INTERNAL (-14)
2318 nigel 75
2319 ph10 416 An unexpected internal error has occurred. This error could be caused
2320 nigel 75 by a bug in PCRE or by overwriting of the compiled pattern.
2321    
2322 nigel 77 PCRE_ERROR_BADCOUNT (-15)
2323 nigel 75
2324 ph10 392 This error is given if the value of the ovecsize argument is negative.
2325 nigel 75
2326 nigel 93 PCRE_ERROR_RECURSIONLIMIT (-21)
2327 nigel 75
2328 nigel 93 The internal recursion limit, as specified by the match_limit_recursion
2329 ph10 416 field in a pcre_extra structure (or defaulted) was reached. See the
2330 nigel 93 description above.
2331    
2332     PCRE_ERROR_BADNEWLINE (-23)
2333    
2334     An invalid combination of PCRE_NEWLINE_xxx options was given.
2335    
2336 ph10 197 Error numbers -16 to -20 and -22 are not used by pcre_exec().
2337 nigel 93
2338    
2339 nigel 63 EXTRACTING CAPTURED SUBSTRINGS BY NUMBER
2340    
2341 nigel 73 int pcre_copy_substring(const char *subject, int *ovector,
2342     int stringcount, int stringnumber, char *buffer,
2343     int buffersize);
2344 nigel 63
2345 nigel 73 int pcre_get_substring(const char *subject, int *ovector,
2346     int stringcount, int stringnumber,
2347     const char **stringptr);
2348 nigel 63
2349 nigel 73 int pcre_get_substring_list(const char *subject,
2350     int *ovector, int stringcount, const char ***listptr);
2351 nigel 63
2352 ph10 416 Captured substrings can be accessed directly by using the offsets
2353     returned by pcre_exec() in ovector. For convenience, the functions
2354 nigel 73 pcre_copy_substring(), pcre_get_substring(), and pcre_get_sub-
2355 ph10 416 string_list() are provided for extracting captured substrings as new,
2356     separate, zero-terminated strings. These functions identify substrings
2357     by number. The next section describes functions for extracting named
2358 nigel 91 substrings.
2359 nigel 41
2360 ph10 416 A substring that contains a binary zero is correctly extracted and has
2361     a further zero added on the end, but the result is not, of course, a C
2362     string. However, you can process such a string by referring to the
2363     length that is returned by pcre_copy_substring() and pcre_get_sub-
2364 nigel 91 string(). Unfortunately, the interface to pcre_get_substring_list() is
2365 ph10 416 not adequate for handling strings containing binary zeros, because the
2366 nigel 91 end of the final string is not independently indicated.
2367    
2368 ph10 416 The first three arguments are the same for all three of these func-
2369     tions: subject is the subject string that has just been successfully
2370 nigel 73 matched, ovector is a pointer to the vector of integer offsets that was
2371     passed to pcre_exec(), and stringcount is the number of substrings that
2372 ph10 416 were captured by the match, including the substring that matched the
2373 nigel 75 entire regular expression. This is the value returned by pcre_exec() if
2374 ph10 416 it is greater than zero. If pcre_exec() returned zero, indicating that
2375     it ran out of space in ovector, the value passed as stringcount should
2376 nigel 75 be the number of elements in the vector divided by three.
2377 nigel 41
2378 ph10 416 The functions pcre_copy_substring() and pcre_get_substring() extract a
2379     single substring, whose number is given as stringnumber. A value of
2380     zero extracts the substring that matched the entire pattern, whereas
2381     higher values extract the captured substrings. For pcre_copy_sub-
2382     string(), the string is placed in buffer, whose length is given by
2383     buffersize, while for pcre_get_substring() a new block of memory is
2384     obtained via pcre_malloc, and its address is returned via stringptr.
2385     The yield of the function is the length of the string, not including
2386 nigel 93 the terminating zero, or one of these error codes:
2387 nigel 41
2388 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2389 nigel 41
2390 ph10 416 The buffer was too small for pcre_copy_substring(), or the attempt to
2391 nigel 73 get memory failed for pcre_get_substring().
2392 nigel 41
2393 nigel 73 PCRE_ERROR_NOSUBSTRING (-7)
2394 nigel 41
2395 nigel 73 There is no substring whose number is stringnumber.
2396 nigel 41
2397 ph10 416 The pcre_get_substring_list() function extracts all available sub-
2398     strings and builds a list of pointers to them. All this is done in a
2399 nigel 75 single block of memory that is obtained via pcre_malloc. The address of
2400 ph10 416 the memory block is returned via listptr, which is also the start of
2401     the list of string pointers. The end of the list is marked by a NULL
2402     pointer. The yield of the function is zero if all went well, or the
2403 nigel 93 error code
2404 nigel 41
2405 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2406 nigel 41
2407 nigel 73 if the attempt to get the memory block failed.
2408 nigel 41
2409 ph10 416 When any of these functions encounter a substring that is unset, which
2410     can happen when capturing subpattern number n+1 matches some part of
2411     the subject, but subpattern n has not been used at all, they return an
2412 nigel 73 empty string. This can be distinguished from a genuine zero-length sub-
2413 ph10 416 string by inspecting the appropriate offset in ovector, which is nega-
2414 nigel 73 tive for unset substrings.
2415 nigel 41
2416 ph10 416 The two convenience functions pcre_free_substring() and pcre_free_sub-
2417     string_list() can be used to free the memory returned by a previous
2418 nigel 75 call of pcre_get_substring() or pcre_get_substring_list(), respec-
2419 ph10 416 tively. They do nothing more than call the function pointed to by
2420     pcre_free, which of course could be called directly from a C program.
2421     However, PCRE is used in some situations where it is linked via a spe-
2422     cial interface to another programming language that cannot use
2423     pcre_free directly; it is for these cases that the functions are pro-
2424 nigel 77 vided.
2425 nigel 41
2426 nigel 73
2427 nigel 63 EXTRACTING CAPTURED SUBSTRINGS BY NAME
2428 nigel 41
2429 nigel 75 int pcre_get_stringnumber(const pcre *code,
2430     const char *name);
2431    
2432 nigel 73 int pcre_copy_named_substring(const pcre *code,
2433     const char *subject, int *ovector,
2434     int stringcount, const char *stringname,
2435     char *buffer, int buffersize);
2436 nigel 41
2437 nigel 73 int pcre_get_named_substring(const pcre *code,
2438     const char *subject, int *ovector,
2439     int stringcount, const char *stringname,
2440     const char **stringptr);
2441 nigel 41
2442 ph10 416 To extract a substring by name, you first have to find associated num-
2443 nigel 75 ber. For example, for this pattern
2444 nigel 41
2445 nigel 93 (a+)b(?<xxx>\d+)...
2446 nigel 63
2447 nigel 91 the number of the subpattern called "xxx" is 2. If the name is known to
2448     be unique (PCRE_DUPNAMES was not set), you can find the number from the
2449     name by calling pcre_get_stringnumber(). The first argument is the com-
2450     piled pattern, and the second is the name. The yield of the function is
2451 ph10 416 the subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no
2452 nigel 91 subpattern of that name.
2453 nigel 63
2454 nigel 75 Given the number, you can extract the substring directly, or use one of
2455     the functions described in the previous section. For convenience, there
2456     are also two functions that do the whole job.
2457    
2458 ph10 416 Most of the arguments of pcre_copy_named_substring() and
2459     pcre_get_named_substring() are the same as those for the similarly
2460     named functions that extract by number. As these are described in the
2461     previous section, they are not re-described here. There are just two
2462 nigel 75 differences:
2463 nigel 63
2464 ph10 416 First, instead of a substring number, a substring name is given. Sec-
2465 nigel 73 ond, there is an extra argument, given at the start, which is a pointer
2466 ph10 416 to the compiled pattern. This is needed in order to gain access to the
2467 nigel 73 name-to-number translation table.
2468 nigel 63
2469 ph10 416 These functions call pcre_get_stringnumber(), and if it succeeds, they
2470     then call pcre_copy_substring() or pcre_get_substring(), as appropri-
2471     ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the
2472 ph10 128 behaviour may not be what you want (see the next section).
2473 nigel 63
2474 ph10 461 Warning: If the pattern uses the (?| feature to set up multiple subpat-
2475     terns with the same number, as described in the section on duplicate
2476     subpattern numbers in the pcrepattern page, you cannot use names to
2477     distinguish the different subpatterns, because names are not included
2478     in the compiled code. The matching process uses only numbers. For this
2479     reason, the use of different names for subpatterns of the same number
2480     causes an error at compile time.
2481 nigel 77
2482 ph10 392
2483 nigel 91 DUPLICATE SUBPATTERN NAMES
2484    
2485     int pcre_get_stringtable_entries(const pcre *code,
2486     const char *name, char **first, char **last);
2487    
2488 ph10 461 When a pattern is compiled with the PCRE_DUPNAMES option, names for
2489     subpatterns are not required to be unique. (Duplicate names are always
2490     allowed for subpatterns with the same number, created by using the (?|
2491     feature. Indeed, if such subpatterns are named, they are required to
2492     use the same names.)
2493 ph10 208
2494 ph10 461 Normally, patterns with duplicate names are such that in any one match,
2495     only one of the named subpatterns participates. An example is shown in
2496     the pcrepattern documentation.
2497    
2498     When duplicates are present, pcre_copy_named_substring() and
2499     pcre_get_named_substring() return the first substring corresponding to
2500     the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING
2501     (-7) is returned; no data is returned. The pcre_get_stringnumber()
2502     function returns one of the numbers that are associated with the name,
2503 ph10 208 but it is not defined which it is.
2504 nigel 91
2505 ph10 461 If you want to get full details of all captured substrings for a given
2506     name, you must use the pcre_get_stringtable_entries() function. The
2507 nigel 91 first argument is the compiled pattern, and the second is the name. The
2508 ph10 461 third and fourth are pointers to variables which are updated by the
2509 nigel 91 function. After it has run, they point to the first and last entries in
2510 ph10 461 the name-to-number table for the given name. The function itself
2511     returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if
2512     there are none. The format of the table is described above in the sec-
2513     tion entitled Information about a pattern. Given all the relevant
2514     entries for the name, you can extract each of their numbers, and hence
2515 nigel 93 the captured data, if any.
2516 nigel 91
2517    
2518 nigel 77 FINDING ALL POSSIBLE MATCHES
2519    
2520 ph10 461 The traditional matching function uses a similar algorithm to Perl,
2521 nigel 77 which stops when it finds the first match, starting at a given point in
2522 ph10 461 the subject. If you want to find all possible matches, or the longest
2523     possible match, consider using the alternative matching function (see
2524     below) instead. If you cannot use the alternative function, but still
2525     need to find all possible matches, you can kludge it up by making use
2526 nigel 77 of the callout facility, which is described in the pcrecallout documen-
2527     tation.
2528    
2529     What you have to do is to insert a callout right at the end of the pat-
2530 ph10 461 tern. When your callout function is called, extract and save the cur-
2531     rent matched substring. Then return 1, which forces pcre_exec() to
2532     backtrack and try other alternatives. Ultimately, when it runs out of
2533 nigel 77 matches, pcre_exec() will yield PCRE_ERROR_NOMATCH.
2534    
2535    
2536     MATCHING A PATTERN: THE ALTERNATIVE FUNCTION
2537    
2538     int pcre_dfa_exec(const pcre *code, const pcre_extra *extra,
2539     const char *subject, int length, int startoffset,
2540     int options, int *ovector, int ovecsize,
2541     int *workspace, int wscount);
2542    
2543 ph10 461 The function pcre_dfa_exec() is called to match a subject string
2544     against a compiled pattern, using a matching algorithm that scans the
2545     subject string just once, and does not backtrack. This has different
2546     characteristics to the normal algorithm, and is not compatible with
2547     Perl. Some of the features of PCRE patterns are not supported. Never-
2548     theless, there are times when this kind of matching can be useful. For
2549     a discussion of the two matching algorithms, and a list of features
2550     that pcre_dfa_exec() does not support, see the pcrematching documenta-
2551 ph10 453 tion.
2552 nigel 77
2553 ph10 461 The arguments for the pcre_dfa_exec() function are the same as for
2554 nigel 77 pcre_exec(), plus two extras. The ovector argument is used in a differ-
2555 ph10 461 ent way, and this is described below. The other common arguments are
2556     used in the same way as for pcre_exec(), so their description is not
2557 nigel 77 repeated here.
2558    
2559 ph10 461 The two additional arguments provide workspace for the function. The
2560     workspace vector should contain at least 20 elements. It is used for
2561 nigel 77 keeping track of multiple paths through the pattern tree. More
2562 ph10 461 workspace will be needed for patterns and subjects where there are a
2563 nigel 91 lot of potential matches.
2564 nigel 77
2565 nigel 87 Here is an example of a simple call to pcre_dfa_exec():
2566 nigel 77
2567     int rc;
2568     int ovector[10];
2569     int wspace[20];
2570 nigel 87 rc = pcre_dfa_exec(
2571 nigel 77 re, /* result of pcre_compile() */
2572     NULL, /* we didn't study the pattern */
2573     "some string", /* the subject string */
2574     11, /* the length of the subject string */
2575     0, /* start at offset 0 in the subject */
2576     0, /* default options */
2577     ovector, /* vector of integers for substring information */
2578     10, /* number of elements (NOT size in bytes) */
2579     wspace, /* working space vector */
2580     20); /* number of elements (NOT size in bytes) */
2581    
2582     Option bits for pcre_dfa_exec()
2583    
2584 ph10 461 The unused bits of the options argument for pcre_dfa_exec() must be
2585     zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW-
2586 ph10 453 LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY,
2587     PCRE_NOTEMPTY_ATSTART, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, PCRE_PAR-
2588 ph10 461 TIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last
2589     four of these are exactly the same as for pcre_exec(), so their
2590 ph10 453 description is not repeated here.
2591 nigel 77
2592 ph10 429 PCRE_PARTIAL_HARD
2593     PCRE_PARTIAL_SOFT
2594 nigel 77
2595 ph10 461 These have the same general effect as they do for pcre_exec(), but the
2596     details are slightly different. When PCRE_PARTIAL_HARD is set for
2597     pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub-
2598     ject is reached and there is still at least one matching possibility
2599 ph10 429 that requires additional characters. This happens even if some complete
2600     matches have also been found. When PCRE_PARTIAL_SOFT is set, the return
2601     code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end
2602 ph10 461 of the subject is reached, there have been no complete matches, but
2603     there is still at least one matching possibility. The portion of the
2604     string that was inspected when the longest partial match was found is
2605 ph10 453 set as the first matching string in both cases.
2606 nigel 77
2607     PCRE_DFA_SHORTEST
2608    
2609 ph10 461 Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to
2610 nigel 93 stop as soon as it has found one match. Because of the way the alterna-
2611 ph10 461 tive algorithm works, this is necessarily the shortest possible match
2612 nigel 93 at the first possible matching point in the subject string.
2613 nigel 77
2614     PCRE_DFA_RESTART
2615    
2616 ph10 429 When pcre_dfa_exec() returns a partial match, it is possible to call it
2617 ph10 461 again, with additional subject characters, and have it continue with
2618     the same match. The PCRE_DFA_RESTART option requests this action; when
2619     it is set, the workspace and wscount options must reference the same
2620     vector as before because data about the match so far is left in them
2621 ph10 429 after a partial match. There is more discussion of this facility in the
2622     pcrepartial documentation.
2623 nigel 77
2624     Successful returns from pcre_dfa_exec()
2625    
2626 ph10 461 When pcre_dfa_exec() succeeds, it may have matched more than one sub-
2627 nigel 77 string in the subject. Note, however, that all the matches from one run
2628 ph10 461 of the function start at the same point in the subject. The shorter
2629     matches are all initial substrings of the longer matches. For example,
2630 nigel 77 if the pattern
2631    
2632     <.*>
2633    
2634     is matched against the string
2635    
2636     This is <something> <something else> <something further> no more
2637    
2638     the three matched strings are
2639    
2640     <something>
2641     <something> <something else>
2642     <something> <something else> <something further>
2643    
2644 ph10 461 On success, the yield of the function is a number greater than zero,
2645     which is the number of matched substrings. The substrings themselves
2646     are returned in ovector. Each string uses two elements; the first is
2647     the offset to the start, and the second is the offset to the end. In
2648     fact, all the strings have the same start offset. (Space could have
2649     been saved by giving this only once, but it was decided to retain some
2650     compatibility with the way pcre_exec() returns data, even though the
2651 nigel 93 meaning of the strings is different.)
2652 nigel 77
2653     The strings are returned in reverse order of length; that is, the long-
2654 ph10 461 est matching string is given first. If there were too many matches to
2655     fit into ovector, the yield of the function is zero, and the vector is
2656 nigel 77 filled with the longest matches.
2657    
2658     Error returns from pcre_dfa_exec()
2659    
2660 ph10 461 The pcre_dfa_exec() function returns a negative number when it fails.
2661     Many of the errors are the same as for pcre_exec(), and these are
2662     described above. There are in addition the following errors that are
2663 nigel 77 specific to pcre_dfa_exec():
2664    
2665     PCRE_ERROR_DFA_UITEM (-16)
2666    
2667 ph10 461 This return is given if pcre_dfa_exec() encounters an item in the pat-
2668     tern that it does not support, for instance, the use of \C or a back
2669 nigel 77 reference.
2670    
2671     PCRE_ERROR_DFA_UCOND (-17)
2672    
2673 ph10 461 This return is given if pcre_dfa_exec() encounters a condition item
2674     that uses a back reference for the condition, or a test for recursion
2675 nigel 93 in a specific group. These are not supported.
2676 nigel 77
2677     PCRE_ERROR_DFA_UMLIMIT (-18)
2678    
2679 ph10 461 This return is given if pcre_dfa_exec() is called with an extra block
2680 nigel 77 that contains a setting of the match_limit field. This is not supported
2681     (it is meaningless).
2682    
2683     PCRE_ERROR_DFA_WSSIZE (-19)
2684    
2685 ph10 461 This return is given if pcre_dfa_exec() runs out of space in the
2686 nigel 77 workspace vector.
2687    
2688     PCRE_ERROR_DFA_RECURSE (-20)
2689    
2690 ph10 461 When a recursive subpattern is processed, the matching function calls
2691     itself recursively, using private vectors for ovector and workspace.
2692     This error is given if the output vector is not large enough. This
2693 nigel 77 should be extremely rare, as a vector of size 1000 is used.
2694    
2695 nigel 93
2696     SEE ALSO
2697    
2698 ph10 461 pcrebuild(3), pcrecallout(3), pcrecpp(3)(3), pcrematching(3), pcrepar-
2699 ph10 392 tial(3), pcreposix(3), pcreprecompile(3), pcresample(3), pcrestack(3).
2700 nigel 93
2701 nigel 63
2702 ph10 99 AUTHOR
2703 nigel 63
2704 ph10 99 Philip Hazel
2705     University Computing Service
2706     Cambridge CB2 3QH, England.
2707    
2708    
2709     REVISION
2710    
2711 ph10 461 Last updated: 03 October 2009
2712 ph10 392 Copyright (c) 1997-2009 University of Cambridge.
2713 ph10 99 ------------------------------------------------------------------------------
2714 ph10 461
2715    
2716 nigel 79 PCRECALLOUT(3) PCRECALLOUT(3)
2717 nigel 63
2718 nigel 79
2719 nigel 73 NAME
2720     PCRE - Perl-compatible regular expressions
2721    
2722 nigel 77
2723 nigel 63 PCRE CALLOUTS
2724    
2725 nigel 73 int (*pcre_callout)(pcre_callout_block *);
2726 nigel 63
2727 nigel 73 PCRE provides a feature called "callout", which is a means of temporar-
2728     ily passing control to the caller of PCRE in the middle of pattern
2729     matching. The caller of PCRE provides an external function by putting
2730     its entry point in the global variable pcre_callout. By default, this
2731     variable contains NULL, which disables all calling out.
2732 nigel 63
2733 nigel 73 Within a regular expression, (?C) indicates the points at which the
2734     external function is to be called. Different callout points can be
2735     identified by putting a number less than 256 after the letter C. The
2736     default value is zero. For example, this pattern has two callout
2737     points:
2738 nigel 63
2739 ph10 155 (?C1)abc(?C2)def
2740 nigel 63
2741 ph10 461 If the PCRE_AUTO_CALLOUT option bit is set when pcre_compile() or
2742     pcre_compile2() is called, PCRE automatically inserts callouts, all
2743     with number 255, before each item in the pattern. For example, if
2744     PCRE_AUTO_CALLOUT is used with the pattern
2745 nigel 63
2746 nigel 75 A(\d{2}|--)
2747    
2748     it is processed as if it were
2749    
2750     (?C255)A(?C255)((?C255)\d{2}(?C255)|(?C255)-(?C255)-(?C255))(?C255)
2751    
2752     Notice that there is a callout before and after each parenthesis and
2753     alternation bar. Automatic callouts can be used for tracking the
2754     progress of pattern matching. The pcretest command has an option that
2755     sets automatic callouts; when it is used, the output indicates how the
2756     pattern is matched. This is useful information when you are trying to
2757     optimize the performance of a particular pattern.
2758    
2759    
2760     MISSING CALLOUTS
2761    
2762     You should be aware that, because of optimizations in the way PCRE
2763 ph10 392 matches patterns by default, callouts sometimes do not happen. For
2764     example, if the pattern is
2765 nigel 75
2766     ab(?C4)cd
2767    
2768     PCRE knows that any matching string must contain the letter "d". If the
2769     subject string is "abyz", the lack of "d" means that matching doesn't
2770     ever start, and the callout is never reached. However, with "abyd",
2771     though the result is still no match, the callout is obeyed.
2772    
2773 ph10 461 If the pattern is studied, PCRE knows the minimum length of a matching
2774     string, and will immediately give a "no match" return without actually
2775     running a match if the subject is not long enough, or, for unanchored
2776     patterns, if it has been scanned far enough.
2777    
2778     You can disable these optimizations by passing the PCRE_NO_START_OPTI-
2779     MIZE option to pcre_exec() or pcre_dfa_exec(). This slows down the
2780     matching process, but does ensure that callouts such as the example
2781 ph10 392 above are obeyed.
2782 nigel 75
2783 ph10 392
2784 nigel 75 THE CALLOUT INTERFACE
2785    
2786 ph10 461 During matching, when PCRE reaches a callout point, the external func-
2787     tion defined by pcre_callout is called (if it is set). This applies to
2788     both the pcre_exec() and the pcre_dfa_exec() matching functions. The
2789     only argument to the callout function is a pointer to a pcre_callout
2790 nigel 77 block. This structure contains the following fields:
2791 nigel 75
2792 nigel 73 int version;
2793     int callout_number;
2794     int *offset_vector;
2795     const char *subject;
2796     int subject_length;
2797     int start_match;
2798     int current_position;
2799     int capture_top;
2800     int capture_last;
2801     void *callout_data;
2802 nigel 75 int pattern_position;
2803     int next_item_length;
2804 nigel 63
2805 ph10 461 The version field is an integer containing the version number of the
2806     block format. The initial version was 0; the current version is 1. The
2807     version number will change again in future if additional fields are
2808 nigel 75 added, but the intention is never to remove any of the existing fields.
2809 nigel 63
2810 ph10 461 The callout_number field contains the number of the callout, as com-
2811     piled into the pattern (that is, the number after ?C for manual call-
2812 nigel 75 outs, and 255 for automatically generated callouts).
2813 nigel 63
2814 ph10 461 The offset_vector field is a pointer to the vector of offsets that was
2815     passed by the caller to pcre_exec() or pcre_dfa_exec(). When
2816     pcre_exec() is used, the contents can be inspected in order to extract
2817     substrings that have been matched so far, in the same way as for
2818     extracting substrings after a match has completed. For pcre_dfa_exec()
2819 nigel 77 this field is not useful.
2820 nigel 63
2821 nigel 75 The subject and subject_length fields contain copies of the values that
2822 nigel 73 were passed to pcre_exec().
2823 nigel 63
2824 ph10 461 The start_match field normally contains the offset within the subject
2825     at which the current match attempt started. However, if the escape
2826     sequence \K has been encountered, this value is changed to reflect the
2827     modified starting point. If the pattern is not anchored, the callout
2828 ph10 172 function may be called several times from the same point in the pattern
2829     for different starting points in the subject.
2830 nigel 63
2831 ph10 461 The current_position field contains the offset within the subject of
2832 nigel 73 the current match pointer.
2833 nigel 63
2834 ph10 461 When the pcre_exec() function is used, the capture_top field contains
2835     one more than the number of the highest numbered captured substring so
2836     far. If no substrings have been captured, the value of capture_top is
2837     one. This is always the case when pcre_dfa_exec() is used, because it
2838 nigel 77 does not support captured substrings.
2839 nigel 63
2840 ph10 461 The capture_last field contains the number of the most recently cap-
2841     tured substring. If no substrings have been captured, its value is -1.
2842 nigel 77 This is always the case when pcre_dfa_exec() is used.
2843 nigel 63
2844 ph10 461 The callout_data field contains a value that is passed to pcre_exec()
2845     or pcre_dfa_exec() specifically so that it can be passed back in call-
2846     outs. It is passed in the pcre_callout field of the pcre_extra data
2847     structure. If no such data was passed, the value of callout_data in a
2848     pcre_callout block is NULL. There is a description of the pcre_extra
2849 nigel 73 structure in the pcreapi documentation.
2850 nigel 63
2851 ph10 461 The pattern_position field is present from version 1 of the pcre_call-
2852 nigel 75 out structure. It contains the offset to the next item to be matched in
2853     the pattern string.
2854 nigel 63
2855 ph10 461 The next_item_length field is present from version 1 of the pcre_call-
2856 nigel 75 out structure. It contains the length of the next item to be matched in
2857 ph10 461 the pattern string. When the callout immediately precedes an alterna-
2858     tion bar, a closing parenthesis, or the end of the pattern, the length
2859     is zero. When the callout precedes an opening parenthesis, the length
2860 nigel 75 is that of the entire subpattern.
2861 nigel 73
2862 ph10 461 The pattern_position and next_item_length fields are intended to help
2863     in distinguishing between different automatic callouts, which all have
2864 nigel 75 the same callout number. However, they are set for all callouts.
2865    
2866    
2867 nigel 63 RETURN VALUES
2868    
2869 ph10 461 The external callout function returns an integer to PCRE. If the value
2870     is zero, matching proceeds as normal. If the value is greater than
2871     zero, matching fails at the current point, but the testing of other
2872 nigel 77 matching possibilities goes ahead, just as if a lookahead assertion had
2873 ph10 461 failed. If the value is less than zero, the match is abandoned, and
2874     pcre_exec() or pcre_dfa_exec() returns the negative value.
2875 nigel 63
2876 ph10 461 Negative values should normally be chosen from the set of
2877 nigel 73 PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan-
2878 ph10 461 dard "no match" failure. The error number PCRE_ERROR_CALLOUT is
2879     reserved for use by callout functions; it will never be used by PCRE
2880 nigel 73 itself.
2881 nigel 63
2882    
2883 ph10 99 AUTHOR
2884 nigel 63
2885 ph10 99 Philip Hazel
2886     University Computing Service
2887     Cambridge CB2 3QH, England.
2888    
2889    
2890     REVISION
2891    
2892 ph10 461 Last updated: 29 September 2009
2893 ph10 392 Copyright (c) 1997-2009 University of Cambridge.
2894 ph10 99 ------------------------------------------------------------------------------
2895 ph10 461
2896    
2897 nigel 79 PCRECOMPAT(3) PCRECOMPAT(3)
2898 nigel 63
2899 nigel 79
2900 nigel 73 NAME
2901     PCRE - Perl-compatible regular expressions
2902    
2903 nigel 77
2904 nigel 75 DIFFERENCES BETWEEN PCRE AND PERL
2905 nigel 41
2906 nigel 73 This document describes the differences in the ways that PCRE and Perl
2907 ph10 461 handle regular expressions. The differences described here are with
2908     respect to Perl 5.10.
2909 nigel 41
2910 ph10 461 1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details
2911     of what it does have are given in the section on UTF-8 support in the
2912 nigel 87 main pcre page.
2913 nigel 41
2914 nigel 73 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl
2915 ph10 461 permits them, but they do not mean what you might think. For example,
2916 nigel 73 (?!a){3} does not assert that the next three characters are not "a". It
2917     just asserts that the next character is not "a" three times.
2918 nigel 41
2919 ph10 461 3. Capturing subpatterns that occur inside negative lookahead asser-
2920     tions are counted, but their entries in the offsets vector are never
2921     set. Perl sets its numerical variables from any such patterns that are
2922 nigel 73 matched before the assertion fails to match something (thereby succeed-
2923 ph10 461 ing), but only if the negative lookahead assertion contains just one
2924 nigel 73 branch.
2925 nigel 41
2926 ph10 461 4. Though binary zero characters are supported in the subject string,
2927 nigel 73 they are not allowed in a pattern string because it is passed as a nor-
2928 nigel 75 mal C string, terminated by zero. The escape sequence \0 can be used in
2929     the pattern to represent a binary zero.
2930 nigel 41
2931 ph10 461 5. The following Perl escape sequences are not supported: \l, \u, \L,
2932 nigel 75 \U, and \N. In fact these are implemented by Perl's general string-han-
2933 ph10 461 dling and are not part of its pattern matching engine. If any of these
2934 nigel 75 are encountered by PCRE, an error is generated.
2935 nigel 41
2936 ph10 461 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE
2937     is built with Unicode character property support. The properties that
2938     can be tested with \p and \P are limited to the general category prop-
2939     erties such as Lu and Nd, script names such as Greek or Han, and the
2940     derived properties Any and L&. PCRE does support the Cs (surrogate)
2941     property, which Perl does not; the Perl documentation says "Because
2942 ph10 453 Perl hides the need for the user to understand the internal representa-
2943 ph10 461 tion of Unicode characters, there is no need to implement the somewhat
2944 ph10 453 messy concept of surrogates."
2945 nigel 75
2946     7. PCRE does support the \Q...\E escape for quoting substrings. Charac-
2947 ph10 461 ters in between are treated as literals. This is slightly different
2948     from Perl in that $ and @ are also handled as literals inside the
2949     quotes. In Perl, they cause variable interpolation (but of course PCRE
2950 nigel 73 does not have variables). Note the following examples:
2951 nigel 49
2952 nigel 73 Pattern PCRE matches Perl matches
2953 nigel 41
2954 nigel 73 \Qabc$xyz\E abc$xyz abc followed by the
2955     contents of $xyz
2956     \Qabc\$xyz\E abc\$xyz abc\$xyz
2957     \Qabc\E\$\Qxyz\E abc$xyz abc$xyz
2958 nigel 41
2959 ph10 461 The \Q...\E sequence is recognized both inside and outside character
2960 nigel 73 classes.
2961 nigel 41
2962 nigel 93 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code})
2963 ph10 461 constructions. However, there is support for recursive patterns. This
2964     is not available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE
2965     "callout" feature allows an external function to be called during pat-
2966 nigel 75 tern matching. See the pcrecallout documentation for details.
2967 nigel 63
2968 ph10 461 9. Subpatterns that are called recursively or as "subroutines" are
2969     always treated as atomic groups in PCRE. This is like Python, but
2970     unlike Perl. There is a discussion of an example that explains this in
2971     more detail in the section on recursion differences from Perl in the
2972     pcrepattern page.
2973 nigel 93
2974 ph10 461 10. There are some differences that are concerned with the settings of
2975     captured strings when part of a pattern is repeated. For example,
2976     matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2
2977 nigel 73 unset, but in PCRE it is set to "b".
2978 nigel 41
2979 ph10 211 11. PCRE does support Perl 5.10's backtracking verbs (*ACCEPT),
2980 ph10 461 (*FAIL), (*F), (*COMMIT), (*PRUNE), (*SKIP), and (*THEN), but only in
2981 ph10 453 the forms without an argument. PCRE does not support (*MARK).
2982 ph10 211
2983 ph10 461 12. PCRE's handling of duplicate subpattern numbers and duplicate sub-
2984     pattern names is not as general as Perl's. This is a consequence of the
2985     fact the PCRE works internally just with numbers, using an external ta-
2986     ble to translate between numbers and names. In particular, a pattern
2987     such as (?|(?<a>A)|(?<b)B), where the two capturing parentheses have
2988     the same number but different names, is not supported, and causes an
2989     error at compile time. If it were allowed, it would not be possible to
2990     distinguish which parentheses matched, because both names map to cap-
2991     turing subpattern number 1. To avoid this confusing situation, an error
2992     is given at compile time.
2993 nigel 41
2994 ph10 461 13. PCRE provides some extensions to the Perl regular expression facil-
2995     ities. Perl 5.10 includes new features that are not in earlier ver-
2996     sions of Perl, some of which (such as named parentheses) have been in
2997     PCRE for some time. This list is with respect to Perl 5.10:
2998 nigel 41
2999 ph10 461 (a) Although lookbehind assertions in PCRE must match fixed length
3000     strings, each alternative branch of a lookbehind assertion can match a
3001     different length of string. Perl requires them all to have the same
3002     length.
3003    
3004     (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $
3005 nigel 73 meta-character matches only at the very end of the string.
3006 nigel 41
3007 nigel 73 (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe-
3008 ph10 182 cial meaning is faulted. Otherwise, like Perl, the backslash is quietly
3009     ignored. (Perl can be made to issue a warning.)
3010 nigel 41
3011 ph10 461 (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti-
3012 nigel 73 fiers is inverted, that is, by default they are not greedy, but if fol-
3013     lowed by a question mark they are.
3014 nigel 41
3015 nigel 75 (e) PCRE_ANCHORED can be used at matching time to force a pattern to be
3016     tried only at the first matching position in the subject string.
3017 nigel 41
3018 ph10 453 (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
3019 ph10 461 and PCRE_NO_AUTO_CAPTURE options for pcre_exec() have no Perl equiva-
3020 ph10 453 lents.
3021 nigel 41
3022 ph10 461 (g) The \R escape sequence can be restricted to match only CR, LF, or
3023 ph10 231 CRLF by the PCRE_BSR_ANYCRLF option.
3024 nigel 41
3025 ph10 231 (h) The callout facility is PCRE-specific.
3026 nigel 43
3027 ph10 231 (i) The partial matching facility is PCRE-specific.
3028    
3029     (j) Patterns compiled by PCRE can be saved and re-used at a later time,
3030 nigel 75 even on different hosts that have the other endianness.
3031    
3032 ph10 461 (k) The alternative matching function (pcre_dfa_exec()) matches in a
3033 nigel 77 different way and is not Perl-compatible.
3034    
3035 ph10 461 (l) PCRE recognizes some special sequences such as (*CR) at the start
3036 ph10 231 of a pattern that set overall options that cannot be changed within the
3037     pattern.
3038 nigel 63
3039 ph10 231
3040 ph10 99 AUTHOR
3041 nigel 63
3042 ph10 99 Philip Hazel
3043     University Computing Service
3044     Cambridge CB2 3QH, England.
3045    
3046    
3047     REVISION
3048    
3049 ph10 461 Last updated: 04 October 2009
3050 ph10 429 Copyright (c) 1997-2009 University of Cambridge.
3051 ph10 99 ------------------------------------------------------------------------------
3052 ph10 461
3053    
3054 nigel 79 PCREPATTERN(3) PCREPATTERN(3)
3055 nigel 63
3056 nigel 79
3057 nigel 73 NAME
3058     PCRE - Perl-compatible regular expressions
3059    
3060 nigel 77
3061 nigel 63 PCRE REGULAR EXPRESSION DETAILS
3062    
3063 ph10 208 The syntax and semantics of the regular expressions that are supported
3064     by PCRE are described in detail below. There is a quick-reference syn-
3065 ph10 345 tax summary in the pcresyntax page. PCRE tries to match Perl syntax and
3066     semantics as closely as it can. PCRE also supports some alternative
3067     regular expression syntax (which does not conflict with the Perl syn-
3068     tax) in order to provide some compatibility with regular expressions in
3069     Python, .NET, and Oniguruma.
3070 nigel 49
3071 ph10 345 Perl's regular expressions are described in its own documentation, and
3072     regular expressions in general are covered in a number of books, some
3073     of which have copious examples. Jeffrey Friedl's "Mastering Regular
3074     Expressions", published by O'Reilly, covers regular expressions in
3075     great detail. This description of PCRE's regular expressions is
3076     intended as reference material.
3077    
3078 nigel 75 The original operation of PCRE was on strings of one-byte characters.
3079     However, there is now also support for UTF-8 character strings. To use
3080 ph10 461 this, PCRE must be built to include UTF-8 support, and you must call
3081     pcre_compile() or pcre_compile2() with the PCRE_UTF8 option. There is
3082     also a special sequence that can be given at the start of a pattern:
3083 nigel 41
3084 ph10 416 (*UTF8)
3085    
3086     Starting a pattern with this sequence is equivalent to setting the
3087     PCRE_UTF8 option. This feature is not Perl-compatible. How setting
3088     UTF-8 mode affects pattern matching is mentioned in several places
3089     below. There is also a summary of UTF-8 features in the section on
3090     UTF-8 support in the main pcre page.
3091    
3092 nigel 77 The remainder of this document discusses the patterns that are sup-
3093     ported by PCRE when its main matching function, pcre_exec(), is used.
3094     From release 6.0, PCRE offers a second matching function,
3095     pcre_dfa_exec(), which matches using a different algorithm that is not
3096 ph10 172 Perl-compatible. Some of the features discussed below are not available
3097     when pcre_dfa_exec() is used. The advantages and disadvantages of the
3098     alternative function, and how it differs from the normal function, are
3099     discussed in the pcrematching page.
3100 nigel 77
3101 nigel 93
3102 ph10 227 NEWLINE CONVENTIONS
3103    
3104     PCRE supports five different conventions for indicating line breaks in
3105     strings: a single CR (carriage return) character, a single LF (line-
3106     feed) character, the two-character sequence CRLF, any of the three pre-
3107     ceding, or any Unicode newline sequence. The pcreapi page has further
3108     discussion about newlines, and shows how to set the newline convention
3109     in the options arguments for the compiling and matching functions.
3110    
3111     It is also possible to specify a newline convention by starting a pat-
3112     tern string with one of the following five sequences:
3113    
3114     (*CR) carriage return
3115     (*LF) linefeed
3116     (*CRLF) carriage return, followed by linefeed
3117     (*ANYCRLF) any of the three above
3118     (*ANY) all Unicode newline sequences
3119    
3120 ph10 461 These override the default and the options given to pcre_compile() or
3121     pcre_compile2(). For example, on a Unix system where LF is the default
3122     newline sequence, the pattern
3123 ph10 227
3124     (*CR)a.b
3125    
3126     changes the convention to CR. That pattern matches "a\nb" because LF is
3127     no longer a newline. Note that these special settings, which are not
3128     Perl-compatible, are recognized only at the very start of a pattern,
3129 ph10 231 and that they must be in upper case. If more than one of them is
3130     present, the last one is used.
3131 ph10 227
3132 ph10 231 The newline convention does not affect what the \R escape sequence
3133     matches. By default, this is any Unicode newline sequence, for Perl
3134     compatibility. However, this can be changed; see the description of \R
3135 ph10 247 in the section entitled "Newline sequences" below. A change of \R set-
3136     ting can be combined with a change of newline convention.
3137 ph10 227
3138 ph10 231
3139 nigel 93 CHARACTERS AND METACHARACTERS
3140    
3141 ph10 247 A regular expression is a pattern that is matched against a subject
3142     string from left to right. Most characters stand for themselves in a
3143     pattern, and match the corresponding characters in the subject. As a
3144 nigel 73 trivial example, the pattern
3145 nigel 41
3146 nigel 73 The quick brown fox
3147 nigel 41
3148 nigel 77 matches a portion of a subject string that is identical to itself. When
3149 ph10 247 caseless matching is specified (the PCRE_CASELESS option), letters are
3150     matched independently of case. In UTF-8 mode, PCRE always understands
3151     the concept of case for characters whose values are less than 128, so
3152     caseless matching is always possible. For characters with higher val-
3153     ues, the concept of case is supported if PCRE is compiled with Unicode
3154     property support, but not otherwise. If you want to use caseless
3155     matching for characters 128 and above, you must ensure that PCRE is
3156 nigel 77 compiled with Unicode property support as well as with UTF-8 support.
3157 nigel 41
3158 ph10 247 The power of regular expressions comes from the ability to include
3159     alternatives and repetitions in the pattern. These are encoded in the
3160 nigel 77 pattern by the use of metacharacters, which do not stand for themselves
3161     but instead are interpreted in some special way.
3162    
3163 ph10 247 There are two different sets of metacharacters: those that are recog-
3164     nized anywhere in the pattern except within square brackets, and those
3165     that are recognized within square brackets. Outside square brackets,
3166 nigel 93 the metacharacters are as follows:
3167 nigel 41
3168 nigel 73 \ general escape character with several uses
3169     ^ assert start of string (or line, in multiline mode)
3170     $ assert end of string (or line, in multiline mode)
3171     . match any character except newline (by default)
3172     [ start character class definition
3173     | start of alternative branch
3174     ( start subpattern
3175     ) end subpattern
3176     ? extends the meaning of (
3177     also 0 or 1 quantifier
3178     also quantifier minimizer
3179     * 0 or more quantifier
3180     + 1 or more quantifier
3181     also "possessive quantifier"
3182     { start min/max quantifier
3183 nigel 41
3184 ph10 247 Part of a pattern that is in square brackets is called a "character
3185 nigel 75 class". In a character class the only metacharacters are:
3186 nigel 41
3187 nigel 73 \ general escape character
3188     ^ negate the class, but only if the first character
3189     - indicates character range
3190     [ POSIX character class (only if followed by POSIX
3191     syntax)
3192     ] terminates the character class
3193 nigel 41
3194 ph10 392 The following sections describe the use of each of the metacharacters.
3195 nigel 41
3196    
3197 nigel 63 BACKSLASH
3198 nigel 41
3199 nigel 73 The backslash character has several uses. Firstly, if it is followed by
3200 ph10 392 a non-alphanumeric character, it takes away any special meaning that
3201     character may have. This use of backslash as an escape character
3202 nigel 73 applies both inside and outside character classes.
3203 nigel 41
3204 ph10 392 For example, if you want to match a * character, you write \* in the
3205     pattern. This escaping action applies whether or not the following
3206     character would otherwise be interpreted as a metacharacter, so it is
3207     always safe to precede a non-alphanumeric with backslash to specify
3208     that it stands for itself. In particular, if you want to match a back-
3209 nigel 75 slash, you write \\.
3210 nigel 41
3211 ph10 392 If a pattern is compiled with the PCRE_EXTENDED option, whitespace in
3212     the pattern (other than in a character class) and characters between a
3213 nigel 91 # outside a character class and the next newline are ignored. An escap-
3214 ph10 392 ing backslash can be used to include a whitespace or # character as
3215 nigel 91 part of the pattern.
3216 nigel 41
3217 ph10 392 If you want to remove the special meaning from a sequence of charac-
3218     ters, you can do so by putting them between \Q and \E. This is differ-
3219     ent from Perl in that $ and @ are handled as literals in \Q...\E
3220     sequences in PCRE, whereas in Perl, $ and @ cause variable interpola-
3221 nigel 73 tion. Note the following examples:
3222 nigel 63
3223 nigel 73 Pattern PCRE matches Perl matches
3224 nigel 63
3225 nigel 73 \Qabc$xyz\E abc$xyz abc followed by the
3226     contents of $xyz
3227     \Qabc\$xyz\E abc\$xyz abc\$xyz
3228     \Qabc\E\$\Qxyz\E abc$xyz abc$xyz
3229 nigel 63
3230 ph10 392 The \Q...\E sequence is recognized both inside and outside character
3231 nigel 73 classes.
3232 nigel 63
3233 nigel 75 Non-printing characters
3234    
3235 nigel 73 A second use of backslash provides a way of encoding non-printing char-
3236 ph10 392 acters in patterns in a visible manner. There is no restriction on the
3237     appearance of non-printing characters, apart from the binary zero that
3238     terminates a pattern, but when a pattern is being prepared by text
3239 ph10 461 editing, it is often easier to use one of the following escape
3240 nigel 73 sequences than the binary character it represents:
3241 nigel 63
3242 nigel 73 \a alarm, that is, the BEL character (hex 07)
3243     \cx "control-x", where x is any character
3244     \e escape (hex 1B)
3245     \f formfeed (hex 0C)
3246 ph10 227 \n linefeed (hex 0A)
3247 nigel 73 \r carriage return (hex 0D)
3248     \t tab (hex 09)
3249     \ddd character with octal code ddd, or backreference
3250     \xhh character with hex code hh
3251 nigel 87 \x{hhh..} character with hex code hhh..
3252 nigel 41
3253 ph10 392 The precise effect of \cx is as follows: if x is a lower case letter,
3254     it is converted to upper case. Then bit 6 of the character (hex 40) is
3255     inverted. Thus \cz becomes hex 1A, but \c{ becomes hex 3B, while \c;
3256 nigel 73 becomes hex 7B.
3257 nigel 41
3258 ph10 392 After \x, from zero to two hexadecimal digits are read (letters can be
3259     in upper or lower case). Any number of hexadecimal digits may appear
3260     between \x{ and }, but the value of the character code must be less
3261 ph10 211 than 256 in non-UTF-8 mode, and less than 2**31 in UTF-8 mode. That is,
3262 ph10 392 the maximum value in hexadecimal is 7FFFFFFF. Note that this is bigger
3263 ph10 211 than the largest Unicode code point, which is 10FFFF.
3264 nigel 41
3265 ph10 392 If characters other than hexadecimal digits appear between \x{ and },
3266 ph10 211 or if there is no terminating }, this form of escape is not recognized.
3267 ph10 392 Instead, the initial \x will be interpreted as a basic hexadecimal
3268     escape, with no following digits, giving a character whose value is
3269 ph10 211 zero.
3270    
3271 nigel 73 Characters whose value is less than 256 can be defined by either of the
3272 ph10 392 two syntaxes for \x. There is no difference in the way they are han-
3273 nigel 87 dled. For example, \xdc is exactly the same as \x{dc}.
3274 nigel 41
3275 ph10 392 After \0 up to two further octal digits are read. If there are fewer
3276     than two digits, just those that are present are used. Thus the
3277 nigel 91 sequence \0\x\07 specifies two binary zeros followed by a BEL character
3278 ph10 392 (code value 7). Make sure you supply two digits after the initial zero
3279 nigel 91 if the pattern character that follows is itself an octal digit.
3280 nigel 63
3281 nigel 73 The handling of a backslash followed by a digit other than 0 is compli-
3282     cated. Outside a character class, PCRE reads it and any following dig-
3283 ph10 392 its as a decimal number. If the number is less than 10, or if there
3284 nigel 73 have been at least that many previous capturing left parentheses in the
3285 ph10 392 expression, the entire sequence is taken as a back reference. A
3286     description of how this works is given later, following the discussion
3287 nigel 73 of parenthesized subpatterns.
3288 nigel 41
3289 ph10 392 Inside a character class, or if the decimal number is greater than 9
3290     and there have not been that many capturing subpatterns, PCRE re-reads
3291 nigel 93 up to three octal digits following the backslash, and uses them to gen-
3292 ph10 392 erate a data character. Any subsequent digits stand for themselves. In
3293     non-UTF-8 mode, the value of a character specified in octal must be
3294     less than \400. In UTF-8 mode, values up to \777 are permitted. For
3295 nigel 91 example:
3296 nigel 41
3297 nigel 73 \040 is another way of writing a space
3298     \40 is the same, provided there are fewer than 40
3299     previous capturing subpatterns
3300     \7 is always a back reference
3301     \11 might be a back reference, or another way of
3302     writing a tab
3303     \011 is always a tab
3304     \0113 is a tab followed by the character "3"
3305     \113 might be a back reference, otherwise the
3306     character with octal code 113
3307     \377 might be a back reference, otherwise
3308     the byte consisting entirely of 1 bits
3309     \81 is either a back reference, or a binary zero
3310     followed by the two characters "8" and "1"
3311 nigel 41
3312 ph10 392 Note that octal values of 100 or greater must not be introduced by a
3313 nigel 73 leading zero, because no more than three octal digits are ever read.
3314 nigel 41
3315 nigel 91 All the sequences that define a single character value can be used both
3316 ph10 392 inside and outside character classes. In addition, inside a character
3317     class, the sequence \b is interpreted as the backspace character (hex
3318     08), and the sequences \R and \X are interpreted as the characters "R"
3319     and "X", respectively. Outside a character class, these sequences have
3320 nigel 93 different meanings (see below).
3321 nigel 43
3322 nigel 93 Absolute and relative back references
3323    
3324 ph10 392 The sequence \g followed by an unsigned or a negative number, option-
3325     ally enclosed in braces, is an absolute or relative back reference. A
3326 ph10 208 named back reference can be coded as \g{name}. Back references are dis-
3327     cussed later, following the discussion of parenthesized subpatterns.
3328 nigel 93
3329 ph10 345 Absolute and relative subroutine calls
3330    
3331 ph10 392 For compatibility with Oniguruma, the non-Perl syntax \g followed by a
3332 ph10 345 name or a number enclosed either in angle brackets or single quotes, is
3333 ph10 392 an alternative syntax for referencing a subpattern as a "subroutine".
3334     Details are discussed later. Note that \g{...} (Perl syntax) and
3335     \g<...> (Oniguruma syntax) are not synonymous. The former is a back
3336 ph10 345 reference; the latter is a subroutine call.
3337    
3338 nigel 75 Generic character types
3339 nigel 41
3340 nigel 93 Another use of backslash is for specifying generic character types. The
3341     following are always recognized:
3342 nigel 75
3343 nigel 73 \d any decimal digit
3344