/[pcre]/code/trunk/doc/pcre.txt
ViewVC logotype

Contents of /code/trunk/doc/pcre.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 589 - (hide annotations) (download)
Sat Jan 15 11:31:39 2011 UTC (3 years, 8 months ago) by ph10
File MIME type: text/plain
File size: 368093 byte(s)
Tidies for Release 8.12

1 nigel 75 -----------------------------------------------------------------------------
2 nigel 63 This file contains a concatenation of the PCRE man pages, converted to plain
3     text format for ease of searching with a text editor, or for use on systems
4     that do not have a man page processor. The small individual files that give
5 ph10 589 synopses of each function in the library have not been included. Neither has
6 ph10 429 the pcredemo program. There are separate text files for the pcregrep and
7     pcretest commands.
8 nigel 63 -----------------------------------------------------------------------------
9    
10 nigel 41
11 nigel 79 PCRE(3) PCRE(3)
12 nigel 41
13 nigel 79
14 nigel 73 NAME
15     PCRE - Perl-compatible regular expressions
16    
17 nigel 77
18 nigel 75 INTRODUCTION
19 nigel 41
20 nigel 73 The PCRE library is a set of functions that implement regular expres-
21     sion pattern matching using the same syntax and semantics as Perl, with
22 ph10 461 just a few differences. Some features that appeared in Python and PCRE
23     before they appeared in Perl are also available using the Python syn-
24     tax, there is some support for one or two .NET and Oniguruma syntax
25     items, and there is an option for requesting some minor changes that
26     give better JavaScript compatibility.
27 nigel 63
28 ph10 461 The current implementation of PCRE corresponds approximately with Perl
29 ph10 572 5.12, including support for UTF-8 encoded strings and Unicode general
30     category properties. However, UTF-8 and Unicode support has to be
31 ph10 461 explicitly enabled; it is not the default. The Unicode tables corre-
32 ph10 507 spond to Unicode release 5.2.0.
33 nigel 77
34 nigel 93 In addition to the Perl-compatible matching function, PCRE contains an
35 ph10 461 alternative function that matches the same compiled patterns in a dif-
36     ferent way. In certain circumstances, the alternative function has some
37     advantages. For a discussion of the two matching algorithms, see the
38     pcrematching page.
39 nigel 93
40     PCRE is written in C and released as a C library. A number of people
41     have written wrappers and interfaces of various kinds. In particular,
42     Google Inc. have provided a comprehensive C++ wrapper. This is now
43 nigel 77 included as part of the PCRE distribution. The pcrecpp page has details
44 nigel 93 of this interface. Other people's contributions can be found in the
45 nigel 77 Contrib directory at the primary FTP site, which is:
46 nigel 63
47 nigel 73 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre
48 nigel 63
49 nigel 93 Details of exactly which Perl regular expression features are and are
50 nigel 73 not supported by PCRE are given in separate documents. See the pcrepat-
51 ph10 208 tern and pcrecompat pages. There is a syntax summary in the pcresyntax
52     page.
53 nigel 63
54 ph10 208 Some features of PCRE can be included, excluded, or changed when the
55     library is built. The pcre_config() function makes it possible for a
56     client to discover which features are available. The features them-
57     selves are described in the pcrebuild page. Documentation about build-
58 ph10 461 ing PCRE for various operating systems can be found in the README and
59     NON-UNIX-USE files in the source distribution.
60 nigel 63
61 ph10 208 The library contains a number of undocumented internal functions and
62     data tables that are used by more than one of the exported external
63     functions, but which are not intended for use by external callers.
64     Their names all begin with "_pcre_", which hopefully will not provoke
65 nigel 83 any name clashes. In some environments, it is possible to control which
66 ph10 208 external symbols are exported when a shared library is built, and in
67 nigel 83 these cases the undocumented symbols are not exported.
68 nigel 63
69 nigel 77
70 nigel 63 USER DOCUMENTATION
71    
72 ph10 208 The user documentation for PCRE comprises a number of different sec-
73     tions. In the "man" format, each of these is a separate "man page". In
74     the HTML format, each is a separate page, linked from the index page.
75 ph10 429 In the plain text format, all the sections, except the pcredemo sec-
76     tion, are concatenated, for ease of searching. The sections are as fol-
77     lows:
78 nigel 63
79 nigel 73 pcre this document
80 ph10 153 pcre-config show PCRE installation configuration information
81 nigel 77 pcreapi details of PCRE's native C API
82 nigel 73 pcrebuild options for building PCRE
83     pcrecallout details of the callout feature
84     pcrecompat discussion of Perl compatibility
85 nigel 77 pcrecpp details of the C++ wrapper
86 ph10 429 pcredemo a demonstration C program that uses PCRE
87 nigel 73 pcregrep description of the pcregrep command
88 nigel 77 pcrematching discussion of the two matching algorithms
89 nigel 75 pcrepartial details of the partial matching facility
90 nigel 73 pcrepattern syntax and semantics of supported
91     regular expressions
92     pcreperform discussion of performance issues
93 nigel 77 pcreposix the POSIX-compatible C API
94 nigel 75 pcreprecompile details of saving and re-using precompiled patterns
95 ph10 429 pcresample discussion of the pcredemo program
96 nigel 91 pcrestack discussion of stack usage
97 ph10 461 pcresyntax quick syntax reference
98 nigel 75 pcretest description of the pcretest testing command
99 nigel 63
100 ph10 429 In addition, in the "man" and HTML formats, there is a short page for
101 nigel 77 each C library function, listing its arguments and results.
102 nigel 63
103    
104     LIMITATIONS
105    
106 ph10 429 There are some size limitations in PCRE but it is hoped that they will
107 nigel 73 never in practice be relevant.
108 nigel 63
109 ph10 429 The maximum length of a compiled pattern is 65539 (sic) bytes if PCRE
110 nigel 73 is compiled with the default internal linkage size of 2. If you want to
111 ph10 429 process regular expressions that are truly enormous, you can compile
112     PCRE with an internal linkage size of 3 or 4 (see the README file in
113     the source distribution and the pcrebuild documentation for details).
114     In these cases the limit is substantially larger. However, the speed
115 nigel 93 of execution is slower.
116 nigel 63
117 ph10 208 All values in repeating quantifiers must be less than 65536.
118 nigel 63
119 nigel 93 There is no limit to the number of parenthesized subpatterns, but there
120     can be no more than 65535 capturing subpatterns.
121 nigel 63
122 nigel 93 The maximum length of name for a named subpattern is 32 characters, and
123     the maximum number of named subpatterns is 10000.
124 nigel 91
125 ph10 429 The maximum length of a subject string is the largest positive number
126     that an integer variable can hold. However, when using the traditional
127 nigel 77 matching function, PCRE uses recursion to handle subpatterns and indef-
128 ph10 429 inite repetition. This means that the available stack space may limit
129 nigel 77 the size of a subject string that can be processed by certain patterns.
130 nigel 91 For a discussion of stack issues, see the pcrestack documentation.
131 nigel 63
132    
133 nigel 75 UTF-8 AND UNICODE PROPERTY SUPPORT
134 nigel 63
135 ph10 429 From release 3.3, PCRE has had some support for character strings
136     encoded in the UTF-8 format. For release 4.0 this was greatly extended
137     to cover most common requirements, and in release 5.0 additional sup-
138 nigel 75 port for Unicode general category properties was added.
139 nigel 63
140 ph10 429 In order process UTF-8 strings, you must build PCRE to include UTF-8
141     support in the code, and, in addition, you must call pcre_compile()
142     with the PCRE_UTF8 option flag, or the pattern must start with the
143     sequence (*UTF8). When either of these is the case, both the pattern
144     and any subject strings that are matched against it are treated as
145 ph10 461 UTF-8 strings instead of strings of 1-byte characters.
146 nigel 63
147 ph10 429 If you compile PCRE with UTF-8 support, but do not use it at run time,
148     the library will be a bit bigger, but the additional run time overhead
149 nigel 93 is limited to testing the PCRE_UTF8 flag occasionally, so should not be
150     very big.
151 nigel 63
152 nigel 75 If PCRE is built with Unicode character property support (which implies
153 ph10 429 UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup-
154 nigel 75 ported. The available properties that can be tested are limited to the
155 ph10 429 general category properties such as Lu for an upper case letter or Nd
156     for a decimal number, the Unicode script names such as Arabic or Han,
157     and the derived properties Any and L&. A full list is given in the
158 nigel 87 pcrepattern documentation. Only the short names for properties are sup-
159 ph10 429 ported. For example, \p{L} matches a letter. Its Perl synonym, \p{Let-
160     ter}, is not supported. Furthermore, in Perl, many properties may
161     optionally be prefixed by "Is", for compatibility with Perl 5.6. PCRE
162 nigel 87 does not support this.
163 nigel 75
164 ph10 211 Validity of UTF-8 strings
165 nigel 63
166 ph10 429 When you set the PCRE_UTF8 flag, the strings passed as patterns and
167 ph10 211 subjects are (by default) checked for validity on entry to the relevant
168 ph10 429 functions. From release 7.3 of PCRE, the check is according the rules
169     of RFC 3629, which are themselves derived from the Unicode specifica-
170     tion. Earlier releases of PCRE followed the rules of RFC 2279, which
171     allows the full range of 31-bit values (0 to 0x7FFFFFFF). The current
172 ph10 211 check allows only values in the range U+0 to U+10FFFF, excluding U+D800
173     to U+DFFF.
174 nigel 63
175 ph10 429 The excluded code points are the "Low Surrogate Area" of Unicode, of
176     which the Unicode Standard says this: "The Low Surrogate Area does not
177     contain any character assignments, consequently no character code
178 ph10 211 charts or namelists are provided for this area. Surrogates are reserved
179 ph10 429 for use with UTF-16 and then must be used in pairs." The code points
180     that are encoded by UTF-16 pairs are available as independent code
181     points in the UTF-8 encoding. (In other words, the whole surrogate
182 ph10 211 thing is a fudge for UTF-16 which unfortunately messes up UTF-8.)
183    
184 ph10 429 If an invalid UTF-8 string is passed to PCRE, an error return
185 ph10 211 (PCRE_ERROR_BADUTF8) is given. In some situations, you may already know
186     that your strings are valid, and therefore want to skip these checks in
187     order to improve performance. If you set the PCRE_NO_UTF8_CHECK flag at
188 ph10 429 compile time or at run time, PCRE assumes that the pattern or subject
189     it is given (respectively) contains only valid UTF-8 codes. In this
190 ph10 211 case, it does not diagnose an invalid UTF-8 string.
191    
192 ph10 429 If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set,
193     what happens depends on why the string is invalid. If the string con-
194 ph10 211 forms to the "old" definition of UTF-8 (RFC 2279), it is processed as a
195 ph10 429 string of characters in the range 0 to 0x7FFFFFFF. In other words,
196 ph10 211 apart from the initial validity test, PCRE (when in UTF-8 mode) handles
197 ph10 429 strings according to the more liberal rules of RFC 2279. However, if
198     the string does not even conform to RFC 2279, the result is undefined.
199 ph10 211 Your program may crash.
200    
201 ph10 429 If you want to process strings of values in the full range 0 to
202     0x7FFFFFFF, encoded in a UTF-8-like manner as per the old RFC, you can
203 ph10 211 set PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in
204     this situation, you will have to apply your own validity check.
205    
206     General comments about UTF-8 mode
207    
208 ph10 429 1. An unbraced hexadecimal escape sequence (such as \xb3) matches a
209 nigel 87 two-byte UTF-8 character if the value is greater than 127.
210 nigel 63
211 ph10 429 2. Octal numbers up to \777 are recognized, and match two-byte UTF-8
212 nigel 91 characters for values greater than \177.
213    
214 ph10 429 3. Repeat quantifiers apply to complete UTF-8 characters, not to indi-
215 nigel 73 vidual bytes, for example: \x{100}{3}.
216 nigel 63
217 ph10 429 4. The dot metacharacter matches one UTF-8 character instead of a sin-
218 nigel 75 gle byte.
219 nigel 63
220 ph10 429 5. The escape sequence \C can be used to match a single byte in UTF-8
221     mode, but its use can lead to some strange effects. This facility is
222 nigel 77 not available in the alternative matching function, pcre_dfa_exec().
223 nigel 63
224 ph10 429 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly
225 ph10 518 test characters of any code value, but, by default, the characters that
226     PCRE recognizes as digits, spaces, or word characters remain the same
227     set as before, all with values less than 256. This remains true even
228     when PCRE is built to include Unicode property support, because to do
229 ph10 567 otherwise would slow down PCRE in many common cases. Note in particular
230     that this applies to \b and \B, because they are defined in terms of \w
231     and \W. If you really want to test for a wider sense of, say, "digit",
232     you can use explicit Unicode property tests such as \p{Nd}. Alterna-
233     tively, if you set the PCRE_UCP option, the way that the character
234     escapes work is changed so that Unicode properties are used to deter-
235     mine which characters match. There are more details in the section on
236     generic character types in the pcrepattern documentation.
237 nigel 63
238 ph10 429 7. Similarly, characters that match the POSIX named character classes
239 ph10 518 are all low-valued characters, unless the PCRE_UCP option is set.
240 nigel 63
241 ph10 572 8. However, the horizontal and vertical whitespace matching escapes
242     (\h, \H, \v, and \V) do match all the appropriate Unicode characters,
243     whether or not PCRE_UCP is set.
244 ph10 182
245 ph10 429 9. Case-insensitive matching applies only to characters whose values
246     are less than 128, unless PCRE is built with Unicode property support.
247     Even when Unicode property support is available, PCRE still uses its
248     own character tables when checking the case of low-valued characters,
249     so as not to degrade performance. The Unicode property information is
250 ph10 572 used only for characters with higher values. Furthermore, PCRE supports
251     case-insensitive matching only when there is a one-to-one mapping
252     between a letter's cases. There are a small number of many-to-one map-
253     pings in Unicode; these are not supported by PCRE.
254 nigel 63
255    
256     AUTHOR
257    
258 nigel 77 Philip Hazel
259 ph10 99 University Computing Service
260 nigel 93 Cambridge CB2 3QH, England.
261 nigel 63
262 ph10 572 Putting an actual email address here seems to have been a spam magnet,
263     so I've taken it away. If you want to email me, use my two initials,
264 ph10 153 followed by the two digits 10, at the domain cam.ac.uk.
265 nigel 77
266 nigel 63
267 ph10 99 REVISION
268 nigel 63
269 ph10 572 Last updated: 13 November 2010
270 ph10 507 Copyright (c) 1997-2010 University of Cambridge.
271 ph10 99 ------------------------------------------------------------------------------
272 ph10 589
273    
274 nigel 79 PCREBUILD(3) PCREBUILD(3)
275 nigel 63
276 nigel 79
277 nigel 73 NAME
278     PCRE - Perl-compatible regular expressions
279    
280 nigel 77
281 nigel 63 PCRE BUILD-TIME OPTIONS
282    
283 nigel 73 This document describes the optional features of PCRE that can be
284 ph10 261 selected when the library is compiled. It assumes use of the configure
285     script, where the optional features are selected or deselected by pro-
286     viding options to configure before running the make command. However,
287     the same options can be selected in both Unix-like and non-Unix-like
288 ph10 453 environments using the GUI facility of cmake-gui if you are using CMake
289     instead of configure to build PCRE.
290 nigel 63
291 ph10 453 There is a lot more information about building PCRE in non-Unix-like
292     environments in the file called NON_UNIX_USE, which is part of the PCRE
293     distribution. You should consult this file as well as the README file
294     if you are building in a non-Unix-like environment.
295    
296 ph10 261 The complete list of options for configure (which includes the standard
297 ph10 453 ones such as the selection of the installation directory) can be
298 ph10 261 obtained by running
299    
300 nigel 73 ./configure --help
301 nigel 63
302 ph10 453 The following sections include descriptions of options whose names
303 ph10 128 begin with --enable or --disable. These settings specify changes to the
304 ph10 453 defaults for the configure command. Because of the way that configure
305     works, --enable and --disable always come in pairs, so the complemen-
306     tary option always exists as well, but as it specifies the default, it
307 ph10 128 is not described.
308 nigel 63
309    
310 nigel 83 C++ SUPPORT
311    
312     By default, the configure script will search for a C++ compiler and C++
313     header files. If it finds them, it automatically builds the C++ wrapper
314     library for PCRE. You can disable this by adding
315    
316     --disable-cpp
317    
318     to the configure command.
319    
320    
321 nigel 63 UTF-8 SUPPORT
322    
323 ph10 392 To build PCRE with support for UTF-8 Unicode character strings, add
324 nigel 63
325 nigel 73 --enable-utf8
326 nigel 63
327 ph10 453 to the configure command. Of itself, this does not make PCRE treat
328     strings as UTF-8. As well as compiling PCRE with this option, you also
329     have have to set the PCRE_UTF8 option when you call the pcre_compile()
330 ph10 461 or pcre_compile2() functions.
331 nigel 63
332 ph10 453 If you set --enable-utf8 when compiling in an EBCDIC environment, PCRE
333 ph10 392 expects its input to be either ASCII or UTF-8 (depending on the runtime
334 ph10 453 option). It is not possible to support both EBCDIC and UTF-8 codes in
335     the same version of the library. Consequently, --enable-utf8 and
336 ph10 392 --enable-ebcdic are mutually exclusive.
337 nigel 63
338 ph10 392
339 nigel 75 UNICODE CHARACTER PROPERTY SUPPORT
340    
341 ph10 453 UTF-8 support allows PCRE to process character values greater than 255
342     in the strings that it handles. On its own, however, it does not pro-
343 nigel 75 vide any facilities for accessing the properties of such characters. If
344 ph10 453 you want to be able to use the pattern escapes \P, \p, and \X, which
345 nigel 75 refer to Unicode character properties, you must add
346    
347     --enable-unicode-properties
348    
349 ph10 453 to the configure command. This implies UTF-8 support, even if you have
350 nigel 75 not explicitly requested it.
351    
352 ph10 453 Including Unicode property support adds around 30K of tables to the
353     PCRE library. Only the general category properties such as Lu and Nd
354 ph10 128 are supported. Details are given in the pcrepattern documentation.
355 nigel 75
356    
357 nigel 63 CODE VALUE OF NEWLINE
358    
359 ph10 453 By default, PCRE interprets the linefeed (LF) character as indicating
360     the end of a line. This is the normal newline character on Unix-like
361     systems. You can compile PCRE to use carriage return (CR) instead, by
362 ph10 392 adding
363 nigel 63
364 nigel 73 --enable-newline-is-cr
365 nigel 63
366 ph10 453 to the configure command. There is also a --enable-newline-is-lf
367 nigel 91 option, which explicitly specifies linefeed as the newline character.
368 nigel 63
369 nigel 91 Alternatively, you can specify that line endings are to be indicated by
370     the two character sequence CRLF. If you want this, add
371 nigel 63
372 nigel 91 --enable-newline-is-crlf
373    
374 nigel 93 to the configure command. There is a fourth option, specified by
375 nigel 91
376 ph10 150 --enable-newline-is-anycrlf
377    
378 ph10 453 which causes PCRE to recognize any of the three sequences CR, LF, or
379 ph10 150 CRLF as indicating a line ending. Finally, a fifth option, specified by
380    
381 nigel 93 --enable-newline-is-any
382 nigel 91
383 ph10 150 causes PCRE to recognize any Unicode newline sequence.
384 nigel 93
385 ph10 453 Whatever line ending convention is selected when PCRE is built can be
386     overridden when the library functions are called. At build time it is
387 nigel 93 conventional to use the standard for your operating system.
388    
389    
390 ph10 231 WHAT \R MATCHES
391    
392 ph10 453 By default, the sequence \R in a pattern matches any Unicode newline
393     sequence, whatever has been selected as the line ending sequence. If
394 ph10 231 you specify
395    
396     --enable-bsr-anycrlf
397    
398 ph10 453 the default is changed so that \R matches only CR, LF, or CRLF. What-
399     ever is selected when PCRE is built can be overridden when the library
400 ph10 231 functions are called.
401    
402    
403 nigel 63 BUILDING SHARED AND STATIC LIBRARIES
404    
405 ph10 453 The PCRE building process uses libtool to build both shared and static
406     Unix libraries by default. You can suppress one of these by adding one
407 nigel 73 of
408 nigel 63
409 nigel 73 --disable-shared
410     --disable-static
411 nigel 63
412 nigel 73 to the configure command, as required.
413 nigel 63
414    
415     POSIX MALLOC USAGE
416    
417 nigel 75 When PCRE is called through the POSIX interface (see the pcreposix doc-
418 ph10 453 umentation), additional working storage is required for holding the
419     pointers to capturing substrings, because PCRE requires three integers
420     per substring, whereas the POSIX interface provides only two. If the
421 nigel 73 number of expected substrings is small, the wrapper function uses space
422     on the stack, because this is faster than using malloc() for each call.
423     The default threshold above which the stack is no longer used is 10; it
424     can be changed by adding a setting such as
425 nigel 63
426 nigel 73 --with-posix-malloc-threshold=20
427 nigel 63
428 nigel 73 to the configure command.
429 nigel 63
430    
431     HANDLING VERY LARGE PATTERNS
432    
433 ph10 453 Within a compiled pattern, offset values are used to point from one
434     part to another (for example, from an opening parenthesis to an alter-
435     nation metacharacter). By default, two-byte values are used for these
436     offsets, leading to a maximum size for a compiled pattern of around
437     64K. This is sufficient to handle all but the most gigantic patterns.
438 ph10 461 Nevertheless, some people do want to process truyl enormous patterns,
439     so it is possible to compile PCRE to use three-byte or four-byte off-
440     sets by adding a setting such as
441 nigel 63
442 nigel 73 --with-link-size=3
443 nigel 63
444 ph10 453 to the configure command. The value given must be 2, 3, or 4. Using
445     longer offsets slows down the operation of PCRE because it has to load
446 nigel 73 additional bytes when handling them.
447 nigel 63
448    
449 nigel 73 AVOIDING EXCESSIVE STACK USAGE
450    
451 nigel 77 When matching with the pcre_exec() function, PCRE implements backtrack-
452 ph10 453 ing by making recursive calls to an internal function called match().
453     In environments where the size of the stack is limited, this can se-
454     verely limit PCRE's operation. (The Unix environment does not usually
455 nigel 91 suffer from this problem, but it may sometimes be necessary to increase
456 ph10 453 the maximum stack size. There is a discussion in the pcrestack docu-
457     mentation.) An alternative approach to recursion that uses memory from
458     the heap to remember data, instead of using recursive function calls,
459     has been implemented to work round the problem of limited stack size.
460 nigel 91 If you want to build a version of PCRE that works this way, add
461 nigel 73
462     --disable-stack-for-recursion
463    
464 ph10 453 to the configure command. With this configuration, PCRE will use the
465     pcre_stack_malloc and pcre_stack_free variables to call memory manage-
466     ment functions. By default these point to malloc() and free(), but you
467 ph10 461 can replace the pointers so that your own functions are used instead.
468 nigel 73
469 ph10 453 Separate functions are provided rather than using pcre_malloc and
470     pcre_free because the usage is very predictable: the block sizes
471     requested are always the same, and the blocks are always freed in
472     reverse order. A calling program might be able to implement optimized
473     functions that perform better than malloc() and free(). PCRE runs
474 ph10 182 noticeably more slowly when built in this way. This option affects only
475 ph10 461 the pcre_exec() function; it is not relevant for pcre_dfa_exec().
476 nigel 73
477 ph10 182
478 nigel 91 LIMITING PCRE RESOURCE USAGE
479    
480 ph10 461 Internally, PCRE has a function called match(), which it calls repeat-
481     edly (sometimes recursively) when matching a pattern with the
482     pcre_exec() function. By controlling the maximum number of times this
483     function may be called during a single matching operation, a limit can
484     be placed on the resources used by a single call to pcre_exec(). The
485     limit can be changed at run time, as described in the pcreapi documen-
486     tation. The default is 10 million, but this can be changed by adding a
487 nigel 91 setting such as
488    
489     --with-match-limit=500000
490    
491 ph10 461 to the configure command. This setting has no effect on the
492 nigel 91 pcre_dfa_exec() matching function.
493    
494 ph10 461 In some environments it is desirable to limit the depth of recursive
495 nigel 91 calls of match() more strictly than the total number of calls, in order
496 ph10 461 to restrict the maximum amount of stack (or heap, if --disable-stack-
497 nigel 91 for-recursion is specified) that is used. A second limit controls this;
498 ph10 461 it defaults to the value that is set for --with-match-limit, which
499     imposes no additional constraints. However, you can set a lower limit
500 nigel 91 by adding, for example,
501    
502     --with-match-limit-recursion=10000
503    
504 ph10 461 to the configure command. This value can also be overridden at run
505 nigel 91 time.
506    
507    
508 ph10 128 CREATING CHARACTER TABLES AT BUILD TIME
509    
510 ph10 461 PCRE uses fixed tables for processing characters whose code values are
511     less than 256. By default, PCRE is built with a set of tables that are
512     distributed in the file pcre_chartables.c.dist. These tables are for
513 ph10 128 ASCII codes only. If you add
514    
515     --enable-rebuild-chartables
516    
517 ph10 461 to the configure command, the distributed tables are no longer used.
518     Instead, a program called dftables is compiled and run. This outputs
519 ph10 128 the source for new set of tables, created in the default locale of your
520     C runtime system. (This method of replacing the tables does not work if
521 ph10 461 you are cross compiling, because dftables is run on the local host. If
522     you need to create alternative tables when cross compiling, you will
523 ph10 128 have to do so "by hand".)
524    
525    
526 nigel 73 USING EBCDIC CODE
527    
528 ph10 461 PCRE assumes by default that it will run in an environment where the
529     character code is ASCII (or Unicode, which is a superset of ASCII).
530     This is the case for most computer operating systems. PCRE can, how-
531 ph10 197 ever, be compiled to run in an EBCDIC environment by adding
532 nigel 73
533     --enable-ebcdic
534    
535 ph10 128 to the configure command. This setting implies --enable-rebuild-charta-
536 ph10 461 bles. You should only use it if you know that you are in an EBCDIC
537     environment (for example, an IBM mainframe operating system). The
538 ph10 392 --enable-ebcdic option is incompatible with --enable-utf8.
539 nigel 73
540 nigel 93
541 ph10 286 PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT
542    
543     By default, pcregrep reads all files as plain text. You can build it so
544     that it recognizes files whose names end in .gz or .bz2, and reads them
545     with libz or libbz2, respectively, by adding one or both of
546    
547     --enable-pcregrep-libz
548     --enable-pcregrep-libbz2
549    
550     to the configure command. These options naturally require that the rel-
551 ph10 461 evant libraries are installed on your system. Configuration will fail
552 ph10 286 if they are not.
553    
554    
555 ph10 289 PCRETEST OPTION FOR LIBREADLINE SUPPORT
556    
557     If you add
558    
559     --enable-pcretest-libreadline
560    
561 ph10 461 to the configure command, pcretest is linked with the libreadline
562     library, and when its input is from a terminal, it reads it using the
563 ph10 289 readline() function. This provides line-editing and history facilities.
564 ph10 461 Note that libreadline is GPL-licensed, so if you distribute a binary of
565 ph10 289 pcretest linked in this way, there may be licensing issues.
566    
567 ph10 461 Setting this option causes the -lreadline option to be added to the
568     pcretest build. In many operating environments with a sytem-installed
569 ph10 345 libreadline this is sufficient. However, in some environments (e.g. if
570 ph10 461 an unmodified distribution version of readline is in use), some extra
571     configuration may be necessary. The INSTALL file for libreadline says
572 ph10 345 this:
573 ph10 289
574 ph10 345 "Readline uses the termcap functions, but does not link with the
575     termcap or curses library itself, allowing applications which link
576     with readline the to choose an appropriate library."
577    
578 ph10 461 If your environment has not been set up so that an appropriate library
579 ph10 345 is automatically included, you may need to add something like
580    
581     LIBS="-ncurses"
582    
583     immediately before the configure command.
584    
585    
586 nigel 93 SEE ALSO
587    
588     pcreapi(3), pcre_config(3).
589    
590 nigel 63
591 ph10 99 AUTHOR
592 nigel 63
593 ph10 99 Philip Hazel
594     University Computing Service
595     Cambridge CB2 3QH, England.
596    
597    
598     REVISION
599    
600 ph10 461 Last updated: 29 September 2009
601 ph10 392 Copyright (c) 1997-2009 University of Cambridge.
602 ph10 99 ------------------------------------------------------------------------------
603 ph10 589
604    
605 nigel 79 PCREMATCHING(3) PCREMATCHING(3)
606 nigel 63
607 nigel 79
608 nigel 77 NAME
609     PCRE - Perl-compatible regular expressions
610 nigel 73
611 nigel 77
612     PCRE MATCHING ALGORITHMS
613    
614     This document describes the two different algorithms that are available
615     in PCRE for matching a compiled regular expression against a given sub-
616     ject string. The "standard" algorithm is the one provided by the
617     pcre_exec() function. This works in the same was as Perl's matching
618     function, and provides a Perl-compatible matching operation.
619    
620     An alternative algorithm is provided by the pcre_dfa_exec() function;
621     this operates in a different way, and is not Perl-compatible. It has
622     advantages and disadvantages compared with the standard algorithm, and
623     these are described below.
624    
625     When there is only one possible way in which a given subject string can
626     match a pattern, the two algorithms give the same answer. A difference
627     arises, however, when there are multiple possibilities. For example, if
628     the pattern
629    
630     ^<.*>
631    
632     is matched against the string
633    
634     <something> <something else> <something further>
635    
636     there are three possible answers. The standard algorithm finds only one
637 nigel 93 of them, whereas the alternative algorithm finds all three.
638 nigel 77
639    
640     REGULAR EXPRESSIONS AS TREES
641    
642     The set of strings that are matched by a regular expression can be rep-
643     resented as a tree structure. An unlimited repetition in the pattern
644     makes the tree of infinite size, but it is still a tree. Matching the
645     pattern to a given subject string (from a given starting point) can be
646 nigel 91 thought of as a search of the tree. There are two ways to search a
647     tree: depth-first and breadth-first, and these correspond to the two
648     matching algorithms provided by PCRE.
649 nigel 77
650    
651     THE STANDARD MATCHING ALGORITHM
652    
653 ph10 148 In the terminology of Jeffrey Friedl's book "Mastering Regular Expres-
654     sions", the standard algorithm is an "NFA algorithm". It conducts a
655 nigel 77 depth-first search of the pattern tree. That is, it proceeds along a
656     single path through the tree, checking that the subject matches what is
657     required. When there is a mismatch, the algorithm tries any alterna-
658     tives at the current point, and if they all fail, it backs up to the
659     previous branch point in the tree, and tries the next alternative
660     branch at that level. This often involves backing up (moving to the
661     left) in the subject string as well. The order in which repetition
662     branches are tried is controlled by the greedy or ungreedy nature of
663     the quantifier.
664    
665     If a leaf node is reached, a matching string has been found, and at
666     that point the algorithm stops. Thus, if there is more than one possi-
667     ble match, this algorithm returns the first one that it finds. Whether
668     this is the shortest, the longest, or some intermediate length depends
669     on the way the greedy and ungreedy repetition quantifiers are specified
670     in the pattern.
671    
672     Because it ends up with a single path through the tree, it is rela-
673     tively straightforward for this algorithm to keep track of the sub-
674     strings that are matched by portions of the pattern in parentheses.
675     This provides support for capturing parentheses and back references.
676    
677    
678 nigel 93 THE ALTERNATIVE MATCHING ALGORITHM
679 nigel 77
680 nigel 93 This algorithm conducts a breadth-first search of the tree. Starting
681     from the first matching point in the subject, it scans the subject
682     string from left to right, once, character by character, and as it does
683     this, it remembers all the paths through the tree that represent valid
684     matches. In Friedl's terminology, this is a kind of "DFA algorithm",
685     though it is not implemented as a traditional finite state machine (it
686     keeps multiple states active simultaneously).
687 nigel 77
688 ph10 461 Although the general principle of this matching algorithm is that it
689     scans the subject string only once, without backtracking, there is one
690     exception: when a lookaround assertion is encountered, the characters
691     following or preceding the current point have to be independently
692     inspected.
693    
694 nigel 93 The scan continues until either the end of the subject is reached, or
695     there are no more unterminated paths. At this point, terminated paths
696     represent the different matching possibilities (if there are none, the
697     match has failed). Thus, if there is more than one possible match,
698 nigel 77 this algorithm finds all of them, and in particular, it finds the long-
699 ph10 572 est. The matches are returned in decreasing order of length. There is
700     an option to stop the algorithm after the first match (which is neces-
701     sarily the shortest) is found.
702 nigel 77
703     Note that all the matches that are found start at the same point in the
704     subject. If the pattern
705    
706 ph10 572 cat(er(pillar)?)?
707 nigel 77
708 ph10 572 is matched against the string "the caterpillar catchment", the result
709     will be the three strings "caterpillar", "cater", and "cat" that start
710     at the fifth character of the subject. The algorithm does not automati-
711     cally move on to find matches that start at later positions.
712 nigel 77
713     There are a number of features of PCRE regular expressions that are not
714 nigel 93 supported by the alternative matching algorithm. They are as follows:
715 nigel 77
716 ph10 572 1. Because the algorithm finds all possible matches, the greedy or
717     ungreedy nature of repetition quantifiers is not relevant. Greedy and
718 nigel 93 ungreedy quantifiers are treated in exactly the same way. However, pos-
719 ph10 572 sessive quantifiers can make a difference when what follows could also
720 nigel 93 match what is quantified, for example in a pattern like this:
721 nigel 77
722 nigel 93 ^a++\w!
723    
724 ph10 572 This pattern matches "aaab!" but not "aaa!", which would be matched by
725     a non-possessive quantifier. Similarly, if an atomic group is present,
726     it is matched as if it were a standalone pattern at the current point,
727     and the longest match is then "locked in" for the rest of the overall
728 nigel 93 pattern.
729    
730 nigel 77 2. When dealing with multiple paths through the tree simultaneously, it
731 ph10 572 is not straightforward to keep track of captured substrings for the
732     different matching possibilities, and PCRE's implementation of this
733 nigel 77 algorithm does not attempt to do this. This means that no captured sub-
734     strings are available.
735    
736 ph10 572 3. Because no substrings are captured, back references within the pat-
737 nigel 77 tern are not supported, and cause errors if encountered.
738    
739 ph10 572 4. For the same reason, conditional expressions that use a backrefer-
740     ence as the condition or test for a specific group recursion are not
741 nigel 93 supported.
742 nigel 77
743 ph10 572 5. Because many paths through the tree may be active, the \K escape
744 ph10 172 sequence, which resets the start of the match when encountered (but may
745 ph10 572 be on some paths and not on others), is not supported. It causes an
746 ph10 172 error if encountered.
747    
748 ph10 572 6. Callouts are supported, but the value of the capture_top field is
749 nigel 77 always 1, and the value of the capture_last field is always -1.
750    
751 ph10 572 7. The \C escape sequence, which (in the standard algorithm) matches a
752     single byte, even in UTF-8 mode, is not supported because the alterna-
753     tive algorithm moves through the subject string one character at a
754 nigel 93 time, for all active paths through the tree.
755 nigel 77
756 ph10 572 8. Except for (*FAIL), the backtracking control verbs such as (*PRUNE)
757     are not supported. (*FAIL) is supported, and behaves like a failing
758 ph10 345 negative assertion.
759 nigel 77
760 ph10 211
761 nigel 93 ADVANTAGES OF THE ALTERNATIVE ALGORITHM
762 nigel 77
763 ph10 572 Using the alternative matching algorithm provides the following advan-
764 nigel 93 tages:
765 nigel 77
766     1. All possible matches (at a single point in the subject) are automat-
767 ph10 572 ically found, and in particular, the longest match is found. To find
768 nigel 77 more than one match using the standard algorithm, you have to do kludgy
769     things with callouts.
770    
771 ph10 572 2. Because the alternative algorithm scans the subject string just
772     once, and never needs to backtrack, it is possible to pass very long
773     subject strings to the matching function in several pieces, checking
774     for partial matching each time. Although it is possible to do multi-
775     segment matching using the standard algorithm (pcre_exec()), by retain-
776     ing partially matched substrings, it is more complicated. The pcrepar-
777     tial documentation gives details of partial matching and discusses
778     multi-segment matching.
779 nigel 77
780    
781 nigel 93 DISADVANTAGES OF THE ALTERNATIVE ALGORITHM
782 nigel 77
783 nigel 93 The alternative algorithm suffers from a number of disadvantages:
784 nigel 77
785 ph10 453 1. It is substantially slower than the standard algorithm. This is
786     partly because it has to search for all possible matches, but is also
787 nigel 77 because it is less susceptible to optimization.
788    
789     2. Capturing parentheses and back references are not supported.
790    
791 nigel 93 3. Although atomic groups are supported, their use does not provide the
792     performance advantage that it does for the standard algorithm.
793 nigel 77
794    
795 ph10 99 AUTHOR
796 nigel 77
797 ph10 99 Philip Hazel
798     University Computing Service
799     Cambridge CB2 3QH, England.
800    
801    
802     REVISION
803    
804 ph10 572 Last updated: 17 November 2010
805 ph10 567 Copyright (c) 1997-2010 University of Cambridge.
806 ph10 99 ------------------------------------------------------------------------------
807 ph10 589
808    
809 nigel 79 PCREAPI(3) PCREAPI(3)
810 nigel 77
811 nigel 79
812 nigel 73 NAME
813     PCRE - Perl-compatible regular expressions
814    
815 nigel 77
816 nigel 75 PCRE NATIVE API
817 nigel 63
818 nigel 73 #include <pcre.h>
819 nigel 41
820 nigel 73 pcre *pcre_compile(const char *pattern, int options,
821     const char **errptr, int *erroffset,
822     const unsigned char *tableptr);
823 nigel 41
824 nigel 77 pcre *pcre_compile2(const char *pattern, int options,
825     int *errorcodeptr,
826     const char **errptr, int *erroffset,
827     const unsigned char *tableptr);
828    
829 nigel 73 pcre_extra *pcre_study(const pcre *code, int options,
830     const char **errptr);
831 nigel 41
832 nigel 73 int pcre_exec(const pcre *code, const pcre_extra *extra,
833     const char *subject, int length, int startoffset,
834     int options, int *ovector, int ovecsize);
835 nigel 41
836 nigel 77 int pcre_dfa_exec(const pcre *code, const pcre_extra *extra,
837     const char *subject, int length, int startoffset,
838     int options, int *ovector, int ovecsize,
839     int *workspace, int wscount);
840    
841 nigel 73 int pcre_copy_named_substring(const pcre *code,
842     const char *subject, int *ovector,
843     int stringcount, const char *stringname,
844     char *buffer, int buffersize);
845 nigel 63
846 nigel 73 int pcre_copy_substring(const char *subject, int *ovector,
847     int stringcount, int stringnumber, char *buffer,
848     int buffersize);
849 nigel 41
850 nigel 73 int pcre_get_named_substring(const pcre *code,
851     const char *subject, int *ovector,
852     int stringcount, const char *stringname,
853     const char **stringptr);
854 nigel 63
855 nigel 73 int pcre_get_stringnumber(const pcre *code,
856     const char *name);
857 nigel 63
858 nigel 91 int pcre_get_stringtable_entries(const pcre *code,
859     const char *name, char **first, char **last);
860    
861 nigel 73 int pcre_get_substring(const char *subject, int *ovector,
862     int stringcount, int stringnumber,
863     const char **stringptr);
864 nigel 41
865 nigel 73 int pcre_get_substring_list(const char *subject,
866     int *ovector, int stringcount, const char ***listptr);
867 nigel 41
868 nigel 73 void pcre_free_substring(const char *stringptr);
869 nigel 49
870 nigel 73 void pcre_free_substring_list(const char **stringptr);
871 nigel 49
872 nigel 73 const unsigned char *pcre_maketables(void);
873 nigel 41
874 nigel 73 int pcre_fullinfo(const pcre *code, const pcre_extra *extra,
875     int what, void *where);
876 nigel 43
877 nigel 73 int pcre_info(const pcre *code, int *optptr, int *firstcharptr);
878 nigel 63
879 nigel 77 int pcre_refcount(pcre *code, int adjust);
880    
881 nigel 73 int pcre_config(int what, void *where);
882 nigel 41
883 nigel 73 char *pcre_version(void);
884 nigel 63
885 nigel 73 void *(*pcre_malloc)(size_t);
886 nigel 41
887 nigel 73 void (*pcre_free)(void *);
888 nigel 41
889 nigel 73 void *(*pcre_stack_malloc)(size_t);
890 nigel 41
891 nigel 73 void (*pcre_stack_free)(void *);
892 nigel 41
893 nigel 73 int (*pcre_callout)(pcre_callout_block *);
894 nigel 41
895 nigel 73
896 nigel 75 PCRE API OVERVIEW
897 nigel 41
898 nigel 73 PCRE has its own native API, which is described in this document. There
899 nigel 93 are also some wrapper functions that correspond to the POSIX regular
900 nigel 77 expression API. These are described in the pcreposix documentation.
901     Both of these APIs define a set of C function calls. A C++ wrapper is
902     distributed with PCRE. It is documented in the pcrecpp page.
903 nigel 43
904 nigel 77 The native API C function prototypes are defined in the header file
905     pcre.h, and on Unix systems the library itself is called libpcre. It
906 nigel 75 can normally be accessed by adding -lpcre to the command for linking an
907     application that uses PCRE. The header file defines the macros
908     PCRE_MAJOR and PCRE_MINOR to contain the major and minor release num-
909     bers for the library. Applications can use these to include support
910     for different releases of PCRE.
911 nigel 41
912 ph10 535 In a Windows environment, if you want to statically link an application
913     program against a non-dll pcre.a file, you must define PCRE_STATIC
914     before including pcre.h or pcrecpp.h, because otherwise the pcre_mal-
915     loc() and pcre_free() exported functions will be declared
916     __declspec(dllimport), with unwanted results.
917    
918 nigel 77 The functions pcre_compile(), pcre_compile2(), pcre_study(), and
919     pcre_exec() are used for compiling and matching regular expressions in
920     a Perl-compatible manner. A sample program that demonstrates the sim-
921     plest way of using them is provided in the file called pcredemo.c in
922 ph10 429 the PCRE source distribution. A listing of this program is given in the
923     pcredemo documentation, and the pcresample documentation describes how
924     to compile and run it.
925 nigel 49
926 nigel 77 A second matching function, pcre_dfa_exec(), which is not Perl-compati-
927 ph10 429 ble, is also provided. This uses a different algorithm for the match-
928     ing. The alternative algorithm finds all possible matches (at a given
929 ph10 453 point in the subject), and scans the subject just once (unless there
930     are lookbehind assertions). However, this algorithm does not return
931     captured substrings. A description of the two matching algorithms and
932     their advantages and disadvantages is given in the pcrematching docu-
933     mentation.
934 nigel 63
935 ph10 453 In addition to the main compiling and matching functions, there are
936 nigel 77 convenience functions for extracting captured substrings from a subject
937     string that is matched by pcre_exec(). They are:
938    
939 nigel 73 pcre_copy_substring()
940     pcre_copy_named_substring()
941     pcre_get_substring()
942     pcre_get_named_substring()
943     pcre_get_substring_list()
944 nigel 75 pcre_get_stringnumber()
945 nigel 91 pcre_get_stringtable_entries()
946 nigel 63
947 nigel 73 pcre_free_substring() and pcre_free_substring_list() are also provided,
948     to free the memory used for extracted strings.
949 nigel 41
950 ph10 453 The function pcre_maketables() is used to build a set of character
951     tables in the current locale for passing to pcre_compile(),
952     pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is
953     provided for specialist use. Most commonly, no special tables are
954     passed, in which case internal tables that are generated when PCRE is
955 nigel 77 built are used.
956 nigel 49
957 ph10 453 The function pcre_fullinfo() is used to find out information about a
958     compiled pattern; pcre_info() is an obsolete version that returns only
959     some of the available information, but is retained for backwards com-
960     patibility. The function pcre_version() returns a pointer to a string
961 nigel 73 containing the version of PCRE and its date of release.
962 nigel 41
963 ph10 453 The function pcre_refcount() maintains a reference count in a data
964     block containing a compiled pattern. This is provided for the benefit
965 nigel 77 of object-oriented applications.
966    
967 ph10 453 The global variables pcre_malloc and pcre_free initially contain the
968     entry points of the standard malloc() and free() functions, respec-
969 nigel 73 tively. PCRE calls the memory management functions via these variables,
970 ph10 453 so a calling program can replace them if it wishes to intercept the
971 nigel 73 calls. This should be done before calling any PCRE functions.
972 nigel 41
973 ph10 453 The global variables pcre_stack_malloc and pcre_stack_free are also
974     indirections to memory management functions. These special functions
975     are used only when PCRE is compiled to use the heap for remembering
976 nigel 77 data, instead of recursive function calls, when running the pcre_exec()
977 ph10 453 function. See the pcrebuild documentation for details of how to do
978     this. It is a non-standard way of building PCRE, for use in environ-
979     ments that have limited stacks. Because of the greater use of memory
980     management, it runs more slowly. Separate functions are provided so
981     that special-purpose external code can be used for this case. When
982     used, these functions are always called in a stack-like manner (last
983     obtained, first freed), and always for memory blocks of the same size.
984     There is a discussion about PCRE's stack usage in the pcrestack docu-
985 nigel 91 mentation.
986 nigel 41
987 nigel 73 The global variable pcre_callout initially contains NULL. It can be set
988 ph10 453 by the caller to a "callout" function, which PCRE will then call at
989     specified points during a matching operation. Details are given in the
990 nigel 73 pcrecallout documentation.
991 nigel 41
992 nigel 73
993 nigel 91 NEWLINES
994    
995 ph10 453 PCRE supports five different conventions for indicating line breaks in
996     strings: a single CR (carriage return) character, a single LF (line-
997 ph10 150 feed) character, the two-character sequence CRLF, any of the three pre-
998 ph10 453 ceding, or any Unicode newline sequence. The Unicode newline sequences
999     are the three just mentioned, plus the single characters VT (vertical
1000     tab, U+000B), FF (formfeed, U+000C), NEL (next line, U+0085), LS (line
1001 ph10 150 separator, U+2028), and PS (paragraph separator, U+2029).
1002 nigel 93
1003 ph10 453 Each of the first three conventions is used by at least one operating
1004     system as its standard newline sequence. When PCRE is built, a default
1005     can be specified. The default default is LF, which is the Unix stan-
1006     dard. When PCRE is run, the default can be overridden, either when a
1007 nigel 93 pattern is compiled, or when it is matched.
1008    
1009 ph10 227 At compile time, the newline convention can be specified by the options
1010 ph10 453 argument of pcre_compile(), or it can be specified by special text at
1011 ph10 227 the start of the pattern itself; this overrides any other settings. See
1012     the pcrepattern page for details of the special character sequences.
1013    
1014 nigel 91 In the PCRE documentation the word "newline" is used to mean "the char-
1015 ph10 453 acter or pair of characters that indicate a line break". The choice of
1016     newline convention affects the handling of the dot, circumflex, and
1017 nigel 93 dollar metacharacters, the handling of #-comments in /x mode, and, when
1018 ph10 453 CRLF is a recognized line ending sequence, the match position advance-
1019 ph10 227 ment for a non-anchored pattern. There is more detail about this in the
1020 ph10 231 section on pcre_exec() options below.
1021 nigel 91
1022 ph10 453 The choice of newline convention does not affect the interpretation of
1023     the \n or \r escape sequences, nor does it affect what \R matches,
1024 ph10 231 which is controlled in a similar way, but by separate options.
1025 nigel 91
1026 ph10 231
1027 nigel 63 MULTITHREADING
1028    
1029 ph10 453 The PCRE functions can be used in multi-threading applications, with
1030 nigel 73 the proviso that the memory management functions pointed to by
1031     pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the
1032     callout function pointed to by pcre_callout, are shared by all threads.
1033 nigel 41
1034 ph10 453 The compiled form of a regular expression is not altered during match-
1035 nigel 73 ing, so the same compiled pattern can safely be used by several threads
1036     at once.
1037 nigel 41
1038    
1039 nigel 75 SAVING PRECOMPILED PATTERNS FOR LATER USE
1040    
1041     The compiled form of a regular expression can be saved and re-used at a
1042 ph10 453 later time, possibly by a different program, and even on a host other
1043     than the one on which it was compiled. Details are given in the
1044     pcreprecompile documentation. However, compiling a regular expression
1045     with one version of PCRE for use with a different version is not guar-
1046 ph10 155 anteed to work and may cause crashes.
1047 nigel 75
1048    
1049 nigel 63 CHECKING BUILD-TIME OPTIONS
1050 nigel 41
1051 nigel 73 int pcre_config(int what, void *where);
1052 nigel 63
1053 ph10 453 The function pcre_config() makes it possible for a PCRE client to dis-
1054 nigel 73 cover which optional features have been compiled into the PCRE library.
1055 ph10 453 The pcrebuild documentation has more details about these optional fea-
1056 nigel 73 tures.
1057 nigel 63
1058 ph10 453 The first argument for pcre_config() is an integer, specifying which
1059 nigel 73 information is required; the second argument is a pointer to a variable
1060 ph10 453 into which the information is placed. The following information is
1061 nigel 73 available:
1062 nigel 63
1063 nigel 73 PCRE_CONFIG_UTF8
1064 nigel 63
1065 ph10 453 The output is an integer that is set to one if UTF-8 support is avail-
1066 nigel 73 able; otherwise it is set to zero.
1067 nigel 63
1068 nigel 75 PCRE_CONFIG_UNICODE_PROPERTIES
1069    
1070 ph10 453 The output is an integer that is set to one if support for Unicode
1071 nigel 75 character properties is available; otherwise it is set to zero.
1072    
1073 nigel 73 PCRE_CONFIG_NEWLINE
1074 nigel 63
1075 ph10 453 The output is an integer whose value specifies the default character
1076     sequence that is recognized as meaning "newline". The four values that
1077 ph10 150 are supported are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF,
1078 ph10 453 and -1 for ANY. Though they are derived from ASCII, the same values
1079 ph10 392 are returned in EBCDIC environments. The default should normally corre-
1080     spond to the standard sequence for your operating system.
1081 nigel 63
1082 ph10 231 PCRE_CONFIG_BSR
1083    
1084     The output is an integer whose value indicates what character sequences
1085 ph10 453 the \R escape sequence matches by default. A value of 0 means that \R
1086     matches any Unicode line ending sequence; a value of 1 means that \R
1087 ph10 231 matches only CR, LF, or CRLF. The default can be overridden when a pat-
1088     tern is compiled or matched.
1089    
1090 nigel 73 PCRE_CONFIG_LINK_SIZE
1091 nigel 63
1092 ph10 453 The output is an integer that contains the number of bytes used for
1093 nigel 73 internal linkage in compiled regular expressions. The value is 2, 3, or
1094 ph10 453 4. Larger values allow larger regular expressions to be compiled, at
1095     the expense of slower matching. The default value of 2 is sufficient
1096     for all but the most massive patterns, since it allows the compiled
1097 nigel 73 pattern to be up to 64K in size.
1098 nigel 63
1099 nigel 73 PCRE_CONFIG_POSIX_MALLOC_THRESHOLD
1100 nigel 63
1101 ph10 453 The output is an integer that contains the threshold above which the
1102     POSIX interface uses malloc() for output vectors. Further details are
1103 nigel 73 given in the pcreposix documentation.
1104 nigel 63
1105 nigel 73 PCRE_CONFIG_MATCH_LIMIT
1106 nigel 63
1107 ph10 453 The output is a long integer that gives the default limit for the num-
1108     ber of internal matching function calls in a pcre_exec() execution.
1109 ph10 392 Further details are given with pcre_exec() below.
1110 nigel 63
1111 nigel 87 PCRE_CONFIG_MATCH_LIMIT_RECURSION
1112    
1113 ph10 392 The output is a long integer that gives the default limit for the depth
1114 ph10 453 of recursion when calling the internal matching function in a
1115     pcre_exec() execution. Further details are given with pcre_exec()
1116 ph10 392 below.
1117 nigel 87
1118 nigel 73 PCRE_CONFIG_STACKRECURSE
1119 nigel 63
1120 ph10 453 The output is an integer that is set to one if internal recursion when
1121 nigel 77 running pcre_exec() is implemented by recursive function calls that use
1122 ph10 453 the stack to remember their state. This is the usual way that PCRE is
1123 nigel 77 compiled. The output is zero if PCRE was compiled to use blocks of data
1124 ph10 453 on the heap instead of recursive function calls. In this case,
1125     pcre_stack_malloc and pcre_stack_free are called to manage memory
1126 nigel 77 blocks on the heap, thus avoiding the use of the stack.
1127 nigel 73
1128    
1129 nigel 41 COMPILING A PATTERN
1130 nigel 63
1131 nigel 73 pcre *pcre_compile(const char *pattern, int options,
1132     const char **errptr, int *erroffset,
1133     const unsigned char *tableptr);
1134 nigel 63
1135 nigel 77 pcre *pcre_compile2(const char *pattern, int options,
1136     int *errorcodeptr,
1137     const char **errptr, int *erroffset,
1138     const unsigned char *tableptr);
1139 nigel 41
1140 nigel 77 Either of the functions pcre_compile() or pcre_compile2() can be called
1141     to compile a pattern into an internal form. The only difference between
1142 ph10 453 the two interfaces is that pcre_compile2() has an additional argument,
1143 ph10 461 errorcodeptr, via which a numerical error code can be returned. To
1144     avoid too much repetition, we refer just to pcre_compile() below, but
1145     the information applies equally to pcre_compile2().
1146 nigel 77
1147     The pattern is a C string terminated by a binary zero, and is passed in
1148 ph10 453 the pattern argument. A pointer to a single block of memory that is
1149     obtained via pcre_malloc is returned. This contains the compiled code
1150 nigel 77 and related data. The pcre type is defined for the returned block; this
1151     is a typedef for a structure whose contents are not externally defined.
1152 nigel 91 It is up to the caller to free the memory (via pcre_free) when it is no
1153     longer required.
1154 nigel 77
1155 ph10 453 Although the compiled code of a PCRE regex is relocatable, that is, it
1156 nigel 73 does not depend on memory location, the complete pcre data block is not
1157 ph10 453 fully relocatable, because it may contain a copy of the tableptr argu-
1158 nigel 75 ment, which is an address (see below).
1159 nigel 41
1160 nigel 93 The options argument contains various bit settings that affect the com-
1161 ph10 453 pilation. It should be zero if no options are required. The available
1162     options are described below. Some of them (in particular, those that
1163 ph10 461 are compatible with Perl, but some others as well) can also be set and
1164 ph10 453 unset from within the pattern (see the detailed description in the
1165     pcrepattern documentation). For those options that can be different in
1166     different parts of the pattern, the contents of the options argument
1167 ph10 461 specifies their settings at the start of compilation and execution. The
1168 ph10 579 PCRE_ANCHORED, PCRE_BSR_xxx, PCRE_NEWLINE_xxx, PCRE_NO_UTF8_CHECK, and
1169     PCRE_NO_START_OPT options can be set at the time of matching as well as
1170     at compile time.
1171 nigel 41
1172 nigel 73 If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise,
1173 ph10 579 if compilation of a pattern fails, pcre_compile() returns NULL, and
1174 nigel 73 sets the variable pointed to by errptr to point to a textual error mes-
1175 nigel 87 sage. This is a static string that is part of the library. You must not
1176 ph10 579 try to free it. The offset from the start of the pattern to the byte
1177 ph10 572 that was being processed when the error was discovered is placed in the
1178 ph10 579 variable pointed to by erroffset, which must not be NULL. If it is, an
1179 ph10 572 immediate error is given. Some errors are not detected until checks are
1180 ph10 579 carried out when the whole pattern has been scanned; in this case the
1181 ph10 572 offset is set to the end of the pattern.
1182 nigel 53
1183 ph10 579 Note that the offset is in bytes, not characters, even in UTF-8 mode.
1184     It may point into the middle of a UTF-8 character (for example, when
1185 ph10 572 PCRE_ERROR_BADUTF8 is returned for an invalid UTF-8 string).
1186    
1187 ph10 579 If pcre_compile2() is used instead of pcre_compile(), and the error-
1188     codeptr argument is not NULL, a non-zero error code number is returned
1189     via this argument in the event of an error. This is in addition to the
1190 nigel 77 textual error message. Error codes and messages are listed below.
1191    
1192 ph10 579 If the final argument, tableptr, is NULL, PCRE uses a default set of
1193     character tables that are built when PCRE is compiled, using the
1194     default C locale. Otherwise, tableptr must be an address that is the
1195     result of a call to pcre_maketables(). This value is stored with the
1196     compiled pattern, and used again by pcre_exec(), unless another table
1197 nigel 75 pointer is passed to it. For more discussion, see the section on locale
1198     support below.
1199 nigel 53
1200 ph10 579 This code fragment shows a typical straightforward call to pcre_com-
1201 nigel 73 pile():
1202 nigel 41
1203 nigel 73 pcre *re;
1204     const char *error;
1205     int erroffset;
1206     re = pcre_compile(
1207     "^A.*Z", /* the pattern */
1208     0, /* default options */
1209     &error, /* for error message */
1210     &erroffset, /* for error offset */
1211     NULL); /* use default character tables */
1212 nigel 41
1213 ph10 579 The following names for option bits are defined in the pcre.h header
1214 nigel 75 file:
1215 nigel 41
1216 nigel 73 PCRE_ANCHORED
1217 nigel 41
1218 nigel 73 If this bit is set, the pattern is forced to be "anchored", that is, it
1219 ph10 579 is constrained to match only at the first matching point in the string
1220     that is being searched (the "subject string"). This effect can also be
1221     achieved by appropriate constructs in the pattern itself, which is the
1222 nigel 73 only way to do it in Perl.
1223 nigel 41
1224 nigel 75 PCRE_AUTO_CALLOUT
1225    
1226     If this bit is set, pcre_compile() automatically inserts callout items,
1227 ph10 579 all with number 255, before each pattern item. For discussion of the
1228 nigel 75 callout facility, see the pcrecallout documentation.
1229    
1230 ph10 231 PCRE_BSR_ANYCRLF
1231     PCRE_BSR_UNICODE
1232    
1233     These options (which are mutually exclusive) control what the \R escape
1234 ph10 579 sequence matches. The choice is either to match only CR, LF, or CRLF,
1235 ph10 231 or to match any Unicode newline sequence. The default is specified when
1236     PCRE is built. It can be overridden from within the pattern, or by set-
1237     ting an option when a compiled pattern is matched.
1238    
1239 nigel 73 PCRE_CASELESS
1240 nigel 41
1241 ph10 579 If this bit is set, letters in the pattern match both upper and lower
1242     case letters. It is equivalent to Perl's /i option, and it can be
1243     changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE
1244     always understands the concept of case for characters whose values are
1245     less than 128, so caseless matching is always possible. For characters
1246     with higher values, the concept of case is supported if PCRE is com-
1247     piled with Unicode property support, but not otherwise. If you want to
1248     use caseless matching for characters 128 and above, you must ensure
1249     that PCRE is compiled with Unicode property support as well as with
1250 nigel 77 UTF-8 support.
1251 nigel 41
1252 nigel 73 PCRE_DOLLAR_ENDONLY
1253 nigel 41
1254 ph10 579 If this bit is set, a dollar metacharacter in the pattern matches only
1255     at the end of the subject string. Without this option, a dollar also
1256     matches immediately before a newline at the end of the string (but not
1257     before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored
1258     if PCRE_MULTILINE is set. There is no equivalent to this option in
1259 nigel 91 Perl, and no way to set it within a pattern.
1260 nigel 41
1261 nigel 73 PCRE_DOTALL
1262 nigel 41
1263 ph10 579 If this bit is set, a dot metacharacter in the pattern matches a char-
1264 ph10 572 acter of any value, including one that indicates a newline. However, it
1265 ph10 579 only ever matches one character, even if newlines are coded as CRLF.
1266     Without this option, a dot does not match when the current position is
1267 ph10 572 at a newline. This option is equivalent to Perl's /s option, and it can
1268 ph10 579 be changed within a pattern by a (?s) option setting. A negative class
1269 ph10 572 such as [^a] always matches newline characters, independent of the set-
1270     ting of this option.
1271 nigel 63
1272 nigel 91 PCRE_DUPNAMES
1273    
1274 ph10 579 If this bit is set, names used to identify capturing subpatterns need
1275 nigel 91 not be unique. This can be helpful for certain types of pattern when it
1276 ph10 579 is known that only one instance of the named subpattern can ever be
1277     matched. There are more details of named subpatterns below; see also
1278 nigel 91 the pcrepattern documentation.
1279    
1280 nigel 73 PCRE_EXTENDED
1281 nigel 41
1282 ph10 579 If this bit is set, whitespace data characters in the pattern are
1283 nigel 77 totally ignored except when escaped or inside a character class. White-
1284     space does not include the VT character (code 11). In addition, charac-
1285     ters between an unescaped # outside a character class and the next new-
1286 ph10 579 line, inclusive, are also ignored. This is equivalent to Perl's /x
1287     option, and it can be changed within a pattern by a (?x) option set-
1288 nigel 91 ting.
1289 nigel 41
1290 ph10 579 Which characters are interpreted as newlines is controlled by the
1291     options passed to pcre_compile() or by a special sequence at the start
1292     of the pattern, as described in the section entitled "Newline conven-
1293 ph10 572 tions" in the pcrepattern documentation. Note that the end of this type
1294 ph10 579 of comment is a literal newline sequence in the pattern; escape
1295 ph10 572 sequences that happen to represent a newline do not count.
1296 nigel 41
1297 ph10 579 This option makes it possible to include comments inside complicated
1298     patterns. Note, however, that this applies only to data characters.
1299     Whitespace characters may never appear within special character
1300 ph10 572 sequences in a pattern, for example within the sequence (?( that intro-
1301     duces a conditional subpattern.
1302    
1303 nigel 73 PCRE_EXTRA
1304 nigel 41
1305 ph10 579 This option was invented in order to turn on additional functionality
1306     of PCRE that is incompatible with Perl, but it is currently of very
1307     little use. When set, any backslash in a pattern that is followed by a
1308     letter that has no special meaning causes an error, thus reserving
1309     these combinations for future expansion. By default, as in Perl, a
1310     backslash followed by a letter with no special meaning is treated as a
1311 ph10 518 literal. (Perl can, however, be persuaded to give an error for this, by
1312 ph10 579 running it with the -w option.) There are at present no other features
1313     controlled by this option. It can also be set by a (?X) option setting
1314 ph10 518 within a pattern.
1315 nigel 41
1316 nigel 77 PCRE_FIRSTLINE
1317    
1318 ph10 579 If this option is set, an unanchored pattern is required to match
1319     before or at the first newline in the subject string, though the
1320 nigel 91 matched text may continue over the newline.
1321 nigel 77
1322 ph10 345 PCRE_JAVASCRIPT_COMPAT
1323    
1324     If this option is set, PCRE's behaviour is changed in some ways so that
1325 ph10 579 it is compatible with JavaScript rather than Perl. The changes are as
1326 ph10 345 follows:
1327    
1328 ph10 579 (1) A lone closing square bracket in a pattern causes a compile-time
1329     error, because this is illegal in JavaScript (by default it is treated
1330 ph10 345 as a data character). Thus, the pattern AB]CD becomes illegal when this
1331     option is set.
1332    
1333 ph10 579 (2) At run time, a back reference to an unset subpattern group matches
1334     an empty string (by default this causes the current matching alterna-
1335     tive to fail). A pattern such as (\1)(a) succeeds when this option is
1336     set (assuming it can find an "a" in the subject), whereas it fails by
1337 ph10 345 default, for Perl compatibility.
1338    
1339 nigel 73 PCRE_MULTILINE
1340 nigel 41
1341 ph10 579 By default, PCRE treats the subject string as consisting of a single
1342     line of characters (even if it actually contains newlines). The "start
1343     of line" metacharacter (^) matches only at the start of the string,
1344     while the "end of line" metacharacter ($) matches only at the end of
1345 nigel 75 the string, or before a terminating newline (unless PCRE_DOLLAR_ENDONLY
1346     is set). This is the same as Perl.
1347 nigel 63
1348 ph10 579 When PCRE_MULTILINE it is set, the "start of line" and "end of line"
1349     constructs match immediately following or immediately before internal
1350     newlines in the subject string, respectively, as well as at the very
1351     start and end. This is equivalent to Perl's /m option, and it can be
1352 nigel 91 changed within a pattern by a (?m) option setting. If there are no new-
1353 ph10 579 lines in a subject string, or no occurrences of ^ or $ in a pattern,
1354 nigel 73 setting PCRE_MULTILINE has no effect.
1355 nigel 63
1356 nigel 91 PCRE_NEWLINE_CR
1357     PCRE_NEWLINE_LF
1358     PCRE_NEWLINE_CRLF
1359 ph10 150 PCRE_NEWLINE_ANYCRLF
1360 nigel 93 PCRE_NEWLINE_ANY
1361 nigel 91
1362 ph10 579 These options override the default newline definition that was chosen
1363     when PCRE was built. Setting the first or the second specifies that a
1364     newline is indicated by a single character (CR or LF, respectively).
1365     Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the
1366     two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies
1367 ph10 150 that any of the three preceding sequences should be recognized. Setting
1368 ph10 579 PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be
1369 ph10 150 recognized. The Unicode newline sequences are the three just mentioned,
1370 ph10 579 plus the single characters VT (vertical tab, U+000B), FF (formfeed,
1371     U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS
1372     (paragraph separator, U+2029). The last two are recognized only in
1373 ph10 150 UTF-8 mode.
1374 nigel 91
1375 ph10 579 The newline setting in the options word uses three bits that are
1376 ph10 150 treated as a number, giving eight possibilities. Currently only six are
1377 ph10 579 used (default plus the five values above). This means that if you set
1378     more than one newline option, the combination may or may not be sensi-
1379 ph10 150 ble. For example, PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to
1380 ph10 579 PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and
1381 ph10 150 cause an error.
1382 nigel 91
1383 ph10 579 The only time that a line break in a pattern is specially recognized
1384     when compiling is when PCRE_EXTENDED is set. CR and LF are whitespace
1385     characters, and so are ignored in this mode. Also, an unescaped # out-
1386     side a character class indicates a comment that lasts until after the
1387     next line break sequence. In other circumstances, line break sequences
1388 ph10 572 in patterns are treated as literal data.
1389 nigel 93
1390     The newline option that is set at compile time becomes the default that
1391 ph10 392 is used for pcre_exec() and pcre_dfa_exec(), but it can be overridden.
1392 nigel 93
1393 nigel 73 PCRE_NO_AUTO_CAPTURE
1394 nigel 41
1395 nigel 73 If this option is set, it disables the use of numbered capturing paren-
1396 ph10 579 theses in the pattern. Any opening parenthesis that is not followed by
1397     ? behaves as if it were followed by ?: but named parentheses can still
1398     be used for capturing (and they acquire numbers in the usual way).
1399 nigel 73 There is no equivalent of this option in Perl.
1400 nigel 41
1401 ph10 579 NO_START_OPTIMIZE
1402    
1403     This is an option that acts at matching time; that is, it is really an
1404     option for pcre_exec() or pcre_dfa_exec(). If it is set at compile
1405     time, it is remembered with the compiled pattern and assumed at match-
1406     ing time. For details see the discussion of PCRE_NO_START_OPTIMIZE
1407     below.
1408    
1409 ph10 535 PCRE_UCP
1410    
1411 ph10 579 This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W,
1412     \w, and some of the POSIX character classes. By default, only ASCII
1413     characters are recognized, but if PCRE_UCP is set, Unicode properties
1414     are used instead to classify characters. More details are given in the
1415     section on generic character types in the pcrepattern page. If you set
1416     PCRE_UCP, matching one of the items it affects takes much longer. The
1417     option is available only if PCRE has been compiled with Unicode prop-
1418 ph10 572 erty support.
1419 ph10 535
1420 nigel 73 PCRE_UNGREEDY
1421 nigel 41
1422 ph10 579 This option inverts the "greediness" of the quantifiers so that they
1423     are not greedy by default, but become greedy if followed by "?". It is
1424     not compatible with Perl. It can also be set by a (?U) option setting
1425 nigel 73 within the pattern.
1426 nigel 41
1427 nigel 73 PCRE_UTF8
1428 nigel 49
1429 ph10 579 This option causes PCRE to regard both the pattern and the subject as
1430     strings of UTF-8 characters instead of single-byte character strings.
1431     However, it is available only when PCRE is built to include UTF-8 sup-
1432     port. If not, the use of this option provokes an error. Details of how
1433     this option changes the behaviour of PCRE are given in the section on
1434 nigel 75 UTF-8 support in the main pcre page.
1435 nigel 71
1436 nigel 73 PCRE_NO_UTF8_CHECK
1437 nigel 71
1438 nigel 73 When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is
1439 ph10 579 automatically checked. There is a discussion about the validity of
1440     UTF-8 strings in the main pcre page. If an invalid UTF-8 sequence of
1441     bytes is found, pcre_compile() returns an error. If you already know
1442 ph10 211 that your pattern is valid, and you want to skip this check for perfor-
1443 ph10 579 mance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is
1444     set, the effect of passing an invalid UTF-8 string as a pattern is
1445     undefined. It may cause your program to crash. Note that this option
1446     can also be passed to pcre_exec() and pcre_dfa_exec(), to suppress the
1447 ph10 211 UTF-8 validity checking of subject strings.
1448 nigel 71
1449 nigel 73
1450 nigel 77 COMPILATION ERROR CODES
1451    
1452 ph10 579 The following table lists the error codes than may be returned by
1453     pcre_compile2(), along with the error messages that may be returned by
1454     both compiling functions. As PCRE has developed, some error codes have
1455 nigel 93 fallen out of use. To avoid confusion, they have not been re-used.
1456 nigel 77
1457     0 no error
1458     1 \ at end of pattern
1459     2 \c at end of pattern
1460     3 unrecognized character follows \
1461     4 numbers out of order in {} quantifier
1462     5 number too big in {} quantifier
1463     6 missing terminating ] for character class
1464     7 invalid escape sequence in character class
1465     8 range out of order in character class
1466     9 nothing to repeat
1467 nigel 93 10 [this code is not in use]
1468 nigel 77 11 internal error: unexpected repeat
1469 ph10 292 12 unrecognized character after (? or (?-
1470 nigel 77 13 POSIX named classes are supported only within a class
1471     14 missing )
1472     15 reference to non-existent subpattern
1473     16 erroffset passed as NULL
1474     17 unknown option bit(s) set
1475     18 missing ) after comment
1476 nigel 93 19 [this code is not in use]
1477 ph10 292 20 regular expression is too large
1478 nigel 77 21 failed to get memory
1479     22 unmatched parentheses
1480     23 internal error: code overflow
1481     24 unrecognized character after (?<
1482     25 lookbehind assertion is not fixed length
1483 nigel 91 26 malformed number or name after (?(
1484 nigel 77 27 conditional group contains more than two branches
1485     28 assertion expected after (?(
1486 ph10 182 29 (?R or (?[+-]digits must be followed by )
1487 nigel 77 30 unknown POSIX class name
1488     31 POSIX collating elements are not supported
1489     32 this version of PCRE is not compiled with PCRE_UTF8 support
1490 nigel 93 33 [this code is not in use]
1491 nigel 77 34 character value in \x{...} sequence is too large
1492     35 invalid condition (?(0)
1493     36 \C not allowed in lookbehind assertion
1494     37 PCRE does not support \L, \l, \N, \U, or \u
1495     38 number after (?C is > 255
1496     39 closing ) for (?C expected
1497     40 recursive call could loop indefinitely
1498     41 unrecognized character after (?P
1499 nigel 93 42 syntax error in subpattern name (missing terminator)
1500 nigel 91 43 two named subpatterns have the same name
1501 nigel 77 44 invalid UTF-8 string
1502     45 support for \P, \p, and \X has not been compiled
1503     46 malformed \P or \p sequence
1504     47 unknown property name after \P or \p
1505 nigel 91 48 subpattern name is too long (maximum 32 characters)
1506 ph10 292 49 too many named subpatterns (maximum 10000)
1507 ph10 202 50 [this code is not in use]
1508 nigel 91 51 octal value is greater than \377 (not in UTF-8 mode)
1509 nigel 93 52 internal error: overran compiling workspace
1510 ph10 548 53 internal error: previously-checked referenced subpattern
1511     not found
1512 nigel 93 54 DEFINE group contains more than one branch
1513     55 repeating a DEFINE group is not allowed
1514 ph10 231 56 inconsistent NEWLINE options
1515 ph10 345 57 \g is not followed by a braced, angle-bracketed, or quoted
1516     name/number or by a plain number
1517     58 a numbered reference must not be zero
1518 ph10 512 59 an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT)
1519 ph10 292 60 (*VERB) not recognized
1520     61 number is too big
1521     62 subpattern name expected
1522     63 digit expected after (?+
1523 ph10 345 64 ] is an invalid data character in JavaScript compatibility mode
1524 ph10 548 65 different names for subpatterns of the same number are
1525     not allowed
1526 ph10 512 66 (*MARK) must have an argument
1527 ph10 535 67 this version of PCRE is not compiled with PCRE_UCP support
1528 nigel 77
1529 ph10 579 The numbers 32 and 10000 in errors 48 and 49 are defaults; different
1530 ph10 292 values may be used if the limits were changed when PCRE was built.
1531 nigel 77
1532 ph10 292
1533 nigel 63 STUDYING A PATTERN
1534 nigel 49
1535 nigel 77 pcre_extra *pcre_study(const pcre *code, int options
1536 nigel 73 const char **errptr);
1537 nigel 63
1538 ph10 579 If a compiled pattern is going to be used several times, it is worth
1539 nigel 75 spending more time analyzing it in order to speed up the time taken for
1540 ph10 579 matching. The function pcre_study() takes a pointer to a compiled pat-
1541 nigel 75 tern as its first argument. If studying the pattern produces additional
1542 ph10 579 information that will help speed up matching, pcre_study() returns a
1543     pointer to a pcre_extra block, in which the study_data field points to
1544 nigel 75 the results of the study.
1545 nigel 41
1546 nigel 75 The returned value from pcre_study() can be passed directly to
1547 ph10 579 pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con-
1548     tains other fields that can be set by the caller before the block is
1549 ph10 461 passed; these are described below in the section on matching a pattern.
1550 nigel 63
1551 ph10 579 If studying the pattern does not produce any useful information,
1552 nigel 75 pcre_study() returns NULL. In that circumstance, if the calling program
1553 ph10 579 wants to pass any of the other fields to pcre_exec() or
1554 ph10 461 pcre_dfa_exec(), it must set up its own pcre_extra block.
1555 nigel 41
1556 ph10 579 The second argument of pcre_study() contains option bits. At present,
1557 nigel 75 no options are defined, and this argument should always be zero.
1558    
1559 ph10 579 The third argument for pcre_study() is a pointer for an error message.
1560     If studying succeeds (even if no data is returned), the variable it
1561     points to is set to NULL. Otherwise it is set to point to a textual
1562 nigel 87 error message. This is a static string that is part of the library. You
1563 ph10 579 must not try to free it. You should test the error pointer for NULL
1564 nigel 87 after calling pcre_study(), to be sure that it has run successfully.
1565 nigel 41
1566 nigel 73 This is a typical call to pcre_study():
1567 nigel 53
1568 nigel 73 pcre_extra *pe;
1569     pe = pcre_study(
1570     re, /* result of pcre_compile() */
1571     0, /* no options exist */
1572     &error); /* set to NULL or points to a message */
1573 nigel 53
1574 ph10 461 Studying a pattern does two things: first, a lower bound for the length
1575     of subject string that is needed to match the pattern is computed. This
1576     does not mean that there are any strings of that length that match, but
1577 ph10 579 it does guarantee that no shorter strings match. The value is used by
1578     pcre_exec() and pcre_dfa_exec() to avoid wasting time by trying to
1579     match strings that are shorter than the lower bound. You can find out
1580 ph10 461 the value in a calling program via the pcre_fullinfo() function.
1581 nigel 41
1582 ph10 461 Studying a pattern is also useful for non-anchored patterns that do not
1583 ph10 579 have a single fixed starting character. A bitmap of possible starting
1584     bytes is created. This speeds up finding a position in the subject at
1585 ph10 461 which to start matching.
1586 nigel 41
1587 ph10 579 The two optimizations just described can be disabled by setting the
1588     PCRE_NO_START_OPTIMIZE option when calling pcre_exec() or
1589     pcre_dfa_exec(). You might want to do this if your pattern contains
1590     callouts or (*MARK), and you want to make use of these facilities in
1591     cases where matching fails. See the discussion of PCRE_NO_START_OPTI-
1592 ph10 572 MIZE below.
1593 ph10 461
1594 ph10 548
1595 nigel 63 LOCALE SUPPORT
1596 nigel 41
1597 ph10 579 PCRE handles caseless matching, and determines whether characters are
1598     letters, digits, or whatever, by reference to a set of tables, indexed
1599     by character value. When running in UTF-8 mode, this applies only to
1600     characters with codes less than 128. By default, higher-valued codes
1601 ph10 535 never match escapes such as \w or \d, but they can be tested with \p if
1602 ph10 579 PCRE is built with Unicode character property support. Alternatively,
1603     the PCRE_UCP option can be set at compile time; this causes \w and
1604 ph10 535 friends to use Unicode property support instead of built-in tables. The
1605     use of locales with Unicode is discouraged. If you are handling charac-
1606 ph10 579 ters with codes greater than 128, you should either use UTF-8 and Uni-
1607 ph10 535 code, or use locales, but not try to mix the two.
1608 nigel 41
1609 ph10 579 PCRE contains an internal set of tables that are used when the final
1610     argument of pcre_compile() is NULL. These are sufficient for many
1611 ph10 142 applications. Normally, the internal tables recognize only ASCII char-
1612     acters. However, when PCRE is built, it is possible to cause the inter-
1613     nal tables to be rebuilt in the default "C" locale of the local system,
1614     which may cause them to be different.
1615 nigel 41
1616 ph10 579 The internal tables can always be overridden by tables supplied by the
1617 ph10 142 application that calls PCRE. These may be created in a different locale
1618 ph10 579 from the default. As more and more applications change to using Uni-
1619 ph10 142 code, the need for this locale support is expected to die away.
1620    
1621 ph10 579 External tables are built by calling the pcre_maketables() function,
1622     which has no arguments, in the relevant locale. The result can then be
1623     passed to pcre_compile() or pcre_exec() as often as necessary. For
1624     example, to build and use tables that are appropriate for the French
1625     locale (where accented characters with values greater than 128 are
1626 nigel 75 treated as letters), the following code could be used:
1627    
1628     setlocale(LC_CTYPE, "fr_FR");
1629 nigel 73 tables = pcre_maketables();
1630     re = pcre_compile(..., tables);
1631 nigel 41
1632 ph10 579 The locale name "fr_FR" is used on Linux and other Unix-like systems;
1633 ph10 142 if you are using Windows, the name for the French locale is "french".
1634    
1635 ph10 579 When pcre_maketables() runs, the tables are built in memory that is
1636     obtained via pcre_malloc. It is the caller's responsibility to ensure
1637     that the memory containing the tables remains available for as long as
1638 nigel 75 it is needed.
1639 nigel 41
1640 nigel 75 The pointer that is passed to pcre_compile() is saved with the compiled
1641 ph10 579 pattern, and the same tables are used via this pointer by pcre_study()
1642 nigel 75 and normally also by pcre_exec(). Thus, by default, for any single pat-
1643     tern, compilation, studying and matching all happen in the same locale,
1644     but different patterns can be compiled in different locales.
1645 nigel 41
1646 ph10 579 It is possible to pass a table pointer or NULL (indicating the use of
1647     the internal tables) to pcre_exec(). Although not intended for this
1648     purpose, this facility could be used to match a pattern in a different
1649 nigel 75 locale from the one in which it was compiled. Passing table pointers at
1650     run time is discussed below in the section on matching a pattern.
1651    
1652    
1653 nigel 63 INFORMATION ABOUT A PATTERN
1654 nigel 41
1655 nigel 73 int pcre_fullinfo(const pcre *code, const pcre_extra *extra,
1656     int what, void *where);
1657 nigel 63
1658 ph10 579 The pcre_fullinfo() function returns information about a compiled pat-
1659 nigel 73 tern. It replaces the obsolete pcre_info() function, which is neverthe-
1660     less retained for backwards compability (and is documented below).
1661 nigel 43
1662 ph10 579 The first argument for pcre_fullinfo() is a pointer to the compiled
1663     pattern. The second argument is the result of pcre_study(), or NULL if
1664     the pattern was not studied. The third argument specifies which piece
1665     of information is required, and the fourth argument is a pointer to a
1666     variable to receive the data. The yield of the function is zero for
1667 nigel 73 success, or one of the following negative numbers:
1668 nigel 41
1669 nigel 73 PCRE_ERROR_NULL the argument code was NULL
1670     the argument where was NULL
1671     PCRE_ERROR_BADMAGIC the "magic number" was not found
1672     PCRE_ERROR_BADOPTION the value of what was invalid
1673 nigel 53
1674 ph10 579 The "magic number" is placed at the start of each compiled pattern as
1675     an simple check against passing an arbitrary memory pointer. Here is a
1676     typical call of pcre_fullinfo(), to obtain the length of the compiled
1677 nigel 75 pattern:
1678 nigel 53
1679 nigel 73 int rc;
1680 nigel 91 size_t length;
1681 nigel 73 rc = pcre_fullinfo(
1682     re, /* result of pcre_compile() */
1683     pe, /* result of pcre_study(), or NULL */
1684     PCRE_INFO_SIZE, /* what is required */
1685     &length); /* where to put the data */
1686 nigel 43
1687 ph10 579 The possible values for the third argument are defined in pcre.h, and
1688 nigel 73 are as follows:
1689 nigel 43
1690 nigel 73 PCRE_INFO_BACKREFMAX
1691 nigel 41
1692 ph10 579 Return the number of the highest back reference in the pattern. The
1693     fourth argument should point to an int variable. Zero is returned if
1694 nigel 73 there are no back references.
1695 nigel 43
1696 nigel 73 PCRE_INFO_CAPTURECOUNT
1697 nigel 43
1698 ph10 579 Return the number of capturing subpatterns in the pattern. The fourth
1699 nigel 73 argument should point to an int variable.
1700 nigel 43
1701 nigel 77 PCRE_INFO_DEFAULT_TABLES
1702 nigel 75
1703 ph10 579 Return a pointer to the internal default character tables within PCRE.
1704     The fourth argument should point to an unsigned char * variable. This
1705 nigel 75 information call is provided for internal use by the pcre_study() func-
1706 ph10 579 tion. External callers can cause PCRE to use its internal tables by
1707 nigel 75 passing a NULL table pointer.
1708    
1709 nigel 73 PCRE_INFO_FIRSTBYTE
1710 nigel 43
1711 ph10 579 Return information about the first byte of any matched string, for a
1712     non-anchored pattern. The fourth argument should point to an int vari-
1713     able. (This option used to be called PCRE_INFO_FIRSTCHAR; the old name
1714 nigel 91 is still recognized for backwards compatibility.)
1715 nigel 41
1716 ph10 579 If there is a fixed first byte, for example, from a pattern such as
1717 nigel 93 (cat|cow|coyote), its value is returned. Otherwise, if either
1718 nigel 41
1719 ph10 579 (a) the pattern was compiled with the PCRE_MULTILINE option, and every
1720 nigel 73 branch starts with "^", or
1721 nigel 43
1722 nigel 73 (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not
1723     set (if it were set, the pattern would be anchored),
1724 nigel 41
1725 ph10 579 -1 is returned, indicating that the pattern matches only at the start
1726     of a subject string or after any newline within the string. Otherwise
1727 nigel 73 -2 is returned. For anchored patterns, -2 is returned.
1728 nigel 41
1729 nigel 73 PCRE_INFO_FIRSTTABLE
1730 nigel 41
1731 ph10 579 If the pattern was studied, and this resulted in the construction of a
1732 nigel 73 256-bit table indicating a fixed set of bytes for the first byte in any
1733 ph10 579 matching string, a pointer to the table is returned. Otherwise NULL is
1734     returned. The fourth argument should point to an unsigned char * vari-
1735 nigel 73 able.
1736 nigel 43
1737 ph10 227 PCRE_INFO_HASCRORLF
1738    
1739 ph10 579 Return 1 if the pattern contains any explicit matches for CR or LF
1740     characters, otherwise 0. The fourth argument should point to an int
1741     variable. An explicit match is either a literal CR or LF character, or
1742 ph10 243 \r or \n.
1743 ph10 227
1744 ph10 172 PCRE_INFO_JCHANGED
1745    
1746 ph10 579 Return 1 if the (?J) or (?-J) option setting is used in the pattern,
1747     otherwise 0. The fourth argument should point to an int variable. (?J)
1748 ph10 286 and (?-J) set and unset the local PCRE_DUPNAMES option, respectively.
1749 ph10 172
1750 nigel 73 PCRE_INFO_LASTLITERAL
1751 nigel 43
1752 ph10 579 Return the value of the rightmost literal byte that must exist in any
1753     matched string, other than at its start, if such a byte has been
1754 nigel 73 recorded. The fourth argument should point to an int variable. If there
1755 ph10 579 is no such byte, -1 is returned. For anchored patterns, a last literal
1756     byte is recorded only if it follows something of variable length. For
1757 nigel 73 example, for the pattern /^a\d+z\d+/ the returned value is "z", but for
1758     /^a\dz\d/ the returned value is -1.
1759 nigel 63
1760 ph10 461 PCRE_INFO_MINLENGTH
1761    
1762 ph10 579 If the pattern was studied and a minimum length for matching subject
1763     strings was computed, its value is returned. Otherwise the returned
1764     value is -1. The value is a number of characters, not bytes (this may
1765     be relevant in UTF-8 mode). The fourth argument should point to an int
1766     variable. A non-negative value is a lower bound to the length of any
1767     matching string. There may not be any strings of that length that do
1768 ph10 461 actually match, but every string that does match is at least that long.
1769    
1770 nigel 73 PCRE_INFO_NAMECOUNT
1771     PCRE_INFO_NAMEENTRYSIZE
1772     PCRE_INFO_NAMETABLE
1773 nigel 63
1774 ph10 579 PCRE supports the use of named as well as numbered capturing parenthe-
1775     ses. The names are just an additional way of identifying the parenthe-
1776 nigel 91 ses, which still acquire numbers. Several convenience functions such as
1777 ph10 579 pcre_get_named_substring() are provided for extracting captured sub-
1778     strings by name. It is also possible to extract the data directly, by
1779     first converting the name to a number in order to access the correct
1780 nigel 91 pointers in the output vector (described with pcre_exec() below). To do
1781 ph10 579 the conversion, you need to use the name-to-number map, which is
1782 nigel 91 described by these three values.
1783 nigel 63
1784 nigel 73 The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT
1785     gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size
1786 ph10 579 of each entry; both of these return an int value. The entry size
1787     depends on the length of the longest name. PCRE_INFO_NAMETABLE returns
1788     a pointer to the first entry of the table (a pointer to char). The
1789 nigel 73 first two bytes of each entry are the number of the capturing parenthe-
1790 ph10 579 sis, most significant byte first. The rest of the entry is the corre-
1791 ph10 461 sponding name, zero terminated.
1792 nigel 63
1793 ph10 579 The names are in alphabetical order. Duplicate names may appear if (?|
1794 ph10 461 is used to create multiple groups with the same number, as described in
1795 ph10 579 the section on duplicate subpattern numbers in the pcrepattern page.
1796     Duplicate names for subpatterns with different numbers are permitted
1797     only if PCRE_DUPNAMES is set. In all cases of duplicate names, they
1798     appear in the table in the order in which they were found in the pat-
1799     tern. In the absence of (?| this is the order of increasing number;
1800 ph10 461 when (?| is used this is not necessarily the case because later subpat-
1801     terns may have lower numbers.
1802    
1803 ph10 579 As a simple example of the name/number table, consider the following
1804     pattern (assume PCRE_EXTENDED is set, so white space - including new-
1805 ph10 461 lines - is ignored):
1806    
1807 nigel 93 (?<date> (?<year>(\d\d)?\d\d) -
1808     (?<month>\d\d) - (?<day>\d\d) )
1809 nigel 63
1810 ph10 579 There are four named subpatterns, so the table has four entries, and
1811     each entry in the table is eight bytes long. The table is as follows,
1812 nigel 75 with non-printing bytes shows in hexadecimal, and undefined bytes shown
1813     as ??:
1814 nigel 63
1815 nigel 73 00 01 d a t e 00 ??
1816     00 05 d a y 00 ?? ??
1817     00 04 m o n t h 00
1818     00 02 y e a r 00 ??
1819 nigel 63
1820 ph10 579 When writing code to extract data from named subpatterns using the
1821     name-to-number map, remember that the length of the entries is likely
1822 nigel 91 to be different for each compiled pattern.
1823 nigel 63
1824 ph10 172 PCRE_INFO_OKPARTIAL
1825    
1826 ph10 579 Return 1 if the pattern can be used for partial matching with
1827     pcre_exec(), otherwise 0. The fourth argument should point to an int
1828     variable. From release 8.00, this always returns 1, because the
1829     restrictions that previously applied to partial matching have been
1830     lifted. The pcrepartial documentation gives details of partial match-
1831 ph10 453 ing.
1832 ph10 172
1833 nigel 73 PCRE_INFO_OPTIONS
1834 nigel 63
1835 ph10 579 Return a copy of the options with which the pattern was compiled. The
1836     fourth argument should point to an unsigned long int variable. These
1837 nigel 73 option bits are those specified in the call to pcre_compile(), modified
1838 ph10 197 by any top-level option settings at the start of the pattern itself. In
1839 ph10 579 other words, they are the options that will be in force when matching
1840     starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with
1841     the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE,
1842 ph10 197 and PCRE_EXTENDED.
1843 nigel 63
1844 ph10 579 A pattern is automatically anchored by PCRE if all of its top-level
1845 nigel 73 alternatives begin with one of the following:
1846 nigel 63
1847 nigel 73 ^ unless PCRE_MULTILINE is set
1848     \A always
1849     \G always
1850     .* if PCRE_DOTALL is set and there are no back
1851     references to the subpattern in which .* appears
1852 nigel 63
1853 nigel 73 For such patterns, the PCRE_ANCHORED bit is set in the options returned
1854     by pcre_fullinfo().
1855 nigel 63
1856 nigel 73 PCRE_INFO_SIZE
1857 nigel 63
1858 ph10 579 Return the size of the compiled pattern, that is, the value that was
1859 nigel 73 passed as the argument to pcre_malloc() when PCRE was getting memory in
1860     which to place the compiled data. The fourth argument should point to a
1861     size_t variable.
1862 nigel 63
1863 nigel 73 PCRE_INFO_STUDYSIZE
1864 nigel 63
1865 nigel 75 Return the size of the data block pointed to by the study_data field in
1866 ph10 579 a pcre_extra block. That is, it is the value that was passed to
1867 nigel 73 pcre_malloc() when PCRE was getting memory into which to place the data
1868 ph10 579 created by pcre_study(). If pcre_extra is NULL, or there is no study
1869     data, zero is returned. The fourth argument should point to a size_t
1870 nigel 73 variable.
1871 nigel 63
1872 nigel 73
1873 nigel 63 OBSOLETE INFO FUNCTION
1874    
1875 nigel 73 int pcre_info(const pcre *code, int *optptr, int *firstcharptr);
1876 nigel 63
1877 ph10 579 The pcre_info() function is now obsolete because its interface is too
1878     restrictive to return all the available data about a compiled pattern.
1879     New programs should use pcre_fullinfo() instead. The yield of
1880     pcre_info() is the number of capturing subpatterns, or one of the fol-
1881 nigel 73 lowing negative numbers:
1882 nigel 43
1883 nigel 73 PCRE_ERROR_NULL the argument code was NULL
1884     PCRE_ERROR_BADMAGIC the "magic number" was not found
1885 nigel 43
1886 ph10 579 If the optptr argument is not NULL, a copy of the options with which
1887     the pattern was compiled is placed in the integer it points to (see
1888 nigel 73 PCRE_INFO_OPTIONS above).
1889 nigel 43
1890 ph10 579 If the pattern is not anchored and the firstcharptr argument is not
1891     NULL, it is used to pass back information about the first character of
1892 nigel 73 any matched string (see PCRE_INFO_FIRSTBYTE above).
1893 nigel 43
1894    
1895 nigel 77 REFERENCE COUNTS
1896 nigel 53
1897 nigel 77 int pcre_refcount(pcre *code, int adjust);
1898    
1899 ph10 579 The pcre_refcount() function is used to maintain a reference count in
1900 nigel 77 the data block that contains a compiled pattern. It is provided for the
1901 ph10 579 benefit of applications that operate in an object-oriented manner,
1902 nigel 77 where different parts of the application may be using the same compiled
1903     pattern, but you want to free the block when they are all done.
1904    
1905     When a pattern is compiled, the reference count field is initialized to
1906 ph10 579 zero. It is changed only by calling this function, whose action is to
1907     add the adjust value (which may be positive or negative) to it. The
1908 nigel 77 yield of the function is the new value. However, the value of the count
1909 ph10 579 is constrained to lie between 0 and 65535, inclusive. If the new value
1910 nigel 77 is outside these limits, it is forced to the appropriate limit value.
1911    
1912 ph10 579 Except when it is zero, the reference count is not correctly preserved
1913     if a pattern is compiled on one host and then transferred to a host
1914 nigel 77 whose byte-order is different. (This seems a highly unlikely scenario.)
1915    
1916    
1917     MATCHING A PATTERN: THE TRADITIONAL FUNCTION
1918    
1919 nigel 73 int pcre_exec(const pcre *code, const pcre_extra *extra,
1920     const char *subject, int length, int startoffset,
1921     int options, int *ovector, int ovecsize);
1922 nigel 53
1923 ph10 579 The function pcre_exec() is called to match a subject string against a
1924     compiled pattern, which is passed in the code argument. If the pattern
1925     was studied, the result of the study should be passed in the extra
1926     argument. This function is the main matching facility of the library,
1927 nigel 77 and it operates in a Perl-like manner. For specialist use there is also
1928 ph10 579 an alternative matching function, which is described below in the sec-
1929 nigel 77 tion about the pcre_dfa_exec() function.
1930 nigel 41
1931 ph10 579 In most applications, the pattern will have been compiled (and option-
1932     ally studied) in the same process that calls pcre_exec(). However, it
1933 nigel 75 is possible to save compiled patterns and study data, and then use them
1934 ph10 579 later in different processes, possibly even on different hosts. For a
1935 nigel 75 discussion about this, see the pcreprecompile documentation.
1936    
1937 nigel 73 Here is an example of a simple call to pcre_exec():
1938 nigel 53
1939 nigel 73 int rc;
1940     int ovector[30];
1941     rc = pcre_exec(
1942     re, /* result of pcre_compile() */
1943     NULL, /* we didn't study the pattern */
1944     "some string", /* the subject string */
1945     11, /* the length of the subject string */
1946     0, /* start at offset 0 in the subject */
1947     0, /* default options */
1948 nigel 75 ovector, /* vector of integers for substring information */
1949 nigel 77 30); /* number of elements (NOT size in bytes) */
1950 nigel 53
1951 nigel 75 Extra data for pcre_exec()
1952 nigel 63
1953 ph10 579 If the extra argument is not NULL, it must point to a pcre_extra data
1954     block. The pcre_study() function returns such a block (when it doesn't
1955     return NULL), but you can also create one for yourself, and pass addi-
1956     tional information in it. The pcre_extra block contains the following
1957 nigel 87 fields (not necessarily in this order):
1958 nigel 75
1959 nigel 73 unsigned long int flags;
1960     void *study_data;
1961     unsigned long int match_limit;
1962 nigel 87 unsigned long int match_limit_recursion;
1963 nigel 73 void *callout_data;
1964 nigel 75 const unsigned char *tables;
1965 ph10 512 unsigned char **mark;
1966 nigel 63
1967 ph10 579 The flags field is a bitmap that specifies which of the other fields
1968 nigel 73 are set. The flag bits are:
1969 nigel 63
1970 nigel 73 PCRE_EXTRA_STUDY_DATA
1971     PCRE_EXTRA_MATCH_LIMIT
1972 nigel 87 PCRE_EXTRA_MATCH_LIMIT_RECURSION
1973 nigel 73 PCRE_EXTRA_CALLOUT_DATA
1974 nigel 75 PCRE_EXTRA_TABLES
1975 ph10 512 PCRE_EXTRA_MARK
1976 nigel 63
1977 ph10 579 Other flag bits should be set to zero. The study_data field is set in
1978     the pcre_extra block that is returned by pcre_study(), together with
1979 nigel 75 the appropriate flag bit. You should not set this yourself, but you may
1980 ph10 579 add to the block by setting the other fields and their corresponding
1981 nigel 75 flag bits.
1982 nigel 63
1983 nigel 73 The match_limit field provides a means of preventing PCRE from using up
1984 ph10 579 a vast amount of resources when running patterns that are not going to
1985     match, but which have a very large number of possibilities in their
1986     search trees. The classic example is a pattern that uses nested unlim-
1987 ph10 461 ited repeats.
1988 nigel 63
1989 ph10 579 Internally, PCRE uses a function called match() which it calls repeat-
1990     edly (sometimes recursively). The limit set by match_limit is imposed
1991     on the number of times this function is called during a match, which
1992     has the effect of limiting the amount of backtracking that can take
1993 nigel 87 place. For patterns that are not anchored, the count restarts from zero
1994     for each position in the subject string.
1995 nigel 75
1996 ph10 579 The default value for the limit can be set when PCRE is built; the
1997     default default is 10 million, which handles all but the most extreme
1998     cases. You can override the default by suppling pcre_exec() with a
1999     pcre_extra block in which match_limit is set, and
2000     PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is
2001 nigel 73 exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT.
2002 nigel 63
2003 ph10 579 The match_limit_recursion field is similar to match_limit, but instead
2004 nigel 87 of limiting the total number of times that match() is called, it limits
2005 ph10 579 the depth of recursion. The recursion depth is a smaller number than
2006     the total number of calls, because not all calls to match() are recur-
2007 nigel 87 sive. This limit is of use only if it is set smaller than match_limit.
2008    
2009 ph10 579 Limiting the recursion depth limits the amount of stack that can be
2010 nigel 87 used, or, when PCRE has been compiled to use memory on the heap instead
2011     of the stack, the amount of heap memory that can be used.
2012    
2013 ph10 579 The default value for match_limit_recursion can be set when PCRE is
2014     built; the default default is the same value as the default for
2015     match_limit. You can override the default by suppling pcre_exec() with
2016     a pcre_extra block in which match_limit_recursion is set, and
2017     PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the
2018 nigel 87 limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT.
2019    
2020 ph10 579 The callout_data field is used in conjunction with the "callout" fea-
2021 ph10 453 ture, and is described in the pcrecallout documentation.
2022 nigel 63
2023 ph10 579 The tables field is used to pass a character tables pointer to
2024     pcre_exec(); this overrides the value that is stored with the compiled
2025     pattern. A non-NULL value is stored with the compiled pattern only if
2026     custom tables were supplied to pcre_compile() via its tableptr argu-
2027 nigel 75 ment. If NULL is passed to pcre_exec() using this mechanism, it forces
2028 ph10 579 PCRE's internal tables to be used. This facility is helpful when re-
2029     using patterns that have been saved after compiling with an external
2030     set of tables, because the external tables might be at a different
2031     address when pcre_exec() is called. See the pcreprecompile documenta-
2032 nigel 75 tion for a discussion of saving compiled patterns for later use.
2033 nigel 41
2034 ph10 579 If PCRE_EXTRA_MARK is set in the flags field, the mark field must be
2035     set to point to a char * variable. If the pattern contains any back-
2036     tracking control verbs such as (*MARK:NAME), and the execution ends up
2037     with a name to pass back, a pointer to the name string (zero termi-
2038     nated) is placed in the variable pointed to by the mark field. The
2039     names are within the compiled pattern; if you wish to retain such a
2040     name you must copy it before freeing the memory of a compiled pattern.
2041     If there is no name to pass back, the variable pointed to by the mark
2042     field set to NULL. For details of the backtracking control verbs, see
2043 ph10 512 the section entitled "Backtracking control" in the pcrepattern documen-
2044     tation.
2045    
2046 nigel 75 Option bits for pcre_exec()
2047 nigel 71
2048 ph10 579 The unused bits of the options argument for pcre_exec() must be zero.
2049     The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx,
2050     PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
2051     PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and
2052 ph10 453 PCRE_PARTIAL_HARD.
2053 nigel 41
2054 nigel 75 PCRE_ANCHORED
2055 nigel 41
2056 ph10 579 The PCRE_ANCHORED option limits pcre_exec() to matching at the first
2057     matching position. If a pattern was compiled with PCRE_ANCHORED, or
2058     turned out to be anchored by virtue of its contents, it cannot be made
2059 nigel 75 unachored at matching time.
2060    
2061 ph10 231 PCRE_BSR_ANYCRLF
2062     PCRE_BSR_UNICODE
2063    
2064     These options (which are mutually exclusive) control what the \R escape
2065 ph10 579 sequence matches. The choice is either to match only CR, LF, or CRLF,
2066     or to match any Unicode newline sequence. These options override the
2067 ph10 231 choice that was made or defaulted when the pattern was compiled.
2068    
2069 nigel 91 PCRE_NEWLINE_CR
2070     PCRE_NEWLINE_LF
2071     PCRE_NEWLINE_CRLF
2072 ph10 150 PCRE_NEWLINE_ANYCRLF
2073 nigel 93 PCRE_NEWLINE_ANY
2074 nigel 91
2075 ph10 579 These options override the newline definition that was chosen or
2076     defaulted when the pattern was compiled. For details, see the descrip-
2077     tion of pcre_compile() above. During matching, the newline choice
2078     affects the behaviour of the dot, circumflex, and dollar metacharac-
2079     ters. It may also alter the way the match position is advanced after a
2080 ph10 227 match failure for an unanchored pattern.
2081 nigel 91
2082 ph10 579 When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is
2083     set, and a match attempt for an unanchored pattern fails when the cur-
2084     rent position is at a CRLF sequence, and the pattern contains no
2085     explicit matches for CR or LF characters, the match position is
2086 ph10 227 advanced by two characters instead of one, in other words, to after the
2087     CRLF.
2088    
2089     The above rule is a compromise that makes the most common cases work as
2090 ph10 579 expected. For example, if the pattern is .+A (and the PCRE_DOTALL
2091 ph10 227 option is not set), it does not match the string "\r\nA" because, after
2092 ph10 579 failing at the start, it skips both the CR and the LF before retrying.
2093     However, the pattern [\r\n]A does match that string, because it con-
2094 ph10 227 tains an explicit CR or LF reference, and so advances only by one char-
2095 ph10 231 acter after the first failure.
2096 ph10 227
2097 ph10 231 An explicit match for CR of LF is either a literal appearance of one of
2098 ph10 579 those characters, or one of the \r or \n escape sequences. Implicit
2099     matches such as [^X] do not count, nor does \s (which includes CR and
2100 ph10 231 LF in the characters that it matches).
2101    
2102 ph10 579 Notwithstanding the above, anomalous effects may still occur when CRLF
2103 ph10 227 is a valid newline sequence and explicit \r or \n escapes appear in the
2104     pattern.
2105    
2106 nigel 73 PCRE_NOTBOL
2107 nigel 41
2108 nigel 75 This option specifies that first character of the subject string is not
2109 ph10 579 the beginning of a line, so the circumflex metacharacter should not
2110     match before it. Setting this without PCRE_MULTILINE (at compile time)
2111     causes circumflex never to match. This option affects only the behav-
2112 nigel 77 iour of the circumflex metacharacter. It does not affect \A.
2113 nigel 41
2114 nigel 73 PCRE_NOTEOL
2115 nigel 41
2116 nigel 75 This option specifies that the end of the subject string is not the end
2117 ph10 579 of a line, so the dollar metacharacter should not match it nor (except
2118     in multiline mode) a newline immediately before it. Setting this with-
2119 nigel 75 out PCRE_MULTILINE (at compile time) causes dollar never to match. This
2120 ph10 579 option affects only the behaviour of the dollar metacharacter. It does
2121 nigel 75 not affect \Z or \z.
2122 nigel 41
2123 nigel 73 PCRE_NOTEMPTY
2124 nigel 41
2125 nigel 73 An empty string is not considered to be a valid match if this option is
2126 ph10 579 set. If there are alternatives in the pattern, they are tried. If all
2127     the alternatives match the empty string, the entire match fails. For
2128 nigel 73 example, if the pattern
2129 nigel 41
2130 nigel 73 a?b?
2131 nigel 41
2132 ph10 579 is applied to a string not beginning with "a" or "b", it matches an
2133     empty string at the start of the subject. With PCRE_NOTEMPTY set, this
2134 nigel 73 match is not valid, so PCRE searches further into the string for occur-
2135     rences of "a" or "b".
2136 nigel 41
2137 ph10 453 PCRE_NOTEMPTY_ATSTART
2138 nigel 41
2139 ph10 579 This is like PCRE_NOTEMPTY, except that an empty string match that is
2140     not at the start of the subject is permitted. If the pattern is
2141 ph10 453 anchored, such a match can occur only if the pattern contains \K.
2142    
2143 ph10 579 Perl has no direct equivalent of PCRE_NOTEMPTY or
2144     PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern
2145     match of the empty string within its split() function, and when using
2146     the /g modifier. It is possible to emulate Perl's behaviour after
2147 ph10 453 matching a null string by first trying the match again at the same off-
2148 ph10 579 set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that
2149 ph10 453 fails, by advancing the starting offset (see below) and trying an ordi-
2150 ph10 579 nary match again. There is some code that demonstrates how to do this
2151     in the pcredemo sample program. In the most general case, you have to
2152     check to see if the newline convention recognizes CRLF as a newline,
2153     and if so, and the current character is CR followed by LF, advance the
2154 ph10 567 starting offset by two characters instead of one.
2155 ph10 453
2156 ph10 392 PCRE_NO_START_OPTIMIZE
2157    
2158 ph10 579 There are a number of optimizations that pcre_exec() uses at the start
2159     of a match, in order to speed up the process. For example, if it is
2160 ph10 545 known that an unanchored match must start with a specific character, it
2161 ph10 579 searches the subject for that character, and fails immediately if it
2162     cannot find it, without actually running the main matching function.
2163 ph10 545 This means that a special item such as (*COMMIT) at the start of a pat-
2164 ph10 579 tern is not considered until after a suitable starting point for the
2165     match has been found. When callouts or (*MARK) items are in use, these
2166 ph10 548 "start-up" optimizations can cause them to be skipped if the pattern is
2167 ph10 579 never actually used. The start-up optimizations are in effect a pre-
2168 ph10 548 scan of the subject that takes place before the pattern is run.
2169 ph10 392
2170 ph10 579 The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations,
2171     possibly causing performance to suffer, but ensuring that in cases
2172     where the result is "no match", the callouts do occur, and that items
2173 ph10 548 such as (*COMMIT) and (*MARK) are considered at every possible starting
2174 ph10 579 position in the subject string. If PCRE_NO_START_OPTIMIZE is set at
2175     compile time, it cannot be unset at matching time.
2176 ph10 548
2177 ph10 579 Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching
2178     operation. Consider the pattern
2179    
2180 ph10 548 (*COMMIT)ABC
2181    
2182 ph10 572 When this is compiled, PCRE records the fact that a match must start
2183     with the character "A". Suppose the subject string is "DEFABC". The
2184     start-up optimization scans along the subject, finds "A" and runs the
2185     first match attempt from there. The (*COMMIT) item means that the pat-
2186     tern must match the current starting position, which in this case, it
2187     does. However, if the same match is run with PCRE_NO_START_OPTIMIZE
2188     set, the initial scan along the subject string does not happen. The
2189     first match attempt is run starting from "D" and when this fails,
2190     (*COMMIT) prevents any further matches being tried, so the overall
2191     result is "no match". If the pattern is studied, more start-up opti-
2192     mizations may be used. For example, a minimum length for the subject
2193 ph10 548 may be recorded. Consider the pattern
2194    
2195     (*MARK:A)(X|Y)
2196    
2197 ph10 572 The minimum length for a match is one character. If the subject is
2198     "ABC", there will be attempts to match "ABC", "BC", "C", and then
2199     finally an empty string. If the pattern is studied, the final attempt
2200     does not take place, because PCRE knows that the subject is too short,
2201     and so the (*MARK) is never encountered. In this case, studying the
2202     pattern does not affect the overall match result, which is still "no
2203 ph10 548 match", but it does affect the auxiliary information that is returned.
2204    
2205 nigel 75 PCRE_NO_UTF8_CHECK
2206    
2207     When PCRE_UTF8 is set at compile time, the validity of the subject as a
2208 ph10 572 UTF-8 string is automatically checked when pcre_exec() is subsequently
2209     called. The value of startoffset is also checked to ensure that it
2210     points to the start of a UTF-8 character. There is a discussion about
2211     the validity of UTF-8 strings in the section on UTF-8 support in the
2212     main pcre page. If an invalid UTF-8 sequence of bytes is found,
2213     pcre_exec() returns the error PCRE_ERROR_BADUTF8 or, if PCRE_PAR-
2214     TIAL_HARD is set and the problem is a truncated UTF-8 character at the
2215     end of the subject, PCRE_ERROR_SHORTUTF8. If startoffset contains a
2216     value that does not point to the start of a UTF-8 character (or to the
2217     end of the subject), PCRE_ERROR_BADUTF8_OFFSET is returned.
2218 nigel 75
2219 ph10 572 If you already know that your subject is valid, and you want to skip
2220     these checks for performance reasons, you can set the
2221     PCRE_NO_UTF8_CHECK option when calling pcre_exec(). You might want to
2222     do this for the second and subsequent calls to pcre_exec() if you are
2223     making repeated calls to find all the matches in a single subject
2224     string. However, you should be sure that the value of startoffset
2225     points to the start of a UTF-8 character (or the end of the subject).
2226     When PCRE_NO_UTF8_CHECK is set, the effect of passing an invalid UTF-8
2227     string as a subject or an invalid value of startoffset is undefined.
2228 ph10 567 Your program may crash.
2229 nigel 75
2230 ph10 429 PCRE_PARTIAL_HARD
2231     PCRE_PARTIAL_SOFT
2232 nigel 75
2233 ph10 572 These options turn on the partial matching feature. For backwards com-
2234     patibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial
2235     match occurs if the end of the subject string is reached successfully,
2236     but there are not enough subject characters to complete the match. If
2237 ph10 567 this happens when PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set,
2238 ph10 572 matching continues by testing any remaining alternatives. Only if no
2239     complete match can be found is PCRE_ERROR_PARTIAL returned instead of
2240     PCRE_ERROR_NOMATCH. In other words, PCRE_PARTIAL_SOFT says that the
2241     caller is prepared to handle a partial match, but only if no complete
2242 ph10 567 match can be found.
2243 nigel 75
2244 ph10 572 If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this
2245     case, if a partial match is found, pcre_exec() immediately returns
2246     PCRE_ERROR_PARTIAL, without considering any other alternatives. In
2247     other words, when PCRE_PARTIAL_HARD is set, a partial match is consid-
2248 ph10 567 ered to be more important that an alternative complete match.
2249    
2250 ph10 572 In both cases, the portion of the string that was inspected when the
2251 ph10 567 partial match was found is set as the first matching string. There is a
2252 ph10 572 more detailed discussion of partial and multi-segment matching, with
2253 ph10 567 examples, in the pcrepartial documentation.
2254    
2255 nigel 75 The string to be matched by pcre_exec()
2256    
2257 ph10 572 The subject string is passed to pcre_exec() as a pointer in subject, a
2258 ph10 371 length (in bytes) in length, and a starting byte offset in startoffset.
2259 ph10 572 If this is negative or greater than the length of the subject,
2260     pcre_exec() returns PCRE_ERROR_BADOFFSET. When the starting offset is
2261     zero, the search for a match starts at the beginning of the subject,
2262     and this is by far the most common case. In UTF-8 mode, the byte offset
2263     must point to the start of a UTF-8 character (or the end of the sub-
2264     ject). Unlike the pattern string, the subject may contain binary zero
2265     bytes.
2266 ph10 567
2267     A non-zero starting offset is useful when searching for another match
2268     in the same subject by calling pcre_exec() again after a previous suc-
2269     cess. Setting startoffset differs from just passing over a shortened
2270     string and setting PCRE_NOTBOL in the case of a pattern that begins
2271 nigel 73 with any kind of lookbehind. For example, consider the pattern
2272 nigel 41
2273 nigel 73 \Biss\B
2274 nigel 41
2275 ph10 567 which finds occurrences of "iss" in the middle of words. (\B matches
2276     only if the current position in the subject is not a word boundary.)
2277     When applied to the string "Mississipi" the first call to pcre_exec()
2278     finds the first occurrence. If pcre_exec() is called again with just
2279     the remainder of the subject, namely "issipi", it does not match,
2280 nigel 73 because \B is always false at the start of the subject, which is deemed
2281 ph10 567 to be a word boundary. However, if pcre_exec() is passed the entire
2282 nigel 75 string again, but with startoffset set to 4, it finds the second occur-
2283 ph10 567 rence of "iss" because it is able to look behind the starting point to
2284 nigel 75 discover that it is preceded by a letter.
2285 nigel 41
2286 ph10 567 Finding all the matches in a subject is tricky when the pattern can
2287     match an empty string. It is possible to emulate Perl's /g behaviour by
2288     first trying the match again at the same offset, with the
2289     PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED options, and then if that
2290     fails, advancing the starting offset and trying an ordinary match
2291     again. There is some code that demonstrates how to do this in the pcre-
2292     demo sample program. In the most general case, you have to check to see
2293     if the newline convention recognizes CRLF as a newline, and if so, and
2294     the current character is CR followed by LF, advance the starting offset
2295     by two characters instead of one.
2296    
2297 ph10 548 If a non-zero starting offset is passed when the pattern is anchored,
2298 nigel 75 one attempt to match at the given offset is made. This can only succeed
2299 ph10 548 if the pattern does not require the match to be at the start of the
2300 nigel 75 subject.
2301 nigel 41
2302 nigel 75 How pcre_exec() returns captured substrings
2303    
2304 ph10 548 In general, a pattern matches a certain portion of the subject, and in
2305     addition, further substrings from the subject may be picked out by
2306     parts of the pattern. Following the usage in Jeffrey Friedl's book,
2307     this is called "capturing" in what follows, and the phrase "capturing
2308     subpattern" is used for a fragment of a pattern that picks out a sub-
2309     string. PCRE supports several other kinds of parenthesized subpattern
2310 nigel 73 that do not cause substrings to be captured.
2311 nigel 65
2312 ph10 371 Captured substrings are returned to the caller via a vector of integers
2313 ph10 548 whose address is passed in ovector. The number of elements in the vec-
2314     tor is passed in ovecsize, which must be a non-negative number. Note:
2315 ph10 371 this argument is NOT the size of ovector in bytes.
2316 nigel 41
2317 ph10 548 The first two-thirds of the vector is used to pass back captured sub-
2318     strings, each substring using a pair of integers. The remaining third
2319     of the vector is used as workspace by pcre_exec() while matching cap-
2320     turing subpatterns, and is not available for passing back information.
2321     The number passed in ovecsize should always be a multiple of three. If
2322 nigel 75 it is not, it is rounded down.
2323    
2324 ph10 548 When a match is successful, information about captured substrings is
2325     returned in pairs of integers, starting at the beginning of ovector,
2326     and continuing up to two-thirds of its length at the most. The first
2327     element of each pair is set to the byte offset of the first character
2328     in a substring, and the second is set to the byte offset of the first
2329     character after the end of a substring. Note: these values are always
2330 ph10 371 byte offsets, even in UTF-8 mode. They are not character counts.
2331 nigel 41
2332 ph10 548 The first pair of integers, ovector[0] and ovector[1], identify the
2333     portion of the subject string matched by the entire pattern. The next
2334     pair is used for the first capturing subpattern, and so on. The value
2335 ph10 371 returned by pcre_exec() is one more than the highest numbered pair that
2336 ph10 548 has been set. For example, if two substrings have been captured, the
2337     returned value is 3. If there are no capturing subpatterns, the return
2338 ph10 371 value from a successful match is 1, indicating that just the first pair
2339     of offsets has been set.
2340    
2341 nigel 73 If a capturing subpattern is matched repeatedly, it is the last portion
2342 nigel 75 of the string that it matched that is returned.
2343 nigel 41
2344 ph10 548 If the vector is too small to hold all the captured substring offsets,
2345 nigel 75 it is used as far as possible (up to two-thirds of its length), and the
2346 ph10 548 function returns a value of zero. If the substring offsets are not of
2347     interest, pcre_exec() may be called with ovector passed as NULL and
2348     ovecsize as zero. However, if the pattern contains back references and
2349     the ovector is not big enough to remember the related substrings, PCRE
2350     has to get additional memory for use during matching. Thus it is usu-
2351 ph10 371 ally advisable to supply an ovector.
2352 nigel 41
2353 ph10 461 The pcre_fullinfo() function can be used to find out how many capturing
2354 ph10 548 subpatterns there are in a compiled pattern. The smallest size for
2355     ovector that will allow for n captured substrings, in addition to the
2356 nigel 91 offsets of the substring matched by the whole pattern, is (n+1)*3.
2357 nigel 41
2358 ph10 548 It is possible for capturing subpattern number n+1 to match some part
2359 nigel 91 of the subject when subpattern n has not been used at all. For example,
2360 ph10 548 if the string "abc" is matched against the pattern (a|(z))(bc) the
2361 nigel 91 return from the function is 4, and subpatterns 1 and 3 are matched, but
2362 ph10 548 2 is not. When this happens, both values in the offset pairs corre-
2363 nigel 91 sponding to unused subpatterns are set to -1.
2364 nigel 75
2365 ph10 548 Offset values that correspond to unused subpatterns at the end of the
2366     expression are also set to -1. For example, if the string "abc" is
2367     matched against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not
2368     matched. The return from the function is 2, because the highest used
2369 ph10 572 capturing subpattern number is 1, and the offsets for for the second
2370     and third capturing subpatterns (assuming the vector is large enough,
2371     of course) are set to -1.
2372 nigel 91
2373 ph10 572 Note: Elements of ovector that do not correspond to capturing parenthe-
2374     ses in the pattern are never changed. That is, if a pattern contains n
2375     capturing parentheses, no more than ovector[0] to ovector[2n+1] are set
2376     by pcre_exec(). The other elements retain whatever values they previ-
2377     ously had.
2378    
2379 ph10 548 Some convenience functions are provided for extracting the captured
2380 nigel 91 substrings as separate strings. These are described below.
2381    
2382     Error return values from pcre_exec()
2383    
2384 ph10 548 If pcre_exec() fails, it returns a negative number. The following are
2385 nigel 73 defined in the header file:
2386 nigel 41
2387 nigel 73 PCRE_ERROR_NOMATCH (-1)
2388 nigel 41
2389 nigel 73 The subject string did not match the pattern.
2390 nigel 41
2391 nigel 73 PCRE_ERROR_NULL (-2)
2392 nigel 41
2393 ph10 548 Either code or subject was passed as NULL, or ovector was NULL and
2394 nigel 73 ovecsize was not zero.
2395 nigel 41
2396 nigel 73 PCRE_ERROR_BADOPTION (-3)
2397 nigel 41
2398 nigel 73 An unrecognized bit was set in the options argument.
2399 nigel 41
2400 nigel 73 PCRE_ERROR_BADMAGIC (-4)
2401 nigel 41
2402 ph10 548 PCRE stores a 4-byte "magic number" at the start of the compiled code,
2403 nigel 75 to catch the case when it is passed a junk pointer and to detect when a
2404     pattern that was compiled in an environment of one endianness is run in
2405 ph10 548 an environment with the other endianness. This is the error that PCRE
2406 nigel 75 gives when the magic number is not present.
2407 nigel 41
2408 nigel 93 PCRE_ERROR_UNKNOWN_OPCODE (-5)
2409 nigel 41
2410 nigel 73 While running the pattern match, an unknown item was encountered in the
2411 ph10 548 compiled pattern. This error could be caused by a bug in PCRE or by
2412 nigel 73 overwriting of the compiled pattern.
2413 nigel 41
2414 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2415 nigel 41
2416 ph10 548 If a pattern contains back references, but the ovector that is passed
2417 nigel 73 to pcre_exec() is not big enough to remember the referenced substrings,
2418 ph10 548 PCRE gets a block of memory at the start of matching to use for this
2419     purpose. If the call via pcre_malloc() fails, this error is given. The
2420 nigel 75 memory is automatically freed at the end of matching.
2421 nigel 41
2422 ph10 548 This error is also given if pcre_stack_malloc() fails in pcre_exec().
2423     This can happen only when PCRE has been compiled with --disable-stack-
2424 ph10 535 for-recursion.
2425    
2426 nigel 73 PCRE_ERROR_NOSUBSTRING (-7)
2427 nigel 53
2428 ph10 548 This error is used by the pcre_copy_substring(), pcre_get_substring(),
2429 nigel 73 and pcre_get_substring_list() functions (see below). It is never
2430     returned by pcre_exec().
2431 nigel 63
2432 nigel 73 PCRE_ERROR_MATCHLIMIT (-8)
2433 nigel 63
2434 ph10 548 The backtracking limit, as specified by the match_limit field in a
2435     pcre_extra structure (or defaulted) was reached. See the description
2436 nigel 87 above.
2437    
2438 nigel 73 PCRE_ERROR_CALLOUT (-9)
2439 nigel 63
2440 nigel 73 This error is never generated by pcre_exec() itself. It is provided for
2441 ph10 548 use by callout functions that want to yield a distinctive error code.
2442 nigel 73 See the pcrecallout documentation for details.
2443 nigel 71
2444 nigel 73 PCRE_ERROR_BADUTF8 (-10)
2445 nigel 71
2446 ph10 548 A string that contains an invalid UTF-8 byte sequence was passed as a
2447 ph10 572 subject. However, if PCRE_PARTIAL_HARD is set and the problem is a
2448     truncated UTF-8 character at the end of the subject, PCRE_ERROR_SHORT-
2449     UTF8 is used instead.
2450 nigel 73
2451     PCRE_ERROR_BADUTF8_OFFSET (-11)
2452    
2453     The UTF-8 byte sequence that was passed as a subject was valid, but the
2454 ph10 548 value of startoffset did not point to the beginning of a UTF-8 charac-
2455 ph10 572 ter or the end of the subject.
2456 nigel 73
2457 nigel 77 PCRE_ERROR_PARTIAL (-12)
2458 nigel 73
2459 ph10 548 The subject string did not match, but it did match partially. See the
2460 nigel 75 pcrepartial documentation for details of partial matching.
2461    
2462 nigel 77 PCRE_ERROR_BADPARTIAL (-13)
2463 nigel 75
2464 ph10 548 This code is no longer in use. It was formerly returned when the
2465     PCRE_PARTIAL option was used with a compiled pattern containing items
2466     that were not supported for partial matching. From release 8.00
2467 ph10 429 onwards, there are no restrictions on partial matching.
2468 nigel 75
2469 nigel 77 PCRE_ERROR_INTERNAL (-14)
2470 nigel 75
2471 ph10 548 An unexpected internal error has occurred. This error could be caused
2472 nigel 75 by a bug in PCRE or by overwriting of the compiled pattern.
2473    
2474 nigel 77 PCRE_ERROR_BADCOUNT (-15)
2475 nigel 75
2476 ph10 392 This error is given if the value of the ovecsize argument is negative.
2477 nigel 75
2478 nigel 93 PCRE_ERROR_RECURSIONLIMIT (-21)
2479 nigel 75
2480 nigel 93 The internal recursion limit, as specified by the match_limit_recursion
2481 ph10 548 field in a pcre_extra structure (or defaulted) was reached. See the
2482 nigel 93 description above.
2483    
2484     PCRE_ERROR_BADNEWLINE (-23)
2485    
2486     An invalid combination of PCRE_NEWLINE_xxx options was given.
2487    
2488 ph10 567 PCRE_ERROR_BADOFFSET (-24)
2489    
2490     The value of startoffset was negative or greater than the length of the
2491     subject, that is, the value in length.
2492    
2493 ph10 572 PCRE_ERROR_SHORTUTF8 (-25)
2494    
2495     The subject string ended with an incomplete (truncated) UTF-8 charac-
2496     ter, and the PCRE_PARTIAL_HARD option was set. Without this option,
2497     PCRE_ERROR_BADUTF8 is returned in this situation.
2498    
2499 ph10 197 Error numbers -16 to -20 and -22 are not used by pcre_exec().
2500 nigel 93
2501    
2502 nigel 63 EXTRACTING CAPTURED SUBSTRINGS BY NUMBER
2503    
2504 nigel 73 int pcre_copy_substring(const char *subject, int *ovector,
2505     int stringcount, int stringnumber, char *buffer,
2506     int buffersize);
2507 nigel 63
2508 nigel 73 int pcre_get_substring(const char *subject, int *ovector,
2509     int stringcount, int stringnumber,
2510     const char **stringptr);
2511 nigel 63
2512 nigel 73 int pcre_get_substring_list(const char *subject,
2513     int *ovector, int stringcount, const char ***listptr);
2514 nigel 63
2515 ph10 567 Captured substrings can be accessed directly by using the offsets
2516     returned by pcre_exec() in ovector. For convenience, the functions
2517 nigel 73 pcre_copy_substring(), pcre_get_substring(), and pcre_get_sub-
2518 ph10 567 string_list() are provided for extracting captured substrings as new,
2519     separate, zero-terminated strings. These functions identify substrings
2520     by number. The next section describes functions for extracting named
2521 nigel 91 substrings.
2522 nigel 41
2523 ph10 567 A substring that contains a binary zero is correctly extracted and has
2524     a further zero added on the end, but the result is not, of course, a C
2525     string. However, you can process such a string by referring to the
2526     length that is returned by pcre_copy_substring() and pcre_get_sub-
2527 nigel 91 string(). Unfortunately, the interface to pcre_get_substring_list() is
2528 ph10 567 not adequate for handling strings containing binary zeros, because the
2529 nigel 91 end of the final string is not independently indicated.
2530    
2531 ph10 567 The first three arguments are the same for all three of these func-
2532     tions: subject is the subject string that has just been successfully
2533 nigel 73 matched, ovector is a pointer to the vector of integer offsets that was
2534     passed to pcre_exec(), and stringcount is the number of substrings that
2535 ph10 567 were captured by the match, including the substring that matched the
2536 nigel 75 entire regular expression. This is the value returned by pcre_exec() if
2537 ph10 567 it is greater than zero. If pcre_exec() returned zero, indicating that
2538     it ran out of space in ovector, the value passed as stringcount should
2539 nigel 75 be the number of elements in the vector divided by three.
2540 nigel 41
2541 ph10 567 The functions pcre_copy_substring() and pcre_get_substring() extract a
2542     single substring, whose number is given as stringnumber. A value of
2543     zero extracts the substring that matched the entire pattern, whereas
2544     higher values extract the captured substrings. For pcre_copy_sub-
2545     string(), the string is placed in buffer, whose length is given by
2546     buffersize, while for pcre_get_substring() a new block of memory is
2547     obtained via pcre_malloc, and its address is returned via stringptr.
2548     The yield of the function is the length of the string, not including
2549 nigel 93 the terminating zero, or one of these error codes:
2550 nigel 41
2551 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2552 nigel 41
2553 ph10 567 The buffer was too small for pcre_copy_substring(), or the attempt to
2554 nigel 73 get memory failed for pcre_get_substring().
2555 nigel 41
2556 nigel 73 PCRE_ERROR_NOSUBSTRING (-7)
2557 nigel 41
2558 nigel 73 There is no substring whose number is stringnumber.
2559 nigel 41
2560 ph10 567 The pcre_get_substring_list() function extracts all available sub-
2561     strings and builds a list of pointers to them. All this is done in a
2562 nigel 75 single block of memory that is obtained via pcre_malloc. The address of
2563 ph10 567 the memory block is returned via listptr, which is also the start of
2564     the list of string pointers. The end of the list is marked by a NULL
2565     pointer. The yield of the function is zero if all went well, or the
2566 nigel 93 error code
2567 nigel 41
2568 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2569 nigel 41
2570 nigel 73 if the attempt to get the memory block failed.
2571 nigel 41
2572 ph10 567 When any of these functions encounter a substring that is unset, which
2573     can happen when capturing subpattern number n+1 matches some part of
2574     the subject, but subpattern n has not been used at all, they return an
2575 nigel 73 empty string. This can be distinguished from a genuine zero-length sub-
2576 ph10 567 string by inspecting the appropriate offset in ovector, which is nega-
2577 nigel 73 tive for unset substrings.
2578 nigel 41
2579 ph10 567 The two convenience functions pcre_free_substring() and pcre_free_sub-
2580     string_list() can be used to free the memory returned by a previous
2581 nigel 75 call of pcre_get_substring() or pcre_get_substring_list(), respec-
2582 ph10 567 tively. They do nothing more than call the function pointed to by
2583     pcre_free, which of course could be called directly from a C program.
2584     However, PCRE is used in some situations where it is linked via a spe-
2585     cial interface to another programming language that cannot use
2586     pcre_free directly; it is for these cases that the functions are pro-
2587 nigel 77 vided.
2588 nigel 41
2589 nigel 73
2590 nigel 63 EXTRACTING CAPTURED SUBSTRINGS BY NAME
2591 nigel 41
2592 nigel 75 int pcre_get_stringnumber(const pcre *code,
2593     const char *name);
2594    
2595 nigel 73 int pcre_copy_named_substring(const pcre *code,
2596     const char *subject, int *ovector,
2597     int stringcount, const char *stringname,
2598     char *buffer, int buffersize);
2599 nigel 41
2600 nigel 73 int pcre_get_named_substring(const pcre *code,
2601     const char *subject, int *ovector,
2602     int stringcount, const char *stringname,
2603     const char **stringptr);
2604 nigel 41
2605 ph10 567 To extract a substring by name, you first have to find associated num-
2606 nigel 75 ber. For example, for this pattern
2607 nigel 41
2608 nigel 93 (a+)b(?<xxx>\d+)...
2609 nigel 63
2610 nigel 91 the number of the subpattern called "xxx" is 2. If the name is known to
2611     be unique (PCRE_DUPNAMES was not set), you can find the number from the
2612     name by calling pcre_get_stringnumber(). The first argument is the com-
2613     piled pattern, and the second is the name. The yield of the function is
2614 ph10 567 the subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no
2615 nigel 91 subpattern of that name.
2616 nigel 63
2617 nigel 75 Given the number, you can extract the substring directly, or use one of
2618     the functions described in the previous section. For convenience, there
2619     are also two functions that do the whole job.
2620    
2621 ph10 567 Most of the arguments of pcre_copy_named_substring() and
2622     pcre_get_named_substring() are the same as those for the similarly
2623     named functions that extract by number. As these are described in the
2624     previous section, they are not re-described here. There are just two
2625 nigel 75 differences:
2626 nigel 63
2627 ph10 567 First, instead of a substring number, a substring name is given. Sec-
2628 nigel 73 ond, there is an extra argument, given at the start, which is a pointer
2629 ph10 567 to the compiled pattern. This is needed in order to gain access to the
2630 nigel 73 name-to-number translation table.
2631 nigel 63
2632 ph10 567 These functions call pcre_get_stringnumber(), and if it succeeds, they
2633     then call pcre_copy_substring() or pcre_get_substring(), as appropri-
2634     ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the
2635 ph10 128 behaviour may not be what you want (see the next section).
2636 nigel 63
2637 ph10 461 Warning: If the pattern uses the (?| feature to set up multiple subpat-
2638 ph10 567 terns with the same number, as described in the section on duplicate
2639     subpattern numbers in the pcrepattern page, you cannot use names to
2640     distinguish the different subpatterns, because names are not included
2641     in the compiled code. The matching process uses only numbers. For this
2642     reason, the use of different names for subpatterns of the same number
2643 ph10 461 causes an error at compile time.
2644 nigel 77
2645 ph10 392
2646 nigel 91 DUPLICATE SUBPATTERN NAMES
2647    
2648     int pcre_get_stringtable_entries(const pcre *code,
2649     const char *name, char **first, char **last);
2650    
2651 ph10 567 When a pattern is compiled with the PCRE_DUPNAMES option, names for
2652     subpatterns are not required to be unique. (Duplicate names are always
2653     allowed for subpatterns with the same number, created by using the (?|
2654     feature. Indeed, if such subpatterns are named, they are required to
2655 ph10 461 use the same names.)
2656 ph10 208
2657 ph10 461 Normally, patterns with duplicate names are such that in any one match,
2658 ph10 567 only one of the named subpatterns participates. An example is shown in
2659 ph10 461 the pcrepattern documentation.
2660    
2661 ph10 567 When duplicates are present, pcre_copy_named_substring() and
2662     pcre_get_named_substring() return the first substring corresponding to
2663     the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING
2664     (-7) is returned; no data is returned. The pcre_get_stringnumber()
2665     function returns one of the numbers that are associated with the name,
2666 ph10 208 but it is not defined which it is.
2667 nigel 91
2668 ph10 567 If you want to get full details of all captured substrings for a given
2669     name, you must use the pcre_get_stringtable_entries() function. The
2670 nigel 91 first argument is the compiled pattern, and the second is the name. The
2671 ph10 567 third and fourth are pointers to variables which are updated by the
2672 nigel 91 function. After it has run, they point to the first and last entries in
2673 ph10 567 the name-to-number table for the given name. The function itself
2674     returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if
2675     there are none. The format of the table is described above in the sec-
2676     tion entitled Information about a pattern. Given all the relevant
2677     entries for the name, you can extract each of their numbers, and hence
2678 nigel 93 the captured data, if any.
2679 nigel 91
2680    
2681 nigel 77 FINDING ALL POSSIBLE MATCHES
2682    
2683 ph10 567 The traditional matching function uses a similar algorithm to Perl,
2684 nigel 77 which stops when it finds the first match, starting at a given point in
2685 ph10 567 the subject. If you want to find all possible matches, or the longest
2686     possible match, consider using the alternative matching function (see
2687     below) instead. If you cannot use the alternative function, but still
2688     need to find all possible matches, you can kludge it up by making use
2689 nigel 77 of the callout facility, which is described in the pcrecallout documen-
2690     tation.
2691    
2692     What you have to do is to insert a callout right at the end of the pat-
2693 ph10 567 tern. When your callout function is called, extract and save the cur-
2694     rent matched substring. Then return 1, which forces pcre_exec() to
2695     backtrack and try other alternatives. Ultimately, when it runs out of
2696 nigel 77 matches, pcre_exec() will yield PCRE_ERROR_NOMATCH.
2697    
2698    
2699     MATCHING A PATTERN: THE ALTERNATIVE FUNCTION
2700    
2701     int pcre_dfa_exec(const pcre *code, const pcre_extra *extra,
2702     const char *subject, int length, int startoffset,
2703     int options, int *ovector, int ovecsize,
2704     int *workspace, int wscount);
2705    
2706 ph10 567 The function pcre_dfa_exec() is called to match a subject string
2707     against a compiled pattern, using a matching algorithm that scans the
2708     subject string just once, and does not backtrack. This has different
2709     characteristics to the normal algorithm, and is not compatible with
2710     Perl. Some of the features of PCRE patterns are not supported. Never-
2711     theless, there are times when this kind of matching can be useful. For
2712     a discussion of the two matching algorithms, and a list of features
2713     that pcre_dfa_exec() does not support, see the pcrematching documenta-
2714 ph10 453 tion.
2715 nigel 77
2716 ph10 567 The arguments for the pcre_dfa_exec() function are the same as for
2717 nigel 77 pcre_exec(), plus two extras. The ovector argument is used in a differ-
2718 ph10 567 ent way, and this is described below. The other common arguments are
2719     used in the same way as for pcre_exec(), so their description is not
2720 nigel 77 repeated here.
2721    
2722 ph10 567 The two additional arguments provide workspace for the function. The
2723     workspace vector should contain at least 20 elements. It is used for
2724 nigel 77 keeping track of multiple paths through the pattern tree. More
2725 ph10 567 workspace will be needed for patterns and subjects where there are a
2726 nigel 91 lot of potential matches.
2727 nigel 77
2728 nigel 87 Here is an example of a simple call to pcre_dfa_exec():
2729 nigel 77
2730     int rc;
2731     int ovector[10];
2732     int wspace[20];
2733 nigel 87 rc = pcre_dfa_exec(
2734 nigel 77 re, /* result of pcre_compile() */
2735     NULL, /* we didn't study the pattern */
2736     "some string", /* the subject string */
2737     11, /* the length of the subject string */
2738     0, /* start at offset 0 in the subject */
2739     0, /* default options */
2740     ovector, /* vector of integers for substring information */
2741     10, /* number of elements (NOT size in bytes) */
2742     wspace, /* working space vector */
2743     20); /* number of elements (NOT size in bytes) */
2744    
2745     Option bits for pcre_dfa_exec()
2746    
2747 ph10 567 The unused bits of the options argument for pcre_dfa_exec() must be
2748     zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW-
2749 ph10 453 LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY,
2750 ph10 567 PCRE_NOTEMPTY_ATSTART, PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF,
2751     PCRE_BSR_UNICODE, PCRE_NO_START_OPTIMIZE, PCRE_PARTIAL_HARD, PCRE_PAR-
2752     TIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last
2753     four of these are exactly the same as for pcre_exec(), so their
2754 ph10 453 description is not repeated here.
2755 nigel 77
2756 ph10 429 PCRE_PARTIAL_HARD
2757     PCRE_PARTIAL_SOFT
2758 nigel 77
2759 ph10 567 These have the same general effect as they do for pcre_exec(), but the
2760     details are slightly different. When PCRE_PARTIAL_HARD is set for
2761     pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub-
2762     ject is reached and there is still at least one matching possibility
2763 ph10 429 that requires additional characters. This happens even if some complete
2764     matches have also been found. When PCRE_PARTIAL_SOFT is set, the return
2765     code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end
2766 ph10 567 of the subject is reached, there have been no complete matches, but
2767     there is still at least one matching possibility. The portion of the
2768     string that was inspected when the longest partial match was found is
2769     set as the first matching string in both cases. There is a more
2770     detailed discussion of partial and multi-segment matching, with exam-
2771     ples, in the pcrepartial documentation.
2772 nigel 77
2773     PCRE_DFA_SHORTEST
2774    
2775 ph10 567 Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to
2776 nigel 93 stop as soon as it has found one match. Because of the way the alterna-
2777 ph10 567 tive algorithm works, this is necessarily the shortest possible match
2778 nigel 93 at the first possible matching point in the subject string.
2779 nigel 77
2780     PCRE_DFA_RESTART
2781    
2782 ph10 429 When pcre_dfa_exec() returns a partial match, it is possible to call it
2783 ph10 567 again, with additional subject characters, and have it continue with
2784     the same match. The PCRE_DFA_RESTART option requests this action; when
2785     it is set, the workspace and wscount options must reference the same
2786     vector as before because data about the match so far is left in them
2787 ph10 429 after a partial match. There is more discussion of this facility in the
2788     pcrepartial documentation.
2789 nigel 77
2790     Successful returns from pcre_dfa_exec()
2791    
2792 ph10 567 When pcre_dfa_exec() succeeds, it may have matched more than one sub-
2793 nigel 77 string in the subject. Note, however, that all the matches from one run
2794 ph10 567 of the function start at the same point in the subject. The shorter
2795     matches are all initial substrings of the longer matches. For example,
2796 nigel 77 if the pattern
2797    
2798     <.*>
2799    
2800     is matched against the string
2801    
2802     This is <something> <something else> <something further> no more
2803    
2804     the three matched strings are
2805    
2806     <something>
2807     <something> <something else>
2808     <something> <something else> <something further>
2809    
2810 ph10 567 On success, the yield of the function is a number greater than zero,
2811     which is the number of matched substrings. The substrings themselves
2812     are returned in ovector. Each string uses two elements; the first is
2813     the offset to the start, and the second is the offset to the end. In
2814     fact, all the strings have the same start offset. (Space could have
2815     been saved by giving this only once, but it was decided to retain some
2816     compatibility with the way pcre_exec() returns data, even though the
2817 nigel 93 meaning of the strings is different.)
2818 nigel 77
2819     The strings are returned in reverse order of length; that is, the long-
2820 ph10 567 est matching string is given first. If there were too many matches to
2821     fit into ovector, the yield of the function is zero, and the vector is
2822 nigel 77 filled with the longest matches.
2823    
2824     Error returns from pcre_dfa_exec()
2825    
2826 ph10 567 The pcre_dfa_exec() function returns a negative number when it fails.
2827     Many of the errors are the same as for pcre_exec(), and these are
2828     described above. There are in addition the following errors that are
2829 nigel 77 specific to pcre_dfa_exec():
2830    
2831     PCRE_ERROR_DFA_UITEM (-16)
2832    
2833 ph10 567 This return is given if pcre_dfa_exec() encounters an item in the pat-
2834     tern that it does not support, for instance, the use of \C or a back
2835 nigel 77 reference.
2836    
2837     PCRE_ERROR_DFA_UCOND (-17)
2838    
2839 ph10 567 This return is given if pcre_dfa_exec() encounters a condition item
2840     that uses a back reference for the condition, or a test for recursion
2841 nigel 93 in a specific group. These are not supported.
2842 nigel 77
2843     PCRE_ERROR_DFA_UMLIMIT (-18)
2844    
2845 ph10 567 This return is given if pcre_dfa_exec() is called with an extra block
2846 nigel 77 that contains a setting of the match_limit field. This is not supported
2847     (it is meaningless).
2848    
2849     PCRE_ERROR_DFA_WSSIZE (-19)
2850    
2851 ph10 567 This return is given if pcre_dfa_exec() runs out of space in the
2852 nigel 77 workspace vector.
2853    
2854     PCRE_ERROR_DFA_RECURSE (-20)
2855    
2856 ph10 567 When a recursive subpattern is processed, the matching function calls
2857     itself recursively, using private vectors for ovector and workspace.
2858     This error is given if the output vector is not large enough. This
2859 nigel 77 should be extremely rare, as a vector of size 1000 is used.
2860    
2861 nigel 93
2862     SEE ALSO
2863    
2864 ph10 567 pcrebuild(3), pcrecallout(3), pcrecpp(3)(3), pcrematching(3), pcrepar-
2865 ph10 392 tial(3), pcreposix(3), pcreprecompile(3), pcresample(3), pcrestack(3).
2866 nigel 93
2867 nigel 63
2868 ph10 99 AUTHOR
2869 nigel 63
2870 ph10 99 Philip Hazel
2871     University Computing Service
2872     Cambridge CB2 3QH, England.
2873    
2874    
2875     REVISION
2876    
2877 ph10 579 Last updated: 21 November 2010
2878 ph10 512 Copyright (c) 1997-2010 University of Cambridge.
2879 ph10 99 ------------------------------------------------------------------------------
2880 ph10 589
2881    
2882 nigel 79 PCRECALLOUT(3) PCRECALLOUT(3)
2883 nigel 63
2884 nigel 79
2885 nigel 73 NAME
2886     PCRE - Perl-compatible regular expressions
2887    
2888 nigel 77
2889 nigel 63 PCRE CALLOUTS
2890    
2891 nigel 73 int (*pcre_callout)(pcre_callout_block *);
2892 nigel 63
2893 nigel 73 PCRE provides a feature called "callout", which is a means of temporar-
2894     ily passing control to the caller of PCRE in the middle of pattern
2895     matching. The caller of PCRE provides an external function by putting
2896     its entry point in the global variable pcre_callout. By default, this
2897     variable contains NULL, which disables all calling out.
2898 nigel 63
2899 nigel 73 Within a regular expression, (?C) indicates the points at which the
2900     external function is to be called. Different callout points can be
2901     identified by putting a number less than 256 after the letter C. The
2902     default value is zero. For example, this pattern has two callout
2903     points:
2904 nigel 63
2905 ph10 155 (?C1)abc(?C2)def
2906 nigel 63
2907 ph10 461 If the PCRE_AUTO_CALLOUT option bit is set when pcre_compile() or
2908     pcre_compile2() is called, PCRE automatically inserts callouts, all
2909     with number 255, before each item in the pattern. For example, if
2910     PCRE_AUTO_CALLOUT is used with the pattern
2911 nigel 63
2912 nigel 75 A(\d{2}|--)
2913    
2914     it is processed as if it were
2915    
2916     (?C255)A(?C255)((?C255)\d{2}(?C255)|(?C255)-(?C255)-(?C255))(?C255)
2917    
2918     Notice that there is a callout before and after each parenthesis and
2919     alternation bar. Automatic callouts can be used for tracking the
2920     progress of pattern matching. The pcretest command has an option that
2921     sets automatic callouts; when it is used, the output indicates how the
2922     pattern is matched. This is useful information when you are trying to
2923     optimize the performance of a particular pattern.
2924    
2925    
2926     MISSING CALLOUTS
2927    
2928     You should be aware that, because of optimizations in the way PCRE
2929 ph10 392 matches patterns by default, callouts sometimes do not happen. For
2930     example, if the pattern is
2931 nigel 75
2932     ab(?C4)cd
2933    
2934     PCRE knows that any matching string must contain the letter "d". If the
2935     subject string is "abyz", the lack of "d" means that matching doesn't
2936     ever start, and the callout is never reached. However, with "abyd",
2937     though the result is still no match, the callout is obeyed.
2938    
2939 ph10 461 If the pattern is studied, PCRE knows the minimum length of a matching
2940     string, and will immediately give a "no match" return without actually
2941     running a match if the subject is not long enough, or, for unanchored
2942     patterns, if it has been scanned far enough.
2943    
2944     You can disable these optimizations by passing the PCRE_NO_START_OPTI-
2945 ph10 579 MIZE option to pcre_compile(), pcre_exec(), or pcre_dfa_exec(), or by
2946     starting the pattern with (*NO_START_OPT). This slows down the matching
2947     process, but does ensure that callouts such as the example above are
2948     obeyed.
2949 nigel 75
2950 ph10 392
2951 nigel 75 THE CALLOUT INTERFACE
2952    
2953 ph10 579 During matching, when PCRE reaches a callout point, the external func-
2954     tion defined by pcre_callout is called (if it is set). This applies to
2955     both the pcre_exec() and the pcre_dfa_exec() matching functions. The
2956     only argument to the callout function is a pointer to a pcre_callout
2957 nigel 77 block. This structure contains the following fields:
2958 nigel 75
2959 nigel 73 int version;
2960     int callout_number;
2961     int *offset_vector;
2962     const char *subject;
2963     int subject_length;
2964     int start_match;
2965     int current_position;
2966     int capture_top;
2967     int capture_last;
2968     void *callout_data;
2969 nigel 75 int pattern_position;
2970     int next_item_length;
2971 nigel 63
2972 ph10 579 The version field is an integer containing the version number of the
2973     block format. The initial version was 0; the current version is 1. The
2974     version number will change again in future if additional fields are
2975 nigel 75 added, but the intention is never to remove any of the existing fields.
2976 nigel 63
2977 ph10 579 The callout_number field contains the number of the callout, as com-
2978     piled into the pattern (that is, the number after ?C for manual call-
2979 nigel 75 outs, and 255 for automatically generated callouts).
2980 nigel 63
2981 ph10 579 The offset_vector field is a pointer to the vector of offsets that was
2982     passed by the caller to pcre_exec() or pcre_dfa_exec(). When
2983     pcre_exec() is used, the contents can be inspected in order to extract
2984     substrings that have been matched so far, in the same way as for
2985     extracting substrings after a match has completed. For pcre_dfa_exec()
2986 nigel 77 this field is not useful.
2987 nigel 63
2988 nigel 75 The subject and subject_length fields contain copies of the values that
2989 nigel 73 were passed to pcre_exec().
2990 nigel 63
2991 ph10 579 The start_match field normally contains the offset within the subject
2992     at which the current match attempt started. However, if the escape
2993     sequence \K has been encountered, this value is changed to reflect the
2994     modified starting point. If the pattern is not anchored, the callout
2995 ph10 172 function may be called several times from the same point in the pattern
2996     for different starting points in the subject.
2997 nigel 63
2998 ph10 579 The current_position field contains the offset within the subject of
2999 nigel 73 the current match pointer.
3000 nigel 63
3001 ph10 579 When the pcre_exec() function is used, the capture_top field contains
3002     one more than the number of the highest numbered captured substring so
3003     far. If no substrings have been captured, the value of capture_top is
3004     one. This is always the case when pcre_dfa_exec() is used, because it
3005 nigel 77 does not support captured substrings.
3006 nigel 63
3007 ph10 579 The capture_last field contains the number of the most recently cap-
3008     tured substring. If no substrings have been captured, its value is -1.
3009 nigel 77 This is always the case when pcre_dfa_exec() is used.
3010 nigel 63
3011 ph10 579 The callout_data field contains a value that is passed to pcre_exec()
3012     or pcre_dfa_exec() specifically so that it can be passed back in call-
3013     outs. It is passed in the pcre_callout field of the pcre_extra data
3014     structure. If no such data was passed, the value of callout_data in a
3015     pcre_callout block is NULL. There is a description of the pcre_extra
3016 nigel 73 structure in the pcreapi documentation.
3017 nigel 63
3018 ph10 579 The pattern_position field is present from version 1 of the pcre_call-
3019 nigel 75 out structure. It contains the offset to the next item to be matched in
3020     the pattern string.
3021 nigel 63
3022 ph10 579 The next_item_length field is present from version 1 of the pcre_call-
3023 nigel 75 out structure. It contains the length of the next item to be matched in
3024 ph10 579 the pattern string. When the callout immediately precedes an alterna-
3025     tion bar, a closing parenthesis, or the end of the pattern, the length
3026     is zero. When the callout precedes an opening parenthesis, the length
3027 nigel 75 is that of the entire subpattern.
3028 nigel 73
3029 ph10 579 The pattern_position and next_item_length fields are intended to help
3030     in distinguishing between different automatic callouts, which all have
3031 nigel 75 the same callout number. However, they are set for all callouts.
3032    
3033    
3034 nigel 63 RETURN VALUES
3035    
3036 ph10 579 The external callout function returns an integer to PCRE. If the value
3037     is zero, matching proceeds as normal. If the value is greater than
3038     zero, matching fails at the current point, but the testing of other
3039 nigel 77 matching possibilities goes ahead, just as if a lookahead assertion had
3040 ph10 579 failed. If the value is less than zero, the match is abandoned, and
3041 ph10 461 pcre_exec() or pcre_dfa_exec() returns the negative value.
3042 nigel 63
3043 ph10 579 Negative values should normally be chosen from the set of
3044 nigel 73 PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan-
3045 ph10 579 dard "no match" failure. The error number PCRE_ERROR_CALLOUT is
3046     reserved for use by callout functions; it will never be used by PCRE
3047 nigel 73 itself.
3048 nigel 63
3049    
3050 ph10 99 AUTHOR
3051 nigel 63
3052 ph10 99 Philip Hazel
3053     University Computing Service
3054     Cambridge CB2 3QH, England.
3055    
3056    
3057     REVISION
3058    
3059 ph10 579 Last updated: 21 November 2010
3060     Copyright (c) 1997-2010 University of Cambridge.
3061 ph10 99 ------------------------------------------------------------------------------
3062 ph10 589
3063    
3064 nigel 79 PCRECOMPAT(3) PCRECOMPAT(3)
3065 nigel 63
3066 nigel 79
3067 nigel 73 NAME
3068     PCRE - Perl-compatible regular expressions
3069    
3070 nigel 77
3071 nigel 75 DIFFERENCES BETWEEN PCRE AND PERL
3072 nigel 41
3073 nigel 73 This document describes the differences in the ways that PCRE and Perl
3074 ph10 461 handle regular expressions. The differences described here are with
3075 ph10 567 respect to Perl versions 5.10 and above.
3076 nigel 41
3077 ph10 461 1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details
3078     of what it does have are given in the section on UTF-8 support in the
3079 nigel 87 main pcre page.
3080 nigel 41
3081 nigel 73 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl
3082 ph10 461 permits them, but they do not mean what you might think. For example,
3083 nigel 73 (?!a){3} does not assert that the next three characters are not "a". It
3084     just asserts that the next character is not "a" three times.
3085 nigel 41
3086 ph10 461 3. Capturing subpatterns that occur inside negative lookahead asser-
3087     tions are counted, but their entries in the offsets vector are never
3088     set. Perl sets its numerical variables from any such patterns that are
3089 nigel 73 matched before the assertion fails to match something (thereby succeed-
3090 ph10 461 ing), but only if the negative lookahead assertion contains just one
3091 nigel 73 branch.
3092 nigel 41
3093 ph10 461 4. Though binary zero characters are supported in the subject string,
3094 nigel 73 they are not allowed in a pattern string because it is passed as a nor-
3095 nigel 75 mal C string, terminated by zero. The escape sequence \0 can be used in
3096     the pattern to represent a binary zero.
3097 nigel 41
3098 ph10 461 5. The following Perl escape sequences are not supported: \l, \u, \L,
3099 nigel 75 \U, and \N. In fact these are implemented by Perl's general string-han-
3100 ph10 461 dling and are not part of its pattern matching engine. If any of these
3101 nigel 75 are encountered by PCRE, an error is generated.
3102 nigel 41
3103 ph10 461 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE
3104     is built with Unicode character property support. The properties that
3105     can be tested with \p and \P are limited to the general category prop-
3106     erties such as Lu and Nd, script names such as Greek or Han, and the
3107     derived properties Any and L&. PCRE does support the Cs (surrogate)
3108     property, which Perl does not; the Perl documentation says "Because
3109 ph10 453 Perl hides the need for the user to understand the internal representa-
3110 ph10 461 tion of Unicode characters, there is no need to implement the somewhat
3111 ph10 453 messy concept of surrogates."
3112 nigel 75
3113     7. PCRE does support the \Q...\E escape for quoting substrings. Charac-
3114 ph10 461 ters in between are treated as literals. This is slightly different
3115     from Perl in that $ and @ are also handled as literals inside the
3116     quotes. In Perl, they cause variable interpolation (but of course PCRE
3117 nigel 73 does not have variables). Note the following examples:
3118 nigel 49
3119 nigel 73 Pattern PCRE matches Perl matches
3120 nigel 41
3121 nigel 73 \Qabc$xyz\E abc$xyz abc followed by the
3122     contents of $xyz
3123     \Qabc\$xyz\E abc\$xyz abc\$xyz
3124     \Qabc\E\$\Qxyz\E abc$xyz abc$xyz
3125 nigel 41
3126 ph10 461 The \Q...\E sequence is recognized both inside and outside character
3127 nigel 73 classes.
3128 nigel 41
3129 nigel 93 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code})
3130 ph10 461 constructions. However, there is support for recursive patterns. This
3131     is not available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE
3132     "callout" feature allows an external function to be called during pat-
3133 nigel 75 tern matching. See the pcrecallout documentation for details.
3134 nigel 63
3135 ph10 461 9. Subpatterns that are called recursively or as "subroutines" are
3136     always treated as atomic groups in PCRE. This is like Python, but
3137     unlike Perl. There is a discussion of an example that explains this in
3138     more detail in the section on recursion differences from Perl in the
3139     pcrepattern page.
3140 nigel 93
3141 ph10 461 10. There are some differences that are concerned with the settings of
3142     captured strings when part of a pattern is repeated. For example,
3143     matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2
3144 nigel 73 unset, but in PCRE it is set to "b".
3145 nigel 41
3146 ph10 535 11. PCRE's handling of duplicate subpattern numbers and duplicate sub-
3147 ph10 461 pattern names is not as general as Perl's. This is a consequence of the
3148     fact the PCRE works internally just with numbers, using an external ta-
3149     ble to translate between numbers and names. In particular, a pattern
3150     such as (?|(?<a>A)|(?<b)B), where the two capturing parentheses have
3151     the same number but different names, is not supported, and causes an
3152     error at compile time. If it were allowed, it would not be possible to
3153     distinguish which parentheses matched, because both names map to cap-
3154     turing subpattern number 1. To avoid this confusing situation, an error
3155     is given at compile time.
3156 nigel 41
3157 ph10 567 12. Perl recognizes comments in some places that PCRE doesn't, for
3158     example, between the ( and ? at the start of a subpattern.
3159    
3160     13. PCRE provides some extensions to the Perl regular expression facil-
3161     ities. Perl 5.10 includes new features that are not in earlier ver-
3162     sions of Perl, some of which (such as named parentheses) have been in
3163 ph10 461 PCRE for some time. This list is with respect to Perl 5.10:
3164 nigel 41
3165 ph10 567 (a) Although lookbehind assertions in PCRE must match fixed length
3166     strings, each alternative branch of a lookbehind assertion can match a
3167     different length of string. Perl requires them all to have the same
3168 ph10 461 length.
3169    
3170 ph10 567 (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $
3171 nigel 73 meta-character matches only at the very end of the string.
3172 nigel 41
3173 nigel 73 (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe-
3174 ph10 182 cial meaning is faulted. Otherwise, like Perl, the backslash is quietly
3175     ignored. (Perl can be made to issue a warning.)
3176 nigel 41
3177 ph10 567 (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti-
3178 nigel 73 fiers is inverted, that is, by default they are not greedy, but if fol-
3179     lowed by a question mark they are.
3180 nigel 41
3181 nigel 75 (e) PCRE_ANCHORED can be used at matching time to force a pattern to be
3182     tried only at the first matching position in the subject string.
3183 nigel 41
3184 ph10 453 (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
3185 ph10 567 and PCRE_NO_AUTO_CAPTURE options for pcre_exec() have no Perl equiva-
3186 ph10 453 lents.
3187 nigel 41
3188 ph10 567 (g) The \R escape sequence can be restricted to match only CR, LF, or
3189 ph10 231 CRLF by the PCRE_BSR_ANYCRLF option.
3190 nigel 41
3191 ph10 231 (h) The callout facility is PCRE-specific.
3192 nigel 43
3193 ph10 231 (i) The partial matching facility is PCRE-specific.
3194    
3195     (j) Patterns compiled by PCRE can be saved and re-used at a later time,
3196 nigel 75 even on different hosts that have the other endianness.
3197    
3198 ph10 567 (k) The alternative matching function (pcre_dfa_exec()) matches in a
3199 nigel 77 different way and is not Perl-compatible.
3200    
3201 ph10 567 (l) PCRE recognizes some special sequences such as (*CR) at the start
3202 ph10 231 of a pattern that set overall options that cannot be changed within the
3203     pattern.
3204 nigel 63
3205 ph10 231
3206 ph10 99 AUTHOR
3207 nigel 63
3208 ph10 99 Philip Hazel
3209     University Computing Service
3210     Cambridge CB2 3QH, England.
3211    
3212    
3213     REVISION
3214    
3215 ph10 567 Last updated: 31 October 2010
3216 ph10 535 Copyright (c) 1997-2010 University of Cambridge.
3217 ph10 99 ------------------------------------------------------------------------------
3218 ph10 589
3219    
3220 nigel 79 PCREPATTERN(3) PCREPATTERN(3)
3221 nigel 63
3222 nigel 79
3223 nigel 73 NAME
3224     PCRE - Perl-compatible regular expressions
3225    
3226 nigel 77
3227 nigel 63 PCRE REGULAR EXPRESSION DETAILS
3228    
3229 ph10 208 The syntax and semantics of the regular expressions that are supported
3230     by PCRE are described in detail below. There is a quick-reference syn-
3231 ph10 345 tax summary in the pcresyntax page. PCRE tries to match Perl syntax and
3232     semantics as closely as it can. PCRE also supports some alternative
3233     regular expression syntax (which does not conflict with the Perl syn-
3234     tax) in order to provide some compatibility with regular expressions in
3235     Python, .NET, and Oniguruma.
3236 nigel 49
3237 ph10 345 Perl's regular expressions are described in its own documentation, and
3238     regular expressions in general are covered in a number of books, some
3239     of which have copious examples. Jeffrey Friedl's "Mastering Regular
3240     Expressions", published by O'Reilly, covers regular expressions in
3241     great detail. This description of PCRE's regular expressions is
3242     intended as reference material.
3243    
3244 nigel 75 The original operation of PCRE was on strings of one-byte characters.
3245     However, there is now also support for UTF-8 character strings. To use
3246 ph10 461 this, PCRE must be built to include UTF-8 support, and you must call
3247     pcre_compile() or pcre_compile2() with the PCRE_UTF8 option. There is
3248     also a special sequence that can be given at the start of a pattern:
3249 nigel 41
3250 ph10 416 (*UTF8)
3251    
3252     Starting a pattern with this sequence is equivalent to setting the
3253     PCRE_UTF8 option. This feature is not Perl-compatible. How setting
3254     UTF-8 mode affects pattern matching is mentioned in several places
3255     below. There is also a summary of UTF-8 features in the section on
3256     UTF-8 support in the main pcre page.
3257    
3258 ph10 535 Another special sequence that may appear at the start of a pattern or
3259     in combination with (*UTF8) is:
3260    
3261     (*UCP)
3262    
3263     This has the same effect as setting the PCRE_UCP option: it causes
3264     sequences such as \d and \w to use Unicode properties to determine
3265     character types, instead of recognizing only characters with codes less
3266     than 128 via a lookup table.
3267    
3268 ph10 579 If a pattern starts with (*NO_START_OPT), it has the same effect as
3269     setting the PCRE_NO_START_OPTIMIZE option either at compile or matching
3270     time. There are also some more of these special sequences that are con-
3271     cerned with the handling of newlines; they are described below.
3272    
3273     The remainder of this document discusses the patterns that are sup-
3274     ported by PCRE when its main matching function, pcre_exec(), is used.
3275     From release 6.0, PCRE offers a second matching function,
3276     pcre_dfa_exec(), which matches using a different algorithm that is not
3277 ph10 172 Perl-compatible. Some of the features discussed below are not available
3278 ph10 579 when pcre_dfa_exec() is used. The advantages and disadvantages of the
3279     alternative function, and how it differs from the normal function, are
3280 ph10 172 discussed in the pcrematching page.
3281 nigel 77
3282 nigel 93
3283 ph10 227 NEWLINE CONVENTIONS
3284    
3285 ph10 579 PCRE supports five different conventions for indicating line breaks in
3286     strings: a single CR (carriage return) character, a single LF (line-
3287 ph10 227 feed) character, the two-character sequence CRLF, any of the three pre-
3288 ph10 579 ceding, or any Unicode newline sequence. The pcreapi page has further
3289     discussion about newlines, and shows how to set the newline convention
3290 ph10 227 in the options arguments for the compiling and matching functions.
3291    
3292 ph10 579 It is also possible to specify a newline convention by starting a pat-
3293 ph10 227 tern string with one of the following five sequences:
3294    
3295     (*CR) carriage return
3296     (*LF) linefeed
3297     (*CRLF) carriage return, followed by linefeed
3298     (*ANYCRLF) any of the three above
3299     (*ANY) all Unicode newline sequences
3300    
3301 ph10 579 These override the default and the options given to pcre_compile() or
3302     pcre_compile2(). For example, on a Unix system where LF is the default
3303 ph10 461 newline sequence, the pattern
3304 ph10 227
3305     (*CR)a.b
3306    
3307     changes the convention to CR. That pattern matches "a\nb" because LF is
3308 ph10 579 no longer a newline. Note that these special settings, which are not
3309     Perl-compatible, are recognized only at the very start of a pattern,
3310     and that they must be in upper case. If more than one of them is
3311 ph10 231 present, the last one is used.
3312 ph10 227
3313 ph10 579 The newline convention affects the interpretation of the dot metachar-
3314     acter when PCRE_DOTALL is not set, and also the behaviour of \N. How-
3315     ever, it does not affect what the \R escape sequence matches. By
3316     default, this is any Unicode newline sequence, for Perl compatibility.
3317     However, this can be changed; see the description of \R in the section
3318     entitled "Newline sequences" below. A change of \R setting can be com-
3319 ph10 518 bined with a change of newline convention.
3320 ph10 227
3321 ph10 231
3322 nigel 93 CHARACTERS AND METACHARACTERS
3323    
3324 ph10 579 A regular expression is a pattern that is matched against a subject
3325     string from left to right. Most characters stand for themselves in a
3326     pattern, and match the corresponding characters in the subject. As a
3327 nigel 73 trivial example, the pattern
3328 nigel 41
3329 nigel 73 The quick brown fox
3330 nigel 41
3331 nigel 77 matches a portion of a subject string that is identical to itself. When
3332 ph10 579 caseless matching is specified (the PCRE_CASELESS option), letters are
3333     matched independently of case. In UTF-8 mode, PCRE always understands
3334     the concept of case for characters whose values are less than 128, so
3335     caseless matching is always possible. For characters with higher val-
3336     ues, the concept of case is supported if PCRE is compiled with Unicode
3337     property support, but not otherwise. If you want to use caseless
3338     matching for characters 128 and above, you must ensure that PCRE is
3339 nigel 77 compiled with Unicode property support as well as with UTF-8 support.
3340 nigel 41
3341 ph10 579 The power of regular expressions comes from the ability to include
3342     alternatives and repetitions in the pattern. These are encoded in the
3343 nigel 77 pattern by the use of metacharacters, which do not stand for themselves
3344     but instead are interpreted in some special way.
3345    
3346 ph10 579 There are two different sets of metacharacters: those that are recog-
3347     nized anywhere in the pattern except within square brackets, and those
3348     that are recognized within square brackets. Outside square brackets,
3349 nigel 93 the metacharacters are as follows:
3350 nigel 41
3351 nigel 73 \ general escape character with several uses
3352