/[pcre]/code/trunk/doc/pcre.txt
ViewVC logotype

Contents of /code/trunk/doc/pcre.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 572 - (hide annotations) (download)
Wed Nov 17 17:55:57 2010 UTC (3 years, 10 months ago) by ph10
File MIME type: text/plain
File size: 365440 byte(s)
Documentation updates and tidies.

1 nigel 75 -----------------------------------------------------------------------------
2 nigel 63 This file contains a concatenation of the PCRE man pages, converted to plain
3     text format for ease of searching with a text editor, or for use on systems
4     that do not have a man page processor. The small individual files that give
5 ph10 567 synopses of each function in the library have not been included. Neither has
6 ph10 429 the pcredemo program. There are separate text files for the pcregrep and
7     pcretest commands.
8 nigel 63 -----------------------------------------------------------------------------
9    
10 nigel 41
11 nigel 79 PCRE(3) PCRE(3)
12 nigel 41
13 nigel 79
14 nigel 73 NAME
15     PCRE - Perl-compatible regular expressions
16    
17 nigel 77
18 nigel 75 INTRODUCTION
19 nigel 41
20 nigel 73 The PCRE library is a set of functions that implement regular expres-
21     sion pattern matching using the same syntax and semantics as Perl, with
22 ph10 461 just a few differences. Some features that appeared in Python and PCRE
23     before they appeared in Perl are also available using the Python syn-
24     tax, there is some support for one or two .NET and Oniguruma syntax
25     items, and there is an option for requesting some minor changes that
26     give better JavaScript compatibility.
27 nigel 63
28 ph10 461 The current implementation of PCRE corresponds approximately with Perl
29 ph10 572 5.12, including support for UTF-8 encoded strings and Unicode general
30     category properties. However, UTF-8 and Unicode support has to be
31 ph10 461 explicitly enabled; it is not the default. The Unicode tables corre-
32 ph10 507 spond to Unicode release 5.2.0.
33 nigel 77
34 nigel 93 In addition to the Perl-compatible matching function, PCRE contains an
35 ph10 461 alternative function that matches the same compiled patterns in a dif-
36     ferent way. In certain circumstances, the alternative function has some
37     advantages. For a discussion of the two matching algorithms, see the
38     pcrematching page.
39 nigel 93
40     PCRE is written in C and released as a C library. A number of people
41     have written wrappers and interfaces of various kinds. In particular,
42     Google Inc. have provided a comprehensive C++ wrapper. This is now
43 nigel 77 included as part of the PCRE distribution. The pcrecpp page has details
44 nigel 93 of this interface. Other people's contributions can be found in the
45 nigel 77 Contrib directory at the primary FTP site, which is:
46 nigel 63
47 nigel 73 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre
48 nigel 63
49 nigel 93 Details of exactly which Perl regular expression features are and are
50 nigel 73 not supported by PCRE are given in separate documents. See the pcrepat-
51 ph10 208 tern and pcrecompat pages. There is a syntax summary in the pcresyntax
52     page.
53 nigel 63
54 ph10 208 Some features of PCRE can be included, excluded, or changed when the
55     library is built. The pcre_config() function makes it possible for a
56     client to discover which features are available. The features them-
57     selves are described in the pcrebuild page. Documentation about build-
58 ph10 461 ing PCRE for various operating systems can be found in the README and
59     NON-UNIX-USE files in the source distribution.
60 nigel 63
61 ph10 208 The library contains a number of undocumented internal functions and
62     data tables that are used by more than one of the exported external
63     functions, but which are not intended for use by external callers.
64     Their names all begin with "_pcre_", which hopefully will not provoke
65 nigel 83 any name clashes. In some environments, it is possible to control which
66 ph10 208 external symbols are exported when a shared library is built, and in
67 nigel 83 these cases the undocumented symbols are not exported.
68 nigel 63
69 nigel 77
70 nigel 63 USER DOCUMENTATION
71    
72 ph10 208 The user documentation for PCRE comprises a number of different sec-
73     tions. In the "man" format, each of these is a separate "man page". In
74     the HTML format, each is a separate page, linked from the index page.
75 ph10 429 In the plain text format, all the sections, except the pcredemo sec-
76     tion, are concatenated, for ease of searching. The sections are as fol-
77     lows:
78 nigel 63
79 nigel 73 pcre this document
80 ph10 153 pcre-config show PCRE installation configuration information
81 nigel 77 pcreapi details of PCRE's native C API
82 nigel 73 pcrebuild options for building PCRE
83     pcrecallout details of the callout feature
84     pcrecompat discussion of Perl compatibility
85 nigel 77 pcrecpp details of the C++ wrapper
86 ph10 429 pcredemo a demonstration C program that uses PCRE
87 nigel 73 pcregrep description of the pcregrep command
88 nigel 77 pcrematching discussion of the two matching algorithms
89 nigel 75 pcrepartial details of the partial matching facility
90 nigel 73 pcrepattern syntax and semantics of supported
91     regular expressions
92     pcreperform discussion of performance issues
93 nigel 77 pcreposix the POSIX-compatible C API
94 nigel 75 pcreprecompile details of saving and re-using precompiled patterns
95 ph10 429 pcresample discussion of the pcredemo program
96 nigel 91 pcrestack discussion of stack usage
97 ph10 461 pcresyntax quick syntax reference
98 nigel 75 pcretest description of the pcretest testing command
99 nigel 63
100 ph10 429 In addition, in the "man" and HTML formats, there is a short page for
101 nigel 77 each C library function, listing its arguments and results.
102 nigel 63
103    
104     LIMITATIONS
105    
106 ph10 429 There are some size limitations in PCRE but it is hoped that they will
107 nigel 73 never in practice be relevant.
108 nigel 63
109 ph10 429 The maximum length of a compiled pattern is 65539 (sic) bytes if PCRE
110 nigel 73 is compiled with the default internal linkage size of 2. If you want to
111 ph10 429 process regular expressions that are truly enormous, you can compile
112     PCRE with an internal linkage size of 3 or 4 (see the README file in
113     the source distribution and the pcrebuild documentation for details).
114     In these cases the limit is substantially larger. However, the speed
115 nigel 93 of execution is slower.
116 nigel 63
117 ph10 208 All values in repeating quantifiers must be less than 65536.
118 nigel 63
119 nigel 93 There is no limit to the number of parenthesized subpatterns, but there
120     can be no more than 65535 capturing subpatterns.
121 nigel 63
122 nigel 93 The maximum length of name for a named subpattern is 32 characters, and
123     the maximum number of named subpatterns is 10000.
124 nigel 91
125 ph10 429 The maximum length of a subject string is the largest positive number
126     that an integer variable can hold. However, when using the traditional
127 nigel 77 matching function, PCRE uses recursion to handle subpatterns and indef-
128 ph10 429 inite repetition. This means that the available stack space may limit
129 nigel 77 the size of a subject string that can be processed by certain patterns.
130 nigel 91 For a discussion of stack issues, see the pcrestack documentation.
131 nigel 63
132    
133 nigel 75 UTF-8 AND UNICODE PROPERTY SUPPORT
134 nigel 63
135 ph10 429 From release 3.3, PCRE has had some support for character strings
136     encoded in the UTF-8 format. For release 4.0 this was greatly extended
137     to cover most common requirements, and in release 5.0 additional sup-
138 nigel 75 port for Unicode general category properties was added.
139 nigel 63
140 ph10 429 In order process UTF-8 strings, you must build PCRE to include UTF-8
141     support in the code, and, in addition, you must call pcre_compile()
142     with the PCRE_UTF8 option flag, or the pattern must start with the
143     sequence (*UTF8). When either of these is the case, both the pattern
144     and any subject strings that are matched against it are treated as
145 ph10 461 UTF-8 strings instead of strings of 1-byte characters.
146 nigel 63
147 ph10 429 If you compile PCRE with UTF-8 support, but do not use it at run time,
148     the library will be a bit bigger, but the additional run time overhead
149 nigel 93 is limited to testing the PCRE_UTF8 flag occasionally, so should not be
150     very big.
151 nigel 63
152 nigel 75 If PCRE is built with Unicode character property support (which implies
153 ph10 429 UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup-
154 nigel 75 ported. The available properties that can be tested are limited to the
155 ph10 429 general category properties such as Lu for an upper case letter or Nd
156     for a decimal number, the Unicode script names such as Arabic or Han,
157     and the derived properties Any and L&. A full list is given in the
158 nigel 87 pcrepattern documentation. Only the short names for properties are sup-
159 ph10 429 ported. For example, \p{L} matches a letter. Its Perl synonym, \p{Let-
160     ter}, is not supported. Furthermore, in Perl, many properties may
161     optionally be prefixed by "Is", for compatibility with Perl 5.6. PCRE
162 nigel 87 does not support this.
163 nigel 75
164 ph10 211 Validity of UTF-8 strings
165 nigel 63
166 ph10 429 When you set the PCRE_UTF8 flag, the strings passed as patterns and
167 ph10 211 subjects are (by default) checked for validity on entry to the relevant
168 ph10 429 functions. From release 7.3 of PCRE, the check is according the rules
169     of RFC 3629, which are themselves derived from the Unicode specifica-
170     tion. Earlier releases of PCRE followed the rules of RFC 2279, which
171     allows the full range of 31-bit values (0 to 0x7FFFFFFF). The current
172 ph10 211 check allows only values in the range U+0 to U+10FFFF, excluding U+D800
173     to U+DFFF.
174 nigel 63
175 ph10 429 The excluded code points are the "Low Surrogate Area" of Unicode, of
176     which the Unicode Standard says this: "The Low Surrogate Area does not
177     contain any character assignments, consequently no character code
178 ph10 211 charts or namelists are provided for this area. Surrogates are reserved
179 ph10 429 for use with UTF-16 and then must be used in pairs." The code points
180     that are encoded by UTF-16 pairs are available as independent code
181     points in the UTF-8 encoding. (In other words, the whole surrogate
182 ph10 211 thing is a fudge for UTF-16 which unfortunately messes up UTF-8.)
183    
184 ph10 429 If an invalid UTF-8 string is passed to PCRE, an error return
185 ph10 211 (PCRE_ERROR_BADUTF8) is given. In some situations, you may already know
186     that your strings are valid, and therefore want to skip these checks in
187     order to improve performance. If you set the PCRE_NO_UTF8_CHECK flag at
188 ph10 429 compile time or at run time, PCRE assumes that the pattern or subject
189     it is given (respectively) contains only valid UTF-8 codes. In this
190 ph10 211 case, it does not diagnose an invalid UTF-8 string.
191    
192 ph10 429 If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set,
193     what happens depends on why the string is invalid. If the string con-
194 ph10 211 forms to the "old" definition of UTF-8 (RFC 2279), it is processed as a
195 ph10 429 string of characters in the range 0 to 0x7FFFFFFF. In other words,
196 ph10 211 apart from the initial validity test, PCRE (when in UTF-8 mode) handles
197 ph10 429 strings according to the more liberal rules of RFC 2279. However, if
198     the string does not even conform to RFC 2279, the result is undefined.
199 ph10 211 Your program may crash.
200    
201 ph10 429 If you want to process strings of values in the full range 0 to
202     0x7FFFFFFF, encoded in a UTF-8-like manner as per the old RFC, you can
203 ph10 211 set PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in
204     this situation, you will have to apply your own validity check.
205    
206     General comments about UTF-8 mode
207    
208 ph10 429 1. An unbraced hexadecimal escape sequence (such as \xb3) matches a
209 nigel 87 two-byte UTF-8 character if the value is greater than 127.
210 nigel 63
211 ph10 429 2. Octal numbers up to \777 are recognized, and match two-byte UTF-8
212 nigel 91 characters for values greater than \177.
213    
214 ph10 429 3. Repeat quantifiers apply to complete UTF-8 characters, not to indi-
215 nigel 73 vidual bytes, for example: \x{100}{3}.
216 nigel 63
217 ph10 429 4. The dot metacharacter matches one UTF-8 character instead of a sin-
218 nigel 75 gle byte.
219 nigel 63
220 ph10 429 5. The escape sequence \C can be used to match a single byte in UTF-8
221     mode, but its use can lead to some strange effects. This facility is
222 nigel 77 not available in the alternative matching function, pcre_dfa_exec().
223 nigel 63
224 ph10 429 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly
225 ph10 518 test characters of any code value, but, by default, the characters that
226     PCRE recognizes as digits, spaces, or word characters remain the same
227     set as before, all with values less than 256. This remains true even
228     when PCRE is built to include Unicode property support, because to do
229 ph10 567 otherwise would slow down PCRE in many common cases. Note in particular
230     that this applies to \b and \B, because they are defined in terms of \w
231     and \W. If you really want to test for a wider sense of, say, "digit",
232     you can use explicit Unicode property tests such as \p{Nd}. Alterna-
233     tively, if you set the PCRE_UCP option, the way that the character
234     escapes work is changed so that Unicode properties are used to deter-
235     mine which characters match. There are more details in the section on
236     generic character types in the pcrepattern documentation.
237 nigel 63
238 ph10 429 7. Similarly, characters that match the POSIX named character classes
239 ph10 518 are all low-valued characters, unless the PCRE_UCP option is set.
240 nigel 63
241 ph10 572 8. However, the horizontal and vertical whitespace matching escapes
242     (\h, \H, \v, and \V) do match all the appropriate Unicode characters,
243     whether or not PCRE_UCP is set.
244 ph10 182
245 ph10 429 9. Case-insensitive matching applies only to characters whose values
246     are less than 128, unless PCRE is built with Unicode property support.
247     Even when Unicode property support is available, PCRE still uses its
248     own character tables when checking the case of low-valued characters,
249     so as not to degrade performance. The Unicode property information is
250 ph10 572 used only for characters with higher values. Furthermore, PCRE supports
251     case-insensitive matching only when there is a one-to-one mapping
252     between a letter's cases. There are a small number of many-to-one map-
253     pings in Unicode; these are not supported by PCRE.
254 nigel 63
255    
256     AUTHOR
257    
258 nigel 77 Philip Hazel
259 ph10 99 University Computing Service
260 nigel 93 Cambridge CB2 3QH, England.
261 nigel 63
262 ph10 572 Putting an actual email address here seems to have been a spam magnet,
263     so I've taken it away. If you want to email me, use my two initials,
264 ph10 153 followed by the two digits 10, at the domain cam.ac.uk.
265 nigel 77
266 nigel 63
267 ph10 99 REVISION
268 nigel 63
269 ph10 572 Last updated: 13 November 2010
270 ph10 507 Copyright (c) 1997-2010 University of Cambridge.
271 ph10 99 ------------------------------------------------------------------------------
272 ph10 567
273    
274 nigel 79 PCREBUILD(3) PCREBUILD(3)
275 nigel 63
276 nigel 79
277 nigel 73 NAME
278     PCRE - Perl-compatible regular expressions
279    
280 nigel 77
281 nigel 63 PCRE BUILD-TIME OPTIONS
282    
283 nigel 73 This document describes the optional features of PCRE that can be
284 ph10 261 selected when the library is compiled. It assumes use of the configure
285     script, where the optional features are selected or deselected by pro-
286     viding options to configure before running the make command. However,
287     the same options can be selected in both Unix-like and non-Unix-like
288 ph10 453 environments using the GUI facility of cmake-gui if you are using CMake
289     instead of configure to build PCRE.
290 nigel 63
291 ph10 453 There is a lot more information about building PCRE in non-Unix-like
292     environments in the file called NON_UNIX_USE, which is part of the PCRE
293     distribution. You should consult this file as well as the README file
294     if you are building in a non-Unix-like environment.
295    
296 ph10 261 The complete list of options for configure (which includes the standard
297 ph10 453 ones such as the selection of the installation directory) can be
298 ph10 261 obtained by running
299    
300 nigel 73 ./configure --help
301 nigel 63
302 ph10 453 The following sections include descriptions of options whose names
303 ph10 128 begin with --enable or --disable. These settings specify changes to the
304 ph10 453 defaults for the configure command. Because of the way that configure
305     works, --enable and --disable always come in pairs, so the complemen-
306     tary option always exists as well, but as it specifies the default, it
307 ph10 128 is not described.
308 nigel 63
309    
310 nigel 83 C++ SUPPORT
311    
312     By default, the configure script will search for a C++ compiler and C++
313     header files. If it finds them, it automatically builds the C++ wrapper
314     library for PCRE. You can disable this by adding
315    
316     --disable-cpp
317    
318     to the configure command.
319    
320    
321 nigel 63 UTF-8 SUPPORT
322    
323 ph10 392 To build PCRE with support for UTF-8 Unicode character strings, add
324 nigel 63
325 nigel 73 --enable-utf8
326 nigel 63
327 ph10 453 to the configure command. Of itself, this does not make PCRE treat
328     strings as UTF-8. As well as compiling PCRE with this option, you also
329     have have to set the PCRE_UTF8 option when you call the pcre_compile()
330 ph10 461 or pcre_compile2() functions.
331 nigel 63
332 ph10 453 If you set --enable-utf8 when compiling in an EBCDIC environment, PCRE
333 ph10 392 expects its input to be either ASCII or UTF-8 (depending on the runtime
334 ph10 453 option). It is not possible to support both EBCDIC and UTF-8 codes in
335     the same version of the library. Consequently, --enable-utf8 and
336 ph10 392 --enable-ebcdic are mutually exclusive.
337 nigel 63
338 ph10 392
339 nigel 75 UNICODE CHARACTER PROPERTY SUPPORT
340    
341 ph10 453 UTF-8 support allows PCRE to process character values greater than 255
342     in the strings that it handles. On its own, however, it does not pro-
343 nigel 75 vide any facilities for accessing the properties of such characters. If
344 ph10 453 you want to be able to use the pattern escapes \P, \p, and \X, which
345 nigel 75 refer to Unicode character properties, you must add
346    
347     --enable-unicode-properties
348    
349 ph10 453 to the configure command. This implies UTF-8 support, even if you have
350 nigel 75 not explicitly requested it.
351    
352 ph10 453 Including Unicode property support adds around 30K of tables to the
353     PCRE library. Only the general category properties such as Lu and Nd
354 ph10 128 are supported. Details are given in the pcrepattern documentation.
355 nigel 75
356    
357 nigel 63 CODE VALUE OF NEWLINE
358    
359 ph10 453 By default, PCRE interprets the linefeed (LF) character as indicating
360     the end of a line. This is the normal newline character on Unix-like
361     systems. You can compile PCRE to use carriage return (CR) instead, by
362 ph10 392 adding
363 nigel 63
364 nigel 73 --enable-newline-is-cr
365 nigel 63
366 ph10 453 to the configure command. There is also a --enable-newline-is-lf
367 nigel 91 option, which explicitly specifies linefeed as the newline character.
368 nigel 63
369 nigel 91 Alternatively, you can specify that line endings are to be indicated by
370     the two character sequence CRLF. If you want this, add
371 nigel 63
372 nigel 91 --enable-newline-is-crlf
373    
374 nigel 93 to the configure command. There is a fourth option, specified by
375 nigel 91
376 ph10 150 --enable-newline-is-anycrlf
377    
378 ph10 453 which causes PCRE to recognize any of the three sequences CR, LF, or
379 ph10 150 CRLF as indicating a line ending. Finally, a fifth option, specified by
380    
381 nigel 93 --enable-newline-is-any
382 nigel 91
383 ph10 150 causes PCRE to recognize any Unicode newline sequence.
384 nigel 93
385 ph10 453 Whatever line ending convention is selected when PCRE is built can be
386     overridden when the library functions are called. At build time it is
387 nigel 93 conventional to use the standard for your operating system.
388    
389    
390 ph10 231 WHAT \R MATCHES
391    
392 ph10 453 By default, the sequence \R in a pattern matches any Unicode newline
393     sequence, whatever has been selected as the line ending sequence. If
394 ph10 231 you specify
395    
396     --enable-bsr-anycrlf
397    
398 ph10 453 the default is changed so that \R matches only CR, LF, or CRLF. What-
399     ever is selected when PCRE is built can be overridden when the library
400 ph10 231 functions are called.
401    
402    
403 nigel 63 BUILDING SHARED AND STATIC LIBRARIES
404    
405 ph10 453 The PCRE building process uses libtool to build both shared and static
406     Unix libraries by default. You can suppress one of these by adding one
407 nigel 73 of
408 nigel 63
409 nigel 73 --disable-shared
410     --disable-static
411 nigel 63
412 nigel 73 to the configure command, as required.
413 nigel 63
414    
415     POSIX MALLOC USAGE
416    
417 nigel 75 When PCRE is called through the POSIX interface (see the pcreposix doc-
418 ph10 453 umentation), additional working storage is required for holding the
419     pointers to capturing substrings, because PCRE requires three integers
420     per substring, whereas the POSIX interface provides only two. If the
421 nigel 73 number of expected substrings is small, the wrapper function uses space
422     on the stack, because this is faster than using malloc() for each call.
423     The default threshold above which the stack is no longer used is 10; it
424     can be changed by adding a setting such as
425 nigel 63
426 nigel 73 --with-posix-malloc-threshold=20
427 nigel 63
428 nigel 73 to the configure command.
429 nigel 63
430    
431     HANDLING VERY LARGE PATTERNS
432    
433 ph10 453 Within a compiled pattern, offset values are used to point from one
434     part to another (for example, from an opening parenthesis to an alter-
435     nation metacharacter). By default, two-byte values are used for these
436     offsets, leading to a maximum size for a compiled pattern of around
437     64K. This is sufficient to handle all but the most gigantic patterns.
438 ph10 461 Nevertheless, some people do want to process truyl enormous patterns,
439     so it is possible to compile PCRE to use three-byte or four-byte off-
440     sets by adding a setting such as
441 nigel 63
442 nigel 73 --with-link-size=3
443 nigel 63
444 ph10 453 to the configure command. The value given must be 2, 3, or 4. Using
445     longer offsets slows down the operation of PCRE because it has to load
446 nigel 73 additional bytes when handling them.
447 nigel 63
448    
449 nigel 73 AVOIDING EXCESSIVE STACK USAGE
450    
451 nigel 77 When matching with the pcre_exec() function, PCRE implements backtrack-
452 ph10 453 ing by making recursive calls to an internal function called match().
453     In environments where the size of the stack is limited, this can se-
454     verely limit PCRE's operation. (The Unix environment does not usually
455 nigel 91 suffer from this problem, but it may sometimes be necessary to increase
456 ph10 453 the maximum stack size. There is a discussion in the pcrestack docu-
457     mentation.) An alternative approach to recursion that uses memory from
458     the heap to remember data, instead of using recursive function calls,
459     has been implemented to work round the problem of limited stack size.
460 nigel 91 If you want to build a version of PCRE that works this way, add
461 nigel 73
462     --disable-stack-for-recursion
463    
464 ph10 453 to the configure command. With this configuration, PCRE will use the
465     pcre_stack_malloc and pcre_stack_free variables to call memory manage-
466     ment functions. By default these point to malloc() and free(), but you
467 ph10 461 can replace the pointers so that your own functions are used instead.
468 nigel 73
469 ph10 453 Separate functions are provided rather than using pcre_malloc and
470     pcre_free because the usage is very predictable: the block sizes
471     requested are always the same, and the blocks are always freed in
472     reverse order. A calling program might be able to implement optimized
473     functions that perform better than malloc() and free(). PCRE runs
474 ph10 182 noticeably more slowly when built in this way. This option affects only
475 ph10 461 the pcre_exec() function; it is not relevant for pcre_dfa_exec().
476 nigel 73
477 ph10 182
478 nigel 91 LIMITING PCRE RESOURCE USAGE
479    
480 ph10 461 Internally, PCRE has a function called match(), which it calls repeat-
481     edly (sometimes recursively) when matching a pattern with the
482     pcre_exec() function. By controlling the maximum number of times this
483     function may be called during a single matching operation, a limit can
484     be placed on the resources used by a single call to pcre_exec(). The
485     limit can be changed at run time, as described in the pcreapi documen-
486     tation. The default is 10 million, but this can be changed by adding a
487 nigel 91 setting such as
488    
489     --with-match-limit=500000
490    
491 ph10 461 to the configure command. This setting has no effect on the
492 nigel 91 pcre_dfa_exec() matching function.
493    
494 ph10 461 In some environments it is desirable to limit the depth of recursive
495 nigel 91 calls of match() more strictly than the total number of calls, in order
496 ph10 461 to restrict the maximum amount of stack (or heap, if --disable-stack-
497 nigel 91 for-recursion is specified) that is used. A second limit controls this;
498 ph10 461 it defaults to the value that is set for --with-match-limit, which
499     imposes no additional constraints. However, you can set a lower limit
500 nigel 91 by adding, for example,
501    
502     --with-match-limit-recursion=10000
503    
504 ph10 461 to the configure command. This value can also be overridden at run
505 nigel 91 time.
506    
507    
508 ph10 128 CREATING CHARACTER TABLES AT BUILD TIME
509    
510 ph10 461 PCRE uses fixed tables for processing characters whose code values are
511     less than 256. By default, PCRE is built with a set of tables that are
512     distributed in the file pcre_chartables.c.dist. These tables are for
513 ph10 128 ASCII codes only. If you add
514    
515     --enable-rebuild-chartables
516    
517 ph10 461 to the configure command, the distributed tables are no longer used.
518     Instead, a program called dftables is compiled and run. This outputs
519 ph10 128 the source for new set of tables, created in the default locale of your
520     C runtime system. (This method of replacing the tables does not work if
521 ph10 461 you are cross compiling, because dftables is run on the local host. If
522     you need to create alternative tables when cross compiling, you will
523 ph10 128 have to do so "by hand".)
524    
525    
526 nigel 73 USING EBCDIC CODE
527    
528 ph10 461 PCRE assumes by default that it will run in an environment where the
529     character code is ASCII (or Unicode, which is a superset of ASCII).
530     This is the case for most computer operating systems. PCRE can, how-
531 ph10 197 ever, be compiled to run in an EBCDIC environment by adding
532 nigel 73
533     --enable-ebcdic
534    
535 ph10 128 to the configure command. This setting implies --enable-rebuild-charta-
536 ph10 461 bles. You should only use it if you know that you are in an EBCDIC
537     environment (for example, an IBM mainframe operating system). The
538 ph10 392 --enable-ebcdic option is incompatible with --enable-utf8.
539 nigel 73
540 nigel 93
541 ph10 286 PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT
542    
543     By default, pcregrep reads all files as plain text. You can build it so
544     that it recognizes files whose names end in .gz or .bz2, and reads them
545     with libz or libbz2, respectively, by adding one or both of
546    
547     --enable-pcregrep-libz
548     --enable-pcregrep-libbz2
549    
550     to the configure command. These options naturally require that the rel-
551 ph10 461 evant libraries are installed on your system. Configuration will fail
552 ph10 286 if they are not.
553    
554    
555 ph10 289 PCRETEST OPTION FOR LIBREADLINE SUPPORT
556    
557     If you add
558    
559     --enable-pcretest-libreadline
560    
561 ph10 461 to the configure command, pcretest is linked with the libreadline
562     library, and when its input is from a terminal, it reads it using the
563 ph10 289 readline() function. This provides line-editing and history facilities.
564 ph10 461 Note that libreadline is GPL-licensed, so if you distribute a binary of
565 ph10 289 pcretest linked in this way, there may be licensing issues.
566    
567 ph10 461 Setting this option causes the -lreadline option to be added to the
568     pcretest build. In many operating environments with a sytem-installed
569 ph10 345 libreadline this is sufficient. However, in some environments (e.g. if
570 ph10 461 an unmodified distribution version of readline is in use), some extra
571     configuration may be necessary. The INSTALL file for libreadline says
572 ph10 345 this:
573 ph10 289
574 ph10 345 "Readline uses the termcap functions, but does not link with the
575     termcap or curses library itself, allowing applications which link
576     with readline the to choose an appropriate library."
577    
578 ph10 461 If your environment has not been set up so that an appropriate library
579 ph10 345 is automatically included, you may need to add something like
580    
581     LIBS="-ncurses"
582    
583     immediately before the configure command.
584    
585    
586 nigel 93 SEE ALSO
587    
588     pcreapi(3), pcre_config(3).
589    
590 nigel 63
591 ph10 99 AUTHOR
592 nigel 63
593 ph10 99 Philip Hazel
594     University Computing Service
595     Cambridge CB2 3QH, England.
596    
597    
598     REVISION
599    
600 ph10 461 Last updated: 29 September 2009
601 ph10 392 Copyright (c) 1997-2009 University of Cambridge.
602 ph10 99 ------------------------------------------------------------------------------
603 ph10 567
604    
605 nigel 79 PCREMATCHING(3) PCREMATCHING(3)
606 nigel 63
607 nigel 79
608 nigel 77 NAME
609     PCRE - Perl-compatible regular expressions
610 nigel 73
611 nigel 77
612     PCRE MATCHING ALGORITHMS
613    
614     This document describes the two different algorithms that are available
615     in PCRE for matching a compiled regular expression against a given sub-
616     ject string. The "standard" algorithm is the one provided by the
617     pcre_exec() function. This works in the same was as Perl's matching
618     function, and provides a Perl-compatible matching operation.
619    
620     An alternative algorithm is provided by the pcre_dfa_exec() function;
621     this operates in a different way, and is not Perl-compatible. It has
622     advantages and disadvantages compared with the standard algorithm, and
623     these are described below.
624    
625     When there is only one possible way in which a given subject string can
626     match a pattern, the two algorithms give the same answer. A difference
627     arises, however, when there are multiple possibilities. For example, if
628     the pattern
629    
630     ^<.*>
631    
632     is matched against the string
633    
634     <something> <something else> <something further>
635    
636     there are three possible answers. The standard algorithm finds only one
637 nigel 93 of them, whereas the alternative algorithm finds all three.
638 nigel 77
639    
640     REGULAR EXPRESSIONS AS TREES
641    
642     The set of strings that are matched by a regular expression can be rep-
643     resented as a tree structure. An unlimited repetition in the pattern
644     makes the tree of infinite size, but it is still a tree. Matching the
645     pattern to a given subject string (from a given starting point) can be
646 nigel 91 thought of as a search of the tree. There are two ways to search a
647     tree: depth-first and breadth-first, and these correspond to the two
648     matching algorithms provided by PCRE.
649 nigel 77
650    
651     THE STANDARD MATCHING ALGORITHM
652    
653 ph10 148 In the terminology of Jeffrey Friedl's book "Mastering Regular Expres-
654     sions", the standard algorithm is an "NFA algorithm". It conducts a
655 nigel 77 depth-first search of the pattern tree. That is, it proceeds along a
656     single path through the tree, checking that the subject matches what is
657     required. When there is a mismatch, the algorithm tries any alterna-
658     tives at the current point, and if they all fail, it backs up to the
659     previous branch point in the tree, and tries the next alternative
660     branch at that level. This often involves backing up (moving to the
661     left) in the subject string as well. The order in which repetition
662     branches are tried is controlled by the greedy or ungreedy nature of
663     the quantifier.
664    
665     If a leaf node is reached, a matching string has been found, and at
666     that point the algorithm stops. Thus, if there is more than one possi-
667     ble match, this algorithm returns the first one that it finds. Whether
668     this is the shortest, the longest, or some intermediate length depends
669     on the way the greedy and ungreedy repetition quantifiers are specified
670     in the pattern.
671    
672     Because it ends up with a single path through the tree, it is rela-
673     tively straightforward for this algorithm to keep track of the sub-
674     strings that are matched by portions of the pattern in parentheses.
675     This provides support for capturing parentheses and back references.
676    
677    
678 nigel 93 THE ALTERNATIVE MATCHING ALGORITHM
679 nigel 77
680 nigel 93 This algorithm conducts a breadth-first search of the tree. Starting
681     from the first matching point in the subject, it scans the subject
682     string from left to right, once, character by character, and as it does
683     this, it remembers all the paths through the tree that represent valid
684     matches. In Friedl's terminology, this is a kind of "DFA algorithm",
685     though it is not implemented as a traditional finite state machine (it
686     keeps multiple states active simultaneously).
687 nigel 77
688 ph10 461 Although the general principle of this matching algorithm is that it
689     scans the subject string only once, without backtracking, there is one
690     exception: when a lookaround assertion is encountered, the characters
691     following or preceding the current point have to be independently
692     inspected.
693    
694 nigel 93 The scan continues until either the end of the subject is reached, or
695     there are no more unterminated paths. At this point, terminated paths
696     represent the different matching possibilities (if there are none, the
697     match has failed). Thus, if there is more than one possible match,
698 nigel 77 this algorithm finds all of them, and in particular, it finds the long-
699 ph10 572 est. The matches are returned in decreasing order of length. There is
700     an option to stop the algorithm after the first match (which is neces-
701     sarily the shortest) is found.
702 nigel 77
703     Note that all the matches that are found start at the same point in the
704     subject. If the pattern
705    
706 ph10 572 cat(er(pillar)?)?
707 nigel 77
708 ph10 572 is matched against the string "the caterpillar catchment", the result
709     will be the three strings "caterpillar", "cater", and "cat" that start
710     at the fifth character of the subject. The algorithm does not automati-
711     cally move on to find matches that start at later positions.
712 nigel 77
713     There are a number of features of PCRE regular expressions that are not
714 nigel 93 supported by the alternative matching algorithm. They are as follows:
715 nigel 77
716 ph10 572 1. Because the algorithm finds all possible matches, the greedy or
717     ungreedy nature of repetition quantifiers is not relevant. Greedy and
718 nigel 93 ungreedy quantifiers are treated in exactly the same way. However, pos-
719 ph10 572 sessive quantifiers can make a difference when what follows could also
720 nigel 93 match what is quantified, for example in a pattern like this:
721 nigel 77
722 nigel 93 ^a++\w!
723    
724 ph10 572 This pattern matches "aaab!" but not "aaa!", which would be matched by
725     a non-possessive quantifier. Similarly, if an atomic group is present,
726     it is matched as if it were a standalone pattern at the current point,
727     and the longest match is then "locked in" for the rest of the overall
728 nigel 93 pattern.
729    
730 nigel 77 2. When dealing with multiple paths through the tree simultaneously, it
731 ph10 572 is not straightforward to keep track of captured substrings for the
732     different matching possibilities, and PCRE's implementation of this
733 nigel 77 algorithm does not attempt to do this. This means that no captured sub-
734     strings are available.
735    
736 ph10 572 3. Because no substrings are captured, back references within the pat-
737 nigel 77 tern are not supported, and cause errors if encountered.
738    
739 ph10 572 4. For the same reason, conditional expressions that use a backrefer-
740     ence as the condition or test for a specific group recursion are not
741 nigel 93 supported.
742 nigel 77
743 ph10 572 5. Because many paths through the tree may be active, the \K escape
744 ph10 172 sequence, which resets the start of the match when encountered (but may
745 ph10 572 be on some paths and not on others), is not supported. It causes an
746 ph10 172 error if encountered.
747    
748 ph10 572 6. Callouts are supported, but the value of the capture_top field is
749 nigel 77 always 1, and the value of the capture_last field is always -1.
750    
751 ph10 572 7. The \C escape sequence, which (in the standard algorithm) matches a
752     single byte, even in UTF-8 mode, is not supported because the alterna-
753     tive algorithm moves through the subject string one character at a
754 nigel 93 time, for all active paths through the tree.
755 nigel 77
756 ph10 572 8. Except for (*FAIL), the backtracking control verbs such as (*PRUNE)
757     are not supported. (*FAIL) is supported, and behaves like a failing
758 ph10 345 negative assertion.
759 nigel 77
760 ph10 211
761 nigel 93 ADVANTAGES OF THE ALTERNATIVE ALGORITHM
762 nigel 77
763 ph10 572 Using the alternative matching algorithm provides the following advan-
764 nigel 93 tages:
765 nigel 77
766     1. All possible matches (at a single point in the subject) are automat-
767 ph10 572 ically found, and in particular, the longest match is found. To find
768 nigel 77 more than one match using the standard algorithm, you have to do kludgy
769     things with callouts.
770    
771 ph10 572 2. Because the alternative algorithm scans the subject string just
772     once, and never needs to backtrack, it is possible to pass very long
773     subject strings to the matching function in several pieces, checking
774     for partial matching each time. Although it is possible to do multi-
775     segment matching using the standard algorithm (pcre_exec()), by retain-
776     ing partially matched substrings, it is more complicated. The pcrepar-
777     tial documentation gives details of partial matching and discusses
778     multi-segment matching.
779 nigel 77
780    
781 nigel 93 DISADVANTAGES OF THE ALTERNATIVE ALGORITHM
782 nigel 77
783 nigel 93 The alternative algorithm suffers from a number of disadvantages:
784 nigel 77
785 ph10 453 1. It is substantially slower than the standard algorithm. This is
786     partly because it has to search for all possible matches, but is also
787 nigel 77 because it is less susceptible to optimization.
788    
789     2. Capturing parentheses and back references are not supported.
790    
791 nigel 93 3. Although atomic groups are supported, their use does not provide the
792     performance advantage that it does for the standard algorithm.
793 nigel 77
794    
795 ph10 99 AUTHOR
796 nigel 77
797 ph10 99 Philip Hazel
798     University Computing Service
799     Cambridge CB2 3QH, England.
800    
801    
802     REVISION
803    
804 ph10 572 Last updated: 17 November 2010
805 ph10 567 Copyright (c) 1997-2010 University of Cambridge.
806 ph10 99 ------------------------------------------------------------------------------
807 ph10 567
808    
809 nigel 79 PCREAPI(3) PCREAPI(3)
810 nigel 77
811 nigel 79
812 nigel 73 NAME
813     PCRE - Perl-compatible regular expressions
814    
815 nigel 77
816 nigel 75 PCRE NATIVE API
817 nigel 63
818 nigel 73 #include <pcre.h>
819 nigel 41
820 nigel 73 pcre *pcre_compile(const char *pattern, int options,
821     const char **errptr, int *erroffset,
822     const unsigned char *tableptr);
823 nigel 41
824 nigel 77 pcre *pcre_compile2(const char *pattern, int options,
825     int *errorcodeptr,
826     const char **errptr, int *erroffset,
827     const unsigned char *tableptr);
828    
829 nigel 73 pcre_extra *pcre_study(const pcre *code, int options,
830     const char **errptr);
831 nigel 41
832 nigel 73 int pcre_exec(const pcre *code, const pcre_extra *extra,
833     const char *subject, int length, int startoffset,
834     int options, int *ovector, int ovecsize);
835 nigel 41
836 nigel 77 int pcre_dfa_exec(const pcre *code, const pcre_extra *extra,
837     const char *subject, int length, int startoffset,
838     int options, int *ovector, int ovecsize,
839     int *workspace, int wscount);
840    
841 nigel 73 int pcre_copy_named_substring(const pcre *code,
842     const char *subject, int *ovector,
843     int stringcount, const char *stringname,
844     char *buffer, int buffersize);
845 nigel 63
846 nigel 73 int pcre_copy_substring(const char *subject, int *ovector,
847     int stringcount, int stringnumber, char *buffer,
848     int buffersize);
849 nigel 41
850 nigel 73 int pcre_get_named_substring(const pcre *code,
851     const char *subject, int *ovector,
852     int stringcount, const char *stringname,
853     const char **stringptr);
854 nigel 63
855 nigel 73 int pcre_get_stringnumber(const pcre *code,
856     const char *name);
857 nigel 63
858 nigel 91 int pcre_get_stringtable_entries(const pcre *code,
859     const char *name, char **first, char **last);
860    
861 nigel 73 int pcre_get_substring(const char *subject, int *ovector,
862     int stringcount, int stringnumber,
863     const char **stringptr);
864 nigel 41
865 nigel 73 int pcre_get_substring_list(const char *subject,
866     int *ovector, int stringcount, const char ***listptr);
867 nigel 41
868 nigel 73 void pcre_free_substring(const char *stringptr);
869 nigel 49
870 nigel 73 void pcre_free_substring_list(const char **stringptr);
871 nigel 49
872 nigel 73 const unsigned char *pcre_maketables(void);
873 nigel 41
874 nigel 73 int pcre_fullinfo(const pcre *code, const pcre_extra *extra,
875     int what, void *where);
876 nigel 43
877 nigel 73 int pcre_info(const pcre *code, int *optptr, int *firstcharptr);
878 nigel 63
879 nigel 77 int pcre_refcount(pcre *code, int adjust);
880    
881 nigel 73 int pcre_config(int what, void *where);
882 nigel 41
883 nigel 73 char *pcre_version(void);
884 nigel 63
885 nigel 73 void *(*pcre_malloc)(size_t);
886 nigel 41
887 nigel 73 void (*pcre_free)(void *);
888 nigel 41
889 nigel 73 void *(*pcre_stack_malloc)(size_t);
890 nigel 41
891 nigel 73 void (*pcre_stack_free)(void *);
892 nigel 41
893 nigel 73 int (*pcre_callout)(pcre_callout_block *);
894 nigel 41
895 nigel 73
896 nigel 75 PCRE API OVERVIEW
897 nigel 41
898 nigel 73 PCRE has its own native API, which is described in this document. There
899 nigel 93 are also some wrapper functions that correspond to the POSIX regular
900 nigel 77 expression API. These are described in the pcreposix documentation.
901     Both of these APIs define a set of C function calls. A C++ wrapper is
902     distributed with PCRE. It is documented in the pcrecpp page.
903 nigel 43
904 nigel 77 The native API C function prototypes are defined in the header file
905     pcre.h, and on Unix systems the library itself is called libpcre. It
906 nigel 75 can normally be accessed by adding -lpcre to the command for linking an
907     application that uses PCRE. The header file defines the macros
908     PCRE_MAJOR and PCRE_MINOR to contain the major and minor release num-
909     bers for the library. Applications can use these to include support
910     for different releases of PCRE.
911 nigel 41
912 ph10 535 In a Windows environment, if you want to statically link an application
913     program against a non-dll pcre.a file, you must define PCRE_STATIC
914     before including pcre.h or pcrecpp.h, because otherwise the pcre_mal-
915     loc() and pcre_free() exported functions will be declared
916     __declspec(dllimport), with unwanted results.
917    
918 nigel 77 The functions pcre_compile(), pcre_compile2(), pcre_study(), and
919     pcre_exec() are used for compiling and matching regular expressions in
920     a Perl-compatible manner. A sample program that demonstrates the sim-
921     plest way of using them is provided in the file called pcredemo.c in
922 ph10 429 the PCRE source distribution. A listing of this program is given in the
923     pcredemo documentation, and the pcresample documentation describes how
924     to compile and run it.
925 nigel 49
926 nigel 77 A second matching function, pcre_dfa_exec(), which is not Perl-compati-
927 ph10 429 ble, is also provided. This uses a different algorithm for the match-
928     ing. The alternative algorithm finds all possible matches (at a given
929 ph10 453 point in the subject), and scans the subject just once (unless there
930     are lookbehind assertions). However, this algorithm does not return
931     captured substrings. A description of the two matching algorithms and
932     their advantages and disadvantages is given in the pcrematching docu-
933     mentation.
934 nigel 63
935 ph10 453 In addition to the main compiling and matching functions, there are
936 nigel 77 convenience functions for extracting captured substrings from a subject
937     string that is matched by pcre_exec(). They are:
938    
939 nigel 73 pcre_copy_substring()
940     pcre_copy_named_substring()
941     pcre_get_substring()
942     pcre_get_named_substring()
943     pcre_get_substring_list()
944 nigel 75 pcre_get_stringnumber()
945 nigel 91 pcre_get_stringtable_entries()
946 nigel 63
947 nigel 73 pcre_free_substring() and pcre_free_substring_list() are also provided,
948     to free the memory used for extracted strings.
949 nigel 41
950 ph10 453 The function pcre_maketables() is used to build a set of character
951     tables in the current locale for passing to pcre_compile(),
952     pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is
953     provided for specialist use. Most commonly, no special tables are
954     passed, in which case internal tables that are generated when PCRE is
955 nigel 77 built are used.
956 nigel 49
957 ph10 453 The function pcre_fullinfo() is used to find out information about a
958     compiled pattern; pcre_info() is an obsolete version that returns only
959     some of the available information, but is retained for backwards com-
960     patibility. The function pcre_version() returns a pointer to a string
961 nigel 73 containing the version of PCRE and its date of release.
962 nigel 41
963 ph10 453 The function pcre_refcount() maintains a reference count in a data
964     block containing a compiled pattern. This is provided for the benefit
965 nigel 77 of object-oriented applications.
966    
967 ph10 453 The global variables pcre_malloc and pcre_free initially contain the
968     entry points of the standard malloc() and free() functions, respec-
969 nigel 73 tively. PCRE calls the memory management functions via these variables,
970 ph10 453 so a calling program can replace them if it wishes to intercept the
971 nigel 73 calls. This should be done before calling any PCRE functions.
972 nigel 41
973 ph10 453 The global variables pcre_stack_malloc and pcre_stack_free are also
974     indirections to memory management functions. These special functions
975     are used only when PCRE is compiled to use the heap for remembering
976 nigel 77 data, instead of recursive function calls, when running the pcre_exec()
977 ph10 453 function. See the pcrebuild documentation for details of how to do
978     this. It is a non-standard way of building PCRE, for use in environ-
979     ments that have limited stacks. Because of the greater use of memory
980     management, it runs more slowly. Separate functions are provided so
981     that special-purpose external code can be used for this case. When
982     used, these functions are always called in a stack-like manner (last
983     obtained, first freed), and always for memory blocks of the same size.
984     There is a discussion about PCRE's stack usage in the pcrestack docu-
985 nigel 91 mentation.
986 nigel 41
987 nigel 73 The global variable pcre_callout initially contains NULL. It can be set
988 ph10 453 by the caller to a "callout" function, which PCRE will then call at
989     specified points during a matching operation. Details are given in the
990 nigel 73 pcrecallout documentation.
991 nigel 41
992 nigel 73
993 nigel 91 NEWLINES
994    
995 ph10 453 PCRE supports five different conventions for indicating line breaks in
996     strings: a single CR (carriage return) character, a single LF (line-
997 ph10 150 feed) character, the two-character sequence CRLF, any of the three pre-
998 ph10 453 ceding, or any Unicode newline sequence. The Unicode newline sequences
999     are the three just mentioned, plus the single characters VT (vertical
1000     tab, U+000B), FF (formfeed, U+000C), NEL (next line, U+0085), LS (line
1001 ph10 150 separator, U+2028), and PS (paragraph separator, U+2029).
1002 nigel 93
1003 ph10 453 Each of the first three conventions is used by at least one operating
1004     system as its standard newline sequence. When PCRE is built, a default
1005     can be specified. The default default is LF, which is the Unix stan-
1006     dard. When PCRE is run, the default can be overridden, either when a
1007 nigel 93 pattern is compiled, or when it is matched.
1008    
1009 ph10 227 At compile time, the newline convention can be specified by the options
1010 ph10 453 argument of pcre_compile(), or it can be specified by special text at
1011 ph10 227 the start of the pattern itself; this overrides any other settings. See
1012     the pcrepattern page for details of the special character sequences.
1013    
1014 nigel 91 In the PCRE documentation the word "newline" is used to mean "the char-
1015 ph10 453 acter or pair of characters that indicate a line break". The choice of
1016     newline convention affects the handling of the dot, circumflex, and
1017 nigel 93 dollar metacharacters, the handling of #-comments in /x mode, and, when
1018 ph10 453 CRLF is a recognized line ending sequence, the match position advance-
1019 ph10 227 ment for a non-anchored pattern. There is more detail about this in the
1020 ph10 231 section on pcre_exec() options below.
1021 nigel 91
1022 ph10 453 The choice of newline convention does not affect the interpretation of
1023     the \n or \r escape sequences, nor does it affect what \R matches,
1024 ph10 231 which is controlled in a similar way, but by separate options.
1025 nigel 91
1026 ph10 231
1027 nigel 63 MULTITHREADING
1028    
1029 ph10 453 The PCRE functions can be used in multi-threading applications, with
1030 nigel 73 the proviso that the memory management functions pointed to by
1031     pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the
1032     callout function pointed to by pcre_callout, are shared by all threads.
1033 nigel 41
1034 ph10 453 The compiled form of a regular expression is not altered during match-
1035 nigel 73 ing, so the same compiled pattern can safely be used by several threads
1036     at once.
1037 nigel 41
1038    
1039 nigel 75 SAVING PRECOMPILED PATTERNS FOR LATER USE
1040    
1041     The compiled form of a regular expression can be saved and re-used at a
1042 ph10 453 later time, possibly by a different program, and even on a host other
1043     than the one on which it was compiled. Details are given in the
1044     pcreprecompile documentation. However, compiling a regular expression
1045     with one version of PCRE for use with a different version is not guar-
1046 ph10 155 anteed to work and may cause crashes.
1047 nigel 75
1048    
1049 nigel 63 CHECKING BUILD-TIME OPTIONS
1050 nigel 41
1051 nigel 73 int pcre_config(int what, void *where);
1052 nigel 63
1053 ph10 453 The function pcre_config() makes it possible for a PCRE client to dis-
1054 nigel 73 cover which optional features have been compiled into the PCRE library.
1055 ph10 453 The pcrebuild documentation has more details about these optional fea-
1056 nigel 73 tures.
1057 nigel 63
1058 ph10 453 The first argument for pcre_config() is an integer, specifying which
1059 nigel 73 information is required; the second argument is a pointer to a variable
1060 ph10 453 into which the information is placed. The following information is
1061 nigel 73 available:
1062 nigel 63
1063 nigel 73 PCRE_CONFIG_UTF8
1064 nigel 63
1065 ph10 453 The output is an integer that is set to one if UTF-8 support is avail-
1066 nigel 73 able; otherwise it is set to zero.
1067 nigel 63
1068 nigel 75 PCRE_CONFIG_UNICODE_PROPERTIES
1069    
1070 ph10 453 The output is an integer that is set to one if support for Unicode
1071 nigel 75 character properties is available; otherwise it is set to zero.
1072    
1073 nigel 73 PCRE_CONFIG_NEWLINE
1074 nigel 63
1075 ph10 453 The output is an integer whose value specifies the default character
1076     sequence that is recognized as meaning "newline". The four values that
1077 ph10 150 are supported are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF,
1078 ph10 453 and -1 for ANY. Though they are derived from ASCII, the same values
1079 ph10 392 are returned in EBCDIC environments. The default should normally corre-
1080     spond to the standard sequence for your operating system.
1081 nigel 63
1082 ph10 231 PCRE_CONFIG_BSR
1083    
1084     The output is an integer whose value indicates what character sequences
1085 ph10 453 the \R escape sequence matches by default. A value of 0 means that \R
1086     matches any Unicode line ending sequence; a value of 1 means that \R
1087 ph10 231 matches only CR, LF, or CRLF. The default can be overridden when a pat-
1088     tern is compiled or matched.
1089    
1090 nigel 73 PCRE_CONFIG_LINK_SIZE
1091 nigel 63
1092 ph10 453 The output is an integer that contains the number of bytes used for
1093 nigel 73 internal linkage in compiled regular expressions. The value is 2, 3, or
1094 ph10 453 4. Larger values allow larger regular expressions to be compiled, at
1095     the expense of slower matching. The default value of 2 is sufficient
1096     for all but the most massive patterns, since it allows the compiled
1097 nigel 73 pattern to be up to 64K in size.
1098 nigel 63
1099 nigel 73 PCRE_CONFIG_POSIX_MALLOC_THRESHOLD
1100 nigel 63
1101 ph10 453 The output is an integer that contains the threshold above which the
1102     POSIX interface uses malloc() for output vectors. Further details are
1103 nigel 73 given in the pcreposix documentation.
1104 nigel 63
1105 nigel 73 PCRE_CONFIG_MATCH_LIMIT
1106 nigel 63
1107 ph10 453 The output is a long integer that gives the default limit for the num-
1108     ber of internal matching function calls in a pcre_exec() execution.
1109 ph10 392 Further details are given with pcre_exec() below.
1110 nigel 63
1111 nigel 87 PCRE_CONFIG_MATCH_LIMIT_RECURSION
1112    
1113 ph10 392 The output is a long integer that gives the default limit for the depth
1114 ph10 453 of recursion when calling the internal matching function in a
1115     pcre_exec() execution. Further details are given with pcre_exec()
1116 ph10 392 below.
1117 nigel 87
1118 nigel 73 PCRE_CONFIG_STACKRECURSE
1119 nigel 63
1120 ph10 453 The output is an integer that is set to one if internal recursion when
1121 nigel 77 running pcre_exec() is implemented by recursive function calls that use
1122 ph10 453 the stack to remember their state. This is the usual way that PCRE is
1123 nigel 77 compiled. The output is zero if PCRE was compiled to use blocks of data
1124 ph10 453 on the heap instead of recursive function calls. In this case,
1125     pcre_stack_malloc and pcre_stack_free are called to manage memory
1126 nigel 77 blocks on the heap, thus avoiding the use of the stack.
1127 nigel 73
1128    
1129 nigel 41 COMPILING A PATTERN
1130 nigel 63
1131 nigel 73 pcre *pcre_compile(const char *pattern, int options,
1132     const char **errptr, int *erroffset,
1133     const unsigned char *tableptr);
1134 nigel 63
1135 nigel 77 pcre *pcre_compile2(const char *pattern, int options,
1136     int *errorcodeptr,
1137     const char **errptr, int *erroffset,
1138     const unsigned char *tableptr);
1139 nigel 41
1140 nigel 77 Either of the functions pcre_compile() or pcre_compile2() can be called
1141     to compile a pattern into an internal form. The only difference between
1142 ph10 453 the two interfaces is that pcre_compile2() has an additional argument,
1143 ph10 461 errorcodeptr, via which a numerical error code can be returned. To
1144     avoid too much repetition, we refer just to pcre_compile() below, but
1145     the information applies equally to pcre_compile2().
1146 nigel 77
1147     The pattern is a C string terminated by a binary zero, and is passed in
1148 ph10 453 the pattern argument. A pointer to a single block of memory that is
1149     obtained via pcre_malloc is returned. This contains the compiled code
1150 nigel 77 and related data. The pcre type is defined for the returned block; this
1151     is a typedef for a structure whose contents are not externally defined.
1152 nigel 91 It is up to the caller to free the memory (via pcre_free) when it is no
1153     longer required.
1154 nigel 77
1155 ph10 453 Although the compiled code of a PCRE regex is relocatable, that is, it
1156 nigel 73 does not depend on memory location, the complete pcre data block is not
1157 ph10 453 fully relocatable, because it may contain a copy of the tableptr argu-
1158 nigel 75 ment, which is an address (see below).
1159 nigel 41
1160 nigel 93 The options argument contains various bit settings that affect the com-
1161 ph10 453 pilation. It should be zero if no options are required. The available
1162     options are described below. Some of them (in particular, those that
1163 ph10 461 are compatible with Perl, but some others as well) can also be set and
1164 ph10 453 unset from within the pattern (see the detailed description in the
1165     pcrepattern documentation). For those options that can be different in
1166     different parts of the pattern, the contents of the options argument
1167 ph10 461 specifies their settings at the start of compilation and execution. The
1168     PCRE_ANCHORED, PCRE_BSR_xxx, and PCRE_NEWLINE_xxx options can be set at
1169     the time of matching as well as at compile time.
1170 nigel 41
1171 nigel 73 If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise,
1172 ph10 453 if compilation of a pattern fails, pcre_compile() returns NULL, and
1173 nigel 73 sets the variable pointed to by errptr to point to a textual error mes-
1174 nigel 87 sage. This is a static string that is part of the library. You must not
1175 ph10 572 try to free it. The offset from the start of the pattern to the byte
1176     that was being processed when the error was discovered is placed in the
1177     variable pointed to by erroffset, which must not be NULL. If it is, an
1178     immediate error is given. Some errors are not detected until checks are
1179     carried out when the whole pattern has been scanned; in this case the
1180     offset is set to the end of the pattern.
1181 nigel 53
1182 ph10 572 Note that the offset is in bytes, not characters, even in UTF-8 mode.
1183     It may point into the middle of a UTF-8 character (for example, when
1184     PCRE_ERROR_BADUTF8 is returned for an invalid UTF-8 string).
1185    
1186 ph10 453 If pcre_compile2() is used instead of pcre_compile(), and the error-
1187     codeptr argument is not NULL, a non-zero error code number is returned
1188     via this argument in the event of an error. This is in addition to the
1189 nigel 77 textual error message. Error codes and messages are listed below.
1190    
1191 ph10 453 If the final argument, tableptr, is NULL, PCRE uses a default set of
1192     character tables that are built when PCRE is compiled, using the
1193     default C locale. Otherwise, tableptr must be an address that is the
1194     result of a call to pcre_maketables(). This value is stored with the
1195     compiled pattern, and used again by pcre_exec(), unless another table
1196 nigel 75 pointer is passed to it. For more discussion, see the section on locale
1197     support below.
1198 nigel 53
1199 ph10 453 This code fragment shows a typical straightforward call to pcre_com-
1200 nigel 73 pile():
1201 nigel 41
1202 nigel 73 pcre *re;
1203     const char *error;
1204     int erroffset;
1205     re = pcre_compile(
1206     "^A.*Z", /* the pattern */
1207     0, /* default options */
1208     &error, /* for error message */
1209     &erroffset, /* for error offset */
1210     NULL); /* use default character tables */
1211 nigel 41
1212 ph10 453 The following names for option bits are defined in the pcre.h header
1213 nigel 75 file:
1214 nigel 41
1215 nigel 73 PCRE_ANCHORED
1216 nigel 41
1217 nigel 73 If this bit is set, the pattern is forced to be "anchored", that is, it
1218 ph10 453 is constrained to match only at the first matching point in the string
1219     that is being searched (the "subject string"). This effect can also be
1220     achieved by appropriate constructs in the pattern itself, which is the
1221 nigel 73 only way to do it in Perl.
1222 nigel 41
1223 nigel 75 PCRE_AUTO_CALLOUT
1224    
1225     If this bit is set, pcre_compile() automatically inserts callout items,
1226 ph10 453 all with number 255, before each pattern item. For discussion of the
1227 nigel 75 callout facility, see the pcrecallout documentation.
1228    
1229 ph10 231 PCRE_BSR_ANYCRLF
1230     PCRE_BSR_UNICODE
1231    
1232     These options (which are mutually exclusive) control what the \R escape
1233 ph10 453 sequence matches. The choice is either to match only CR, LF, or CRLF,
1234 ph10 231 or to match any Unicode newline sequence. The default is specified when
1235     PCRE is built. It can be overridden from within the pattern, or by set-
1236     ting an option when a compiled pattern is matched.
1237    
1238 nigel 73 PCRE_CASELESS
1239 nigel 41
1240 ph10 453 If this bit is set, letters in the pattern match both upper and lower
1241     case letters. It is equivalent to Perl's /i option, and it can be
1242     changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE
1243     always understands the concept of case for characters whose values are
1244     less than 128, so caseless matching is always possible. For characters
1245     with higher values, the concept of case is supported if PCRE is com-
1246     piled with Unicode property support, but not otherwise. If you want to
1247     use caseless matching for characters 128 and above, you must ensure
1248     that PCRE is compiled with Unicode property support as well as with
1249 nigel 77 UTF-8 support.
1250 nigel 41
1251 nigel 73 PCRE_DOLLAR_ENDONLY
1252 nigel 41
1253 ph10 453 If this bit is set, a dollar metacharacter in the pattern matches only
1254     at the end of the subject string. Without this option, a dollar also
1255     matches immediately before a newline at the end of the string (but not
1256     before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored
1257     if PCRE_MULTILINE is set. There is no equivalent to this option in
1258 nigel 91 Perl, and no way to set it within a pattern.
1259 nigel 41
1260 nigel 73 PCRE_DOTALL
1261 nigel 41
1262 ph10 572 If this bit is set, a dot metacharacter in the pattern matches a char-
1263     acter of any value, including one that indicates a newline. However, it
1264     only ever matches one character, even if newlines are coded as CRLF.
1265     Without this option, a dot does not match when the current position is
1266     at a newline. This option is equivalent to Perl's /s option, and it can
1267     be changed within a pattern by a (?s) option setting. A negative class
1268     such as [^a] always matches newline characters, independent of the set-
1269     ting of this option.
1270 nigel 63
1271 nigel 91 PCRE_DUPNAMES
1272    
1273 ph10 453 If this bit is set, names used to identify capturing subpatterns need
1274 nigel 91 not be unique. This can be helpful for certain types of pattern when it
1275 ph10 453 is known that only one instance of the named subpattern can ever be
1276     matched. There are more details of named subpatterns below; see also
1277 nigel 91 the pcrepattern documentation.
1278    
1279 nigel 73 PCRE_EXTENDED
1280 nigel 41
1281 ph10 453 If this bit is set, whitespace data characters in the pattern are
1282 nigel 77 totally ignored except when escaped or inside a character class. White-
1283     space does not include the VT character (code 11). In addition, charac-
1284     ters between an unescaped # outside a character class and the next new-
1285 ph10 453 line, inclusive, are also ignored. This is equivalent to Perl's /x
1286     option, and it can be changed within a pattern by a (?x) option set-
1287 nigel 91 ting.
1288 nigel 41
1289 ph10 572 Which characters are interpreted as newlines is controlled by the
1290     options passed to pcre_compile() or by a special sequence at the start
1291     of the pattern, as described in the section entitled "Newline conven-
1292     tions" in the pcrepattern documentation. Note that the end of this type
1293     of comment is a literal newline sequence in the pattern; escape
1294     sequences that happen to represent a newline do not count.
1295 nigel 41
1296 ph10 572 This option makes it possible to include comments inside complicated
1297     patterns. Note, however, that this applies only to data characters.
1298     Whitespace characters may never appear within special character
1299     sequences in a pattern, for example within the sequence (?( that intro-
1300     duces a conditional subpattern.
1301    
1302 nigel 73 PCRE_EXTRA
1303 nigel 41
1304 ph10 572 This option was invented in order to turn on additional functionality
1305     of PCRE that is incompatible with Perl, but it is currently of very
1306     little use. When set, any backslash in a pattern that is followed by a
1307     letter that has no special meaning causes an error, thus reserving
1308     these combinations for future expansion. By default, as in Perl, a
1309     backslash followed by a letter with no special meaning is treated as a
1310 ph10 518 literal. (Perl can, however, be persuaded to give an error for this, by
1311 ph10 572 running it with the -w option.) There are at present no other features
1312     controlled by this option. It can also be set by a (?X) option setting
1313 ph10 518 within a pattern.
1314 nigel 41
1315 nigel 77 PCRE_FIRSTLINE
1316    
1317 ph10 572 If this option is set, an unanchored pattern is required to match
1318     before or at the first newline in the subject string, though the
1319 nigel 91 matched text may continue over the newline.
1320 nigel 77
1321 ph10 345 PCRE_JAVASCRIPT_COMPAT
1322    
1323     If this option is set, PCRE's behaviour is changed in some ways so that
1324 ph10 572 it is compatible with JavaScript rather than Perl. The changes are as
1325 ph10 345 follows:
1326    
1327 ph10 572 (1) A lone closing square bracket in a pattern causes a compile-time
1328     error, because this is illegal in JavaScript (by default it is treated
1329 ph10 345 as a data character). Thus, the pattern AB]CD becomes illegal when this
1330     option is set.
1331    
1332 ph10 572 (2) At run time, a back reference to an unset subpattern group matches
1333     an empty string (by default this causes the current matching alterna-
1334     tive to fail). A pattern such as (\1)(a) succeeds when this option is
1335     set (assuming it can find an "a" in the subject), whereas it fails by
1336 ph10 345 default, for Perl compatibility.
1337    
1338 nigel 73 PCRE_MULTILINE
1339 nigel 41
1340 ph10 572 By default, PCRE treats the subject string as consisting of a single
1341     line of characters (even if it actually contains newlines). The "start
1342     of line" metacharacter (^) matches only at the start of the string,
1343     while the "end of line" metacharacter ($) matches only at the end of
1344 nigel 75 the string, or before a terminating newline (unless PCRE_DOLLAR_ENDONLY
1345     is set). This is the same as Perl.
1346 nigel 63
1347 ph10 572 When PCRE_MULTILINE it is set, the "start of line" and "end of line"
1348     constructs match immediately following or immediately before internal
1349     newlines in the subject string, respectively, as well as at the very
1350     start and end. This is equivalent to Perl's /m option, and it can be
1351 nigel 91 changed within a pattern by a (?m) option setting. If there are no new-
1352 ph10 572 lines in a subject string, or no occurrences of ^ or $ in a pattern,
1353 nigel 73 setting PCRE_MULTILINE has no effect.
1354 nigel 63
1355 nigel 91 PCRE_NEWLINE_CR
1356     PCRE_NEWLINE_LF
1357     PCRE_NEWLINE_CRLF
1358 ph10 150 PCRE_NEWLINE_ANYCRLF
1359 nigel 93 PCRE_NEWLINE_ANY
1360 nigel 91
1361 ph10 572 These options override the default newline definition that was chosen
1362     when PCRE was built. Setting the first or the second specifies that a
1363     newline is indicated by a single character (CR or LF, respectively).
1364     Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the
1365     two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies
1366 ph10 150 that any of the three preceding sequences should be recognized. Setting
1367 ph10 572 PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be
1368 ph10 150 recognized. The Unicode newline sequences are the three just mentioned,
1369 ph10 572 plus the single characters VT (vertical tab, U+000B), FF (formfeed,
1370     U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS
1371     (paragraph separator, U+2029). The last two are recognized only in
1372 ph10 150 UTF-8 mode.
1373 nigel 91
1374 ph10 572 The newline setting in the options word uses three bits that are
1375 ph10 150 treated as a number, giving eight possibilities. Currently only six are
1376 ph10 572 used (default plus the five values above). This means that if you set
1377     more than one newline option, the combination may or may not be sensi-
1378 ph10 150 ble. For example, PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to
1379 ph10 572 PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and
1380 ph10 150 cause an error.
1381 nigel 91
1382 ph10 572 The only time that a line break in a pattern is specially recognized
1383     when compiling is when PCRE_EXTENDED is set. CR and LF are whitespace
1384     characters, and so are ignored in this mode. Also, an unescaped # out-
1385     side a character class indicates a comment that lasts until after the
1386     next line break sequence. In other circumstances, line break sequences
1387     in patterns are treated as literal data.
1388 nigel 93
1389     The newline option that is set at compile time becomes the default that
1390 ph10 392 is used for pcre_exec() and pcre_dfa_exec(), but it can be overridden.
1391 nigel 93
1392 nigel 73 PCRE_NO_AUTO_CAPTURE
1393 nigel 41
1394 nigel 73 If this option is set, it disables the use of numbered capturing paren-
1395 ph10 518 theses in the pattern. Any opening parenthesis that is not followed by
1396     ? behaves as if it were followed by ?: but named parentheses can still
1397     be used for capturing (and they acquire numbers in the usual way).
1398 nigel 73 There is no equivalent of this option in Perl.
1399 nigel 41
1400 ph10 535 PCRE_UCP
1401    
1402 ph10 572 This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W,
1403     \w, and some of the POSIX character classes. By default, only ASCII
1404     characters are recognized, but if PCRE_UCP is set, Unicode properties
1405     are used instead to classify characters. More details are given in the
1406     section on generic character types in the pcrepattern page. If you set
1407     PCRE_UCP, matching one of the items it affects takes much longer. The
1408     option is available only if PCRE has been compiled with Unicode prop-
1409     erty support.
1410 ph10 535
1411 nigel 73 PCRE_UNGREEDY
1412 nigel 41
1413 ph10 572 This option inverts the "greediness" of the quantifiers so that they
1414     are not greedy by default, but become greedy if followed by "?". It is
1415     not compatible with Perl. It can also be set by a (?U) option setting
1416 nigel 73 within the pattern.
1417 nigel 41
1418 nigel 73 PCRE_UTF8
1419 nigel 49
1420 ph10 572 This option causes PCRE to regard both the pattern and the subject as
1421     strings of UTF-8 characters instead of single-byte character strings.
1422     However, it is available only when PCRE is built to include UTF-8 sup-
1423     port. If not, the use of this option provokes an error. Details of how
1424     this option changes the behaviour of PCRE are given in the section on
1425 nigel 75 UTF-8 support in the main pcre page.
1426 nigel 71
1427 nigel 73 PCRE_NO_UTF8_CHECK
1428 nigel 71
1429 nigel 73 When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is
1430 ph10 572 automatically checked. There is a discussion about the validity of
1431     UTF-8 strings in the main pcre page. If an invalid UTF-8 sequence of
1432     bytes is found, pcre_compile() returns an error. If you already know
1433 ph10 211 that your pattern is valid, and you want to skip this check for perfor-
1434 ph10 572 mance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is
1435     set, the effect of passing an invalid UTF-8 string as a pattern is
1436     undefined. It may cause your program to crash. Note that this option
1437     can also be passed to pcre_exec() and pcre_dfa_exec(), to suppress the
1438 ph10 211 UTF-8 validity checking of subject strings.
1439 nigel 71
1440 nigel 73
1441 nigel 77 COMPILATION ERROR CODES
1442    
1443 ph10 572 The following table lists the error codes than may be returned by
1444     pcre_compile2(), along with the error messages that may be returned by
1445     both compiling functions. As PCRE has developed, some error codes have
1446 nigel 93 fallen out of use. To avoid confusion, they have not been re-used.
1447 nigel 77
1448     0 no error
1449     1 \ at end of pattern
1450     2 \c at end of pattern
1451     3 unrecognized character follows \
1452     4 numbers out of order in {} quantifier
1453     5 number too big in {} quantifier
1454     6 missing terminating ] for character class
1455     7 invalid escape sequence in character class
1456     8 range out of order in character class
1457     9 nothing to repeat
1458 nigel 93 10 [this code is not in use]
1459 nigel 77 11 internal error: unexpected repeat
1460 ph10 292 12 unrecognized character after (? or (?-
1461 nigel 77 13 POSIX named classes are supported only within a class
1462     14 missing )
1463     15 reference to non-existent subpattern
1464     16 erroffset passed as NULL
1465     17 unknown option bit(s) set
1466     18 missing ) after comment
1467 nigel 93 19 [this code is not in use]
1468 ph10 292 20 regular expression is too large
1469 nigel 77 21 failed to get memory
1470     22 unmatched parentheses
1471     23 internal error: code overflow
1472     24 unrecognized character after (?<
1473     25 lookbehind assertion is not fixed length
1474 nigel 91 26 malformed number or name after (?(
1475 nigel 77 27 conditional group contains more than two branches
1476     28 assertion expected after (?(
1477 ph10 182 29 (?R or (?[+-]digits must be followed by )
1478 nigel 77 30 unknown POSIX class name
1479     31 POSIX collating elements are not supported
1480     32 this version of PCRE is not compiled with PCRE_UTF8 support
1481 nigel 93 33 [this code is not in use]
1482 nigel 77 34 character value in \x{...} sequence is too large
1483     35 invalid condition (?(0)
1484     36 \C not allowed in lookbehind assertion
1485     37 PCRE does not support \L, \l, \N, \U, or \u
1486     38 number after (?C is > 255
1487     39 closing ) for (?C expected
1488     40 recursive call could loop indefinitely
1489     41 unrecognized character after (?P
1490 nigel 93 42 syntax error in subpattern name (missing terminator)
1491 nigel 91 43 two named subpatterns have the same name
1492 nigel 77 44 invalid UTF-8 string
1493     45 support for \P, \p, and \X has not been compiled
1494     46 malformed \P or \p sequence
1495     47 unknown property name after \P or \p
1496 nigel 91 48 subpattern name is too long (maximum 32 characters)
1497 ph10 292 49 too many named subpatterns (maximum 10000)
1498 ph10 202 50 [this code is not in use]
1499 nigel 91 51 octal value is greater than \377 (not in UTF-8 mode)
1500 nigel 93 52 internal error: overran compiling workspace
1501 ph10 548 53 internal error: previously-checked referenced subpattern
1502     not found
1503 nigel 93 54 DEFINE group contains more than one branch
1504     55 repeating a DEFINE group is not allowed
1505 ph10 231 56 inconsistent NEWLINE options
1506 ph10 345 57 \g is not followed by a braced, angle-bracketed, or quoted
1507     name/number or by a plain number
1508     58 a numbered reference must not be zero
1509 ph10 512 59 an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT)
1510 ph10 292 60 (*VERB) not recognized
1511     61 number is too big
1512     62 subpattern name expected
1513     63 digit expected after (?+
1514 ph10 345 64 ] is an invalid data character in JavaScript compatibility mode
1515 ph10 548 65 different names for subpatterns of the same number are
1516     not allowed
1517 ph10 512 66 (*MARK) must have an argument
1518 ph10 535 67 this version of PCRE is not compiled with PCRE_UCP support
1519 nigel 77
1520 ph10 572 The numbers 32 and 10000 in errors 48 and 49 are defaults; different
1521 ph10 292 values may be used if the limits were changed when PCRE was built.
1522 nigel 77
1523 ph10 292
1524 nigel 63 STUDYING A PATTERN
1525 nigel 49
1526 nigel 77 pcre_extra *pcre_study(const pcre *code, int options
1527 nigel 73 const char **errptr);
1528 nigel 63
1529 ph10 572 If a compiled pattern is going to be used several times, it is worth
1530 nigel 75 spending more time analyzing it in order to speed up the time taken for
1531 ph10 572 matching. The function pcre_study() takes a pointer to a compiled pat-
1532 nigel 75 tern as its first argument. If studying the pattern produces additional
1533 ph10 572 information that will help speed up matching, pcre_study() returns a
1534     pointer to a pcre_extra block, in which the study_data field points to
1535 nigel 75 the results of the study.
1536 nigel 41
1537 nigel 75 The returned value from pcre_study() can be passed directly to
1538 ph10 572 pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con-
1539     tains other fields that can be set by the caller before the block is
1540 ph10 461 passed; these are described below in the section on matching a pattern.
1541 nigel 63
1542 ph10 572 If studying the pattern does not produce any useful information,
1543 nigel 75 pcre_study() returns NULL. In that circumstance, if the calling program
1544 ph10 572 wants to pass any of the other fields to pcre_exec() or
1545 ph10 461 pcre_dfa_exec(), it must set up its own pcre_extra block.
1546 nigel 41
1547 ph10 572 The second argument of pcre_study() contains option bits. At present,
1548 nigel 75 no options are defined, and this argument should always be zero.
1549    
1550 ph10 572 The third argument for pcre_study() is a pointer for an error message.
1551     If studying succeeds (even if no data is returned), the variable it
1552     points to is set to NULL. Otherwise it is set to point to a textual
1553 nigel 87 error message. This is a static string that is part of the library. You
1554 ph10 572 must not try to free it. You should test the error pointer for NULL
1555 nigel 87 after calling pcre_study(), to be sure that it has run successfully.
1556 nigel 41
1557 nigel 73 This is a typical call to pcre_study():
1558 nigel 53
1559 nigel 73 pcre_extra *pe;
1560     pe = pcre_study(
1561     re, /* result of pcre_compile() */
1562     0, /* no options exist */
1563     &error); /* set to NULL or points to a message */
1564 nigel 53
1565 ph10 461 Studying a pattern does two things: first, a lower bound for the length
1566     of subject string that is needed to match the pattern is computed. This
1567     does not mean that there are any strings of that length that match, but
1568 ph10 572 it does guarantee that no shorter strings match. The value is used by
1569     pcre_exec() and pcre_dfa_exec() to avoid wasting time by trying to
1570     match strings that are shorter than the lower bound. You can find out
1571 ph10 461 the value in a calling program via the pcre_fullinfo() function.
1572 nigel 41
1573 ph10 461 Studying a pattern is also useful for non-anchored patterns that do not
1574 ph10 572 have a single fixed starting character. A bitmap of possible starting
1575     bytes is created. This speeds up finding a position in the subject at
1576 ph10 461 which to start matching.
1577 nigel 41
1578 ph10 572 The two optimizations just described can be disabled by setting the
1579     PCRE_NO_START_OPTIMIZE option when calling pcre_exec() or
1580     pcre_dfa_exec(). You might want to do this if your pattern contains
1581     callouts or (*MARK), and you want to make use of these facilities in
1582     cases where matching fails. See the discussion of PCRE_NO_START_OPTI-
1583     MIZE below.
1584 ph10 461
1585 ph10 548
1586 nigel 63 LOCALE SUPPORT
1587 nigel 41
1588 ph10 572 PCRE handles caseless matching, and determines whether characters are
1589     letters, digits, or whatever, by reference to a set of tables, indexed
1590     by character value. When running in UTF-8 mode, this applies only to
1591     characters with codes less than 128. By default, higher-valued codes
1592 ph10 535 never match escapes such as \w or \d, but they can be tested with \p if
1593 ph10 572 PCRE is built with Unicode character property support. Alternatively,
1594     the PCRE_UCP option can be set at compile time; this causes \w and
1595 ph10 535 friends to use Unicode property support instead of built-in tables. The
1596     use of locales with Unicode is discouraged. If you are handling charac-
1597 ph10 572 ters with codes greater than 128, you should either use UTF-8 and Uni-
1598 ph10 535 code, or use locales, but not try to mix the two.
1599 nigel 41
1600 ph10 572 PCRE contains an internal set of tables that are used when the final
1601     argument of pcre_compile() is NULL. These are sufficient for many
1602 ph10 142 applications. Normally, the internal tables recognize only ASCII char-
1603     acters. However, when PCRE is built, it is possible to cause the inter-
1604     nal tables to be rebuilt in the default "C" locale of the local system,
1605     which may cause them to be different.
1606 nigel 41
1607 ph10 572 The internal tables can always be overridden by tables supplied by the
1608 ph10 142 application that calls PCRE. These may be created in a different locale
1609 ph10 572 from the default. As more and more applications change to using Uni-
1610 ph10 142 code, the need for this locale support is expected to die away.
1611    
1612 ph10 572 External tables are built by calling the pcre_maketables() function,
1613     which has no arguments, in the relevant locale. The result can then be
1614     passed to pcre_compile() or pcre_exec() as often as necessary. For
1615     example, to build and use tables that are appropriate for the French
1616     locale (where accented characters with values greater than 128 are
1617 nigel 75 treated as letters), the following code could be used:
1618    
1619     setlocale(LC_CTYPE, "fr_FR");
1620 nigel 73 tables = pcre_maketables();
1621     re = pcre_compile(..., tables);
1622 nigel 41
1623 ph10 572 The locale name "fr_FR" is used on Linux and other Unix-like systems;
1624 ph10 142 if you are using Windows, the name for the French locale is "french".
1625    
1626 ph10 572 When pcre_maketables() runs, the tables are built in memory that is
1627     obtained via pcre_malloc. It is the caller's responsibility to ensure
1628     that the memory containing the tables remains available for as long as
1629 nigel 75 it is needed.
1630 nigel 41
1631 nigel 75 The pointer that is passed to pcre_compile() is saved with the compiled
1632 ph10 572 pattern, and the same tables are used via this pointer by pcre_study()
1633 nigel 75 and normally also by pcre_exec(). Thus, by default, for any single pat-
1634     tern, compilation, studying and matching all happen in the same locale,
1635     but different patterns can be compiled in different locales.
1636 nigel 41
1637 ph10 572 It is possible to pass a table pointer or NULL (indicating the use of
1638     the internal tables) to pcre_exec(). Although not intended for this
1639     purpose, this facility could be used to match a pattern in a different
1640 nigel 75 locale from the one in which it was compiled. Passing table pointers at
1641     run time is discussed below in the section on matching a pattern.
1642    
1643    
1644 nigel 63 INFORMATION ABOUT A PATTERN
1645 nigel 41
1646 nigel 73 int pcre_fullinfo(const pcre *code, const pcre_extra *extra,
1647     int what, void *where);
1648 nigel 63
1649 ph10 572 The pcre_fullinfo() function returns information about a compiled pat-
1650 nigel 73 tern. It replaces the obsolete pcre_info() function, which is neverthe-
1651     less retained for backwards compability (and is documented below).
1652 nigel 43
1653 ph10 572 The first argument for pcre_fullinfo() is a pointer to the compiled
1654     pattern. The second argument is the result of pcre_study(), or NULL if
1655     the pattern was not studied. The third argument specifies which piece
1656     of information is required, and the fourth argument is a pointer to a
1657     variable to receive the data. The yield of the function is zero for
1658 nigel 73 success, or one of the following negative numbers:
1659 nigel 41
1660 nigel 73 PCRE_ERROR_NULL the argument code was NULL
1661     the argument where was NULL
1662     PCRE_ERROR_BADMAGIC the "magic number" was not found
1663     PCRE_ERROR_BADOPTION the value of what was invalid
1664 nigel 53
1665 ph10 572 The "magic number" is placed at the start of each compiled pattern as
1666     an simple check against passing an arbitrary memory pointer. Here is a
1667     typical call of pcre_fullinfo(), to obtain the length of the compiled
1668 nigel 75 pattern:
1669 nigel 53
1670 nigel 73 int rc;
1671 nigel 91 size_t length;
1672 nigel 73 rc = pcre_fullinfo(
1673     re, /* result of pcre_compile() */
1674     pe, /* result of pcre_study(), or NULL */
1675     PCRE_INFO_SIZE, /* what is required */
1676     &length); /* where to put the data */
1677 nigel 43
1678 ph10 572 The possible values for the third argument are defined in pcre.h, and
1679 nigel 73 are as follows:
1680 nigel 43
1681 nigel 73 PCRE_INFO_BACKREFMAX
1682 nigel 41
1683 ph10 572 Return the number of the highest back reference in the pattern. The
1684     fourth argument should point to an int variable. Zero is returned if
1685 nigel 73 there are no back references.
1686 nigel 43
1687 nigel 73 PCRE_INFO_CAPTURECOUNT
1688 nigel 43
1689 ph10 572 Return the number of capturing subpatterns in the pattern. The fourth
1690 nigel 73 argument should point to an int variable.
1691 nigel 43
1692 nigel 77 PCRE_INFO_DEFAULT_TABLES
1693 nigel 75
1694 ph10 572 Return a pointer to the internal default character tables within PCRE.
1695     The fourth argument should point to an unsigned char * variable. This
1696 nigel 75 information call is provided for internal use by the pcre_study() func-
1697 ph10 572 tion. External callers can cause PCRE to use its internal tables by
1698 nigel 75 passing a NULL table pointer.
1699    
1700 nigel 73 PCRE_INFO_FIRSTBYTE
1701 nigel 43
1702 ph10 572 Return information about the first byte of any matched string, for a
1703     non-anchored pattern. The fourth argument should point to an int vari-
1704     able. (This option used to be called PCRE_INFO_FIRSTCHAR; the old name
1705 nigel 91 is still recognized for backwards compatibility.)
1706 nigel 41
1707 ph10 572 If there is a fixed first byte, for example, from a pattern such as
1708 nigel 93 (cat|cow|coyote), its value is returned. Otherwise, if either
1709 nigel 41
1710 ph10 572 (a) the pattern was compiled with the PCRE_MULTILINE option, and every
1711 nigel 73 branch starts with "^", or
1712 nigel 43
1713 nigel 73 (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not
1714     set (if it were set, the pattern would be anchored),
1715 nigel 41
1716 ph10 572 -1 is returned, indicating that the pattern matches only at the start
1717     of a subject string or after any newline within the string. Otherwise
1718 nigel 73 -2 is returned. For anchored patterns, -2 is returned.
1719 nigel 41
1720 nigel 73 PCRE_INFO_FIRSTTABLE
1721 nigel 41
1722 ph10 572 If the pattern was studied, and this resulted in the construction of a
1723 nigel 73 256-bit table indicating a fixed set of bytes for the first byte in any
1724 ph10 572 matching string, a pointer to the table is returned. Otherwise NULL is
1725     returned. The fourth argument should point to an unsigned char * vari-
1726 nigel 73 able.
1727 nigel 43
1728 ph10 227 PCRE_INFO_HASCRORLF
1729    
1730 ph10 572 Return 1 if the pattern contains any explicit matches for CR or LF
1731     characters, otherwise 0. The fourth argument should point to an int
1732     variable. An explicit match is either a literal CR or LF character, or
1733 ph10 243 \r or \n.
1734 ph10 227
1735 ph10 172 PCRE_INFO_JCHANGED
1736    
1737 ph10 572 Return 1 if the (?J) or (?-J) option setting is used in the pattern,
1738     otherwise 0. The fourth argument should point to an int variable. (?J)
1739 ph10 286 and (?-J) set and unset the local PCRE_DUPNAMES option, respectively.
1740 ph10 172
1741 nigel 73 PCRE_INFO_LASTLITERAL
1742 nigel 43
1743 ph10 572 Return the value of the rightmost literal byte that must exist in any
1744     matched string, other than at its start, if such a byte has been
1745 nigel 73 recorded. The fourth argument should point to an int variable. If there
1746 ph10 572 is no such byte, -1 is returned. For anchored patterns, a last literal
1747     byte is recorded only if it follows something of variable length. For
1748 nigel 73 example, for the pattern /^a\d+z\d+/ the returned value is "z", but for
1749     /^a\dz\d/ the returned value is -1.
1750 nigel 63
1751 ph10 461 PCRE_INFO_MINLENGTH
1752    
1753 ph10 572 If the pattern was studied and a minimum length for matching subject
1754     strings was computed, its value is returned. Otherwise the returned
1755     value is -1. The value is a number of characters, not bytes (this may
1756     be relevant in UTF-8 mode). The fourth argument should point to an int
1757     variable. A non-negative value is a lower bound to the length of any
1758     matching string. There may not be any strings of that length that do
1759 ph10 461 actually match, but every string that does match is at least that long.
1760    
1761 nigel 73 PCRE_INFO_NAMECOUNT
1762     PCRE_INFO_NAMEENTRYSIZE
1763     PCRE_INFO_NAMETABLE
1764 nigel 63
1765 ph10 572 PCRE supports the use of named as well as numbered capturing parenthe-
1766     ses. The names are just an additional way of identifying the parenthe-
1767 nigel 91 ses, which still acquire numbers. Several convenience functions such as
1768 ph10 572 pcre_get_named_substring() are provided for extracting captured sub-
1769     strings by name. It is also possible to extract the data directly, by
1770     first converting the name to a number in order to access the correct
1771 nigel 91 pointers in the output vector (described with pcre_exec() below). To do
1772 ph10 572 the conversion, you need to use the name-to-number map, which is
1773 nigel 91 described by these three values.
1774 nigel 63
1775 nigel 73 The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT
1776     gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size
1777 ph10 572 of each entry; both of these return an int value. The entry size
1778     depends on the length of the longest name. PCRE_INFO_NAMETABLE returns
1779     a pointer to the first entry of the table (a pointer to char). The
1780 nigel 73 first two bytes of each entry are the number of the capturing parenthe-
1781 ph10 572 sis, most significant byte first. The rest of the entry is the corre-
1782 ph10 461 sponding name, zero terminated.
1783 nigel 63
1784 ph10 572 The names are in alphabetical order. Duplicate names may appear if (?|
1785 ph10 461 is used to create multiple groups with the same number, as described in
1786 ph10 572 the section on duplicate subpattern numbers in the pcrepattern page.
1787     Duplicate names for subpatterns with different numbers are permitted
1788     only if PCRE_DUPNAMES is set. In all cases of duplicate names, they
1789     appear in the table in the order in which they were found in the pat-
1790     tern. In the absence of (?| this is the order of increasing number;
1791 ph10 461 when (?| is used this is not necessarily the case because later subpat-
1792     terns may have lower numbers.
1793    
1794 ph10 572 As a simple example of the name/number table, consider the following
1795     pattern (assume PCRE_EXTENDED is set, so white space - including new-
1796 ph10 461 lines - is ignored):
1797    
1798 nigel 93 (?<date> (?<year>(\d\d)?\d\d) -
1799     (?<month>\d\d) - (?<day>\d\d) )
1800 nigel 63
1801 ph10 572 There are four named subpatterns, so the table has four entries, and
1802     each entry in the table is eight bytes long. The table is as follows,
1803 nigel 75 with non-printing bytes shows in hexadecimal, and undefined bytes shown
1804     as ??:
1805 nigel 63
1806 nigel 73 00 01 d a t e 00 ??
1807     00 05 d a y 00 ?? ??
1808     00 04 m o n t h 00
1809     00 02 y e a r 00 ??
1810 nigel 63
1811 ph10 572 When writing code to extract data from named subpatterns using the
1812     name-to-number map, remember that the length of the entries is likely
1813 nigel 91 to be different for each compiled pattern.
1814 nigel 63
1815 ph10 172 PCRE_INFO_OKPARTIAL
1816    
1817 ph10 572 Return 1 if the pattern can be used for partial matching with
1818     pcre_exec(), otherwise 0. The fourth argument should point to an int
1819     variable. From release 8.00, this always returns 1, because the
1820     restrictions that previously applied to partial matching have been
1821     lifted. The pcrepartial documentation gives details of partial match-
1822 ph10 453 ing.
1823 ph10 172
1824 nigel 73 PCRE_INFO_OPTIONS
1825 nigel 63
1826 ph10 572 Return a copy of the options with which the pattern was compiled. The
1827     fourth argument should point to an unsigned long int variable. These
1828 nigel 73 option bits are those specified in the call to pcre_compile(), modified
1829 ph10 197 by any top-level option settings at the start of the pattern itself. In
1830 ph10 572 other words, they are the options that will be in force when matching
1831     starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with
1832     the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE,
1833 ph10 197 and PCRE_EXTENDED.
1834 nigel 63
1835 ph10 572 A pattern is automatically anchored by PCRE if all of its top-level
1836 nigel 73 alternatives begin with one of the following:
1837 nigel 63
1838 nigel 73 ^ unless PCRE_MULTILINE is set
1839     \A always
1840     \G always
1841     .* if PCRE_DOTALL is set and there are no back
1842     references to the subpattern in which .* appears
1843 nigel 63
1844 nigel 73 For such patterns, the PCRE_ANCHORED bit is set in the options returned
1845     by pcre_fullinfo().
1846 nigel 63
1847 nigel 73 PCRE_INFO_SIZE
1848 nigel 63
1849 ph10 572 Return the size of the compiled pattern, that is, the value that was
1850 nigel 73 passed as the argument to pcre_malloc() when PCRE was getting memory in
1851     which to place the compiled data. The fourth argument should point to a
1852     size_t variable.
1853 nigel 63
1854 nigel 73 PCRE_INFO_STUDYSIZE
1855 nigel 63
1856 nigel 75 Return the size of the data block pointed to by the study_data field in
1857 ph10 572 a pcre_extra block. That is, it is the value that was passed to
1858 nigel 73 pcre_malloc() when PCRE was getting memory into which to place the data
1859 ph10 572 created by pcre_study(). If pcre_extra is NULL, or there is no study
1860     data, zero is returned. The fourth argument should point to a size_t
1861 nigel 73 variable.
1862 nigel 63
1863 nigel 73
1864 nigel 63 OBSOLETE INFO FUNCTION
1865    
1866 nigel 73 int pcre_info(const pcre *code, int *optptr, int *firstcharptr);
1867 nigel 63
1868 ph10 572 The pcre_info() function is now obsolete because its interface is too
1869     restrictive to return all the available data about a compiled pattern.
1870     New programs should use pcre_fullinfo() instead. The yield of
1871     pcre_info() is the number of capturing subpatterns, or one of the fol-
1872 nigel 73 lowing negative numbers:
1873 nigel 43
1874 nigel 73 PCRE_ERROR_NULL the argument code was NULL
1875     PCRE_ERROR_BADMAGIC the "magic number" was not found
1876 nigel 43
1877 ph10 572 If the optptr argument is not NULL, a copy of the options with which
1878     the pattern was compiled is placed in the integer it points to (see
1879 nigel 73 PCRE_INFO_OPTIONS above).
1880 nigel 43
1881 ph10 572 If the pattern is not anchored and the firstcharptr argument is not
1882     NULL, it is used to pass back information about the first character of
1883 nigel 73 any matched string (see PCRE_INFO_FIRSTBYTE above).
1884 nigel 43
1885    
1886 nigel 77 REFERENCE COUNTS
1887 nigel 53
1888 nigel 77 int pcre_refcount(pcre *code, int adjust);
1889    
1890 ph10 572 The pcre_refcount() function is used to maintain a reference count in
1891 nigel 77 the data block that contains a compiled pattern. It is provided for the
1892 ph10 572 benefit of applications that operate in an object-oriented manner,
1893 nigel 77 where different parts of the application may be using the same compiled
1894     pattern, but you want to free the block when they are all done.
1895    
1896     When a pattern is compiled, the reference count field is initialized to
1897 ph10 572 zero. It is changed only by calling this function, whose action is to
1898     add the adjust value (which may be positive or negative) to it. The
1899 nigel 77 yield of the function is the new value. However, the value of the count
1900 ph10 572 is constrained to lie between 0 and 65535, inclusive. If the new value
1901 nigel 77 is outside these limits, it is forced to the appropriate limit value.
1902    
1903 ph10 572 Except when it is zero, the reference count is not correctly preserved
1904     if a pattern is compiled on one host and then transferred to a host
1905 nigel 77 whose byte-order is different. (This seems a highly unlikely scenario.)
1906    
1907    
1908     MATCHING A PATTERN: THE TRADITIONAL FUNCTION
1909    
1910 nigel 73 int pcre_exec(const pcre *code, const pcre_extra *extra,
1911     const char *subject, int length, int startoffset,
1912     int options, int *ovector, int ovecsize);
1913 nigel 53
1914 ph10 572 The function pcre_exec() is called to match a subject string against a
1915     compiled pattern, which is passed in the code argument. If the pattern
1916     was studied, the result of the study should be passed in the extra
1917     argument. This function is the main matching facility of the library,
1918 nigel 77 and it operates in a Perl-like manner. For specialist use there is also
1919 ph10 572 an alternative matching function, which is described below in the sec-
1920 nigel 77 tion about the pcre_dfa_exec() function.
1921 nigel 41
1922 ph10 572 In most applications, the pattern will have been compiled (and option-
1923     ally studied) in the same process that calls pcre_exec(). However, it
1924 nigel 75 is possible to save compiled patterns and study data, and then use them
1925 ph10 572 later in different processes, possibly even on different hosts. For a
1926 nigel 75 discussion about this, see the pcreprecompile documentation.
1927    
1928 nigel 73 Here is an example of a simple call to pcre_exec():
1929 nigel 53
1930 nigel 73 int rc;
1931     int ovector[30];
1932     rc = pcre_exec(
1933     re, /* result of pcre_compile() */
1934     NULL, /* we didn't study the pattern */
1935     "some string", /* the subject string */
1936     11, /* the length of the subject string */
1937     0, /* start at offset 0 in the subject */
1938     0, /* default options */
1939 nigel 75 ovector, /* vector of integers for substring information */
1940 nigel 77 30); /* number of elements (NOT size in bytes) */
1941 nigel 53
1942 nigel 75 Extra data for pcre_exec()
1943 nigel 63
1944 ph10 572 If the extra argument is not NULL, it must point to a pcre_extra data
1945     block. The pcre_study() function returns such a block (when it doesn't
1946     return NULL), but you can also create one for yourself, and pass addi-
1947     tional information in it. The pcre_extra block contains the following
1948 nigel 87 fields (not necessarily in this order):
1949 nigel 75
1950 nigel 73 unsigned long int flags;
1951     void *study_data;
1952     unsigned long int match_limit;
1953 nigel 87 unsigned long int match_limit_recursion;
1954 nigel 73 void *callout_data;
1955 nigel 75 const unsigned char *tables;
1956 ph10 512 unsigned char **mark;
1957 nigel 63
1958 ph10 572 The flags field is a bitmap that specifies which of the other fields
1959 nigel 73 are set. The flag bits are:
1960 nigel 63
1961 nigel 73 PCRE_EXTRA_STUDY_DATA
1962     PCRE_EXTRA_MATCH_LIMIT
1963 nigel 87 PCRE_EXTRA_MATCH_LIMIT_RECURSION
1964 nigel 73 PCRE_EXTRA_CALLOUT_DATA
1965 nigel 75 PCRE_EXTRA_TABLES
1966 ph10 512 PCRE_EXTRA_MARK
1967 nigel 63
1968 ph10 572 Other flag bits should be set to zero. The study_data field is set in
1969     the pcre_extra block that is returned by pcre_study(), together with
1970 nigel 75 the appropriate flag bit. You should not set this yourself, but you may
1971 ph10 572 add to the block by setting the other fields and their corresponding
1972 nigel 75 flag bits.
1973 nigel 63
1974 nigel 73 The match_limit field provides a means of preventing PCRE from using up
1975 ph10 572 a vast amount of resources when running patterns that are not going to
1976     match, but which have a very large number of possibilities in their
1977     search trees. The classic example is a pattern that uses nested unlim-
1978 ph10 461 ited repeats.
1979 nigel 63
1980 ph10 572 Internally, PCRE uses a function called match() which it calls repeat-
1981     edly (sometimes recursively). The limit set by match_limit is imposed
1982     on the number of times this function is called during a match, which
1983     has the effect of limiting the amount of backtracking that can take
1984 nigel 87 place. For patterns that are not anchored, the count restarts from zero
1985     for each position in the subject string.
1986 nigel 75
1987 ph10 572 The default value for the limit can be set when PCRE is built; the
1988     default default is 10 million, which handles all but the most extreme
1989     cases. You can override the default by suppling pcre_exec() with a
1990     pcre_extra block in which match_limit is set, and
1991     PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is
1992 nigel 73 exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT.
1993 nigel 63
1994 ph10 572 The match_limit_recursion field is similar to match_limit, but instead
1995 nigel 87 of limiting the total number of times that match() is called, it limits
1996 ph10 572 the depth of recursion. The recursion depth is a smaller number than
1997     the total number of calls, because not all calls to match() are recur-
1998 nigel 87 sive. This limit is of use only if it is set smaller than match_limit.
1999    
2000 ph10 572 Limiting the recursion depth limits the amount of stack that can be
2001 nigel 87 used, or, when PCRE has been compiled to use memory on the heap instead
2002     of the stack, the amount of heap memory that can be used.
2003    
2004 ph10 572 The default value for match_limit_recursion can be set when PCRE is
2005     built; the default default is the same value as the default for
2006     match_limit. You can override the default by suppling pcre_exec() with
2007     a pcre_extra block in which match_limit_recursion is set, and
2008     PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the
2009 nigel 87 limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT.
2010    
2011 ph10 572 The callout_data field is used in conjunction with the "callout" fea-
2012 ph10 453 ture, and is described in the pcrecallout documentation.
2013 nigel 63
2014 ph10 572 The tables field is used to pass a character tables pointer to
2015     pcre_exec(); this overrides the value that is stored with the compiled
2016     pattern. A non-NULL value is stored with the compiled pattern only if
2017     custom tables were supplied to pcre_compile() via its tableptr argu-
2018 nigel 75 ment. If NULL is passed to pcre_exec() using this mechanism, it forces
2019 ph10 572 PCRE's internal tables to be used. This facility is helpful when re-
2020     using patterns that have been saved after compiling with an external
2021     set of tables, because the external tables might be at a different
2022     address when pcre_exec() is called. See the pcreprecompile documenta-
2023 nigel 75 tion for a discussion of saving compiled patterns for later use.
2024 nigel 41
2025 ph10 572 If PCRE_EXTRA_MARK is set in the flags field, the mark field must be
2026     set to point to a char * variable. If the pattern contains any back-
2027     tracking control verbs such as (*MARK:NAME), and the execution ends up
2028     with a name to pass back, a pointer to the name string (zero termi-
2029     nated) is placed in the variable pointed to by the mark field. The
2030     names are within the compiled pattern; if you wish to retain such a
2031     name you must copy it before freeing the memory of a compiled pattern.
2032     If there is no name to pass back, the variable pointed to by the mark
2033     field set to NULL. For details of the backtracking control verbs, see
2034 ph10 512 the section entitled "Backtracking control" in the pcrepattern documen-
2035     tation.
2036    
2037 nigel 75 Option bits for pcre_exec()
2038 nigel 71
2039 ph10 572 The unused bits of the options argument for pcre_exec() must be zero.
2040     The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx,
2041     PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
2042     PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and
2043 ph10 453 PCRE_PARTIAL_HARD.
2044 nigel 41
2045 nigel 75 PCRE_ANCHORED
2046 nigel 41
2047 ph10 572 The PCRE_ANCHORED option limits pcre_exec() to matching at the first
2048     matching position. If a pattern was compiled with PCRE_ANCHORED, or
2049     turned out to be anchored by virtue of its contents, it cannot be made
2050 nigel 75 unachored at matching time.
2051    
2052 ph10 231 PCRE_BSR_ANYCRLF
2053     PCRE_BSR_UNICODE
2054    
2055     These options (which are mutually exclusive) control what the \R escape
2056 ph10 572 sequence matches. The choice is either to match only CR, LF, or CRLF,
2057     or to match any Unicode newline sequence. These options override the
2058 ph10 231 choice that was made or defaulted when the pattern was compiled.
2059    
2060 nigel 91 PCRE_NEWLINE_CR
2061     PCRE_NEWLINE_LF
2062     PCRE_NEWLINE_CRLF
2063 ph10 150 PCRE_NEWLINE_ANYCRLF
2064 nigel 93 PCRE_NEWLINE_ANY
2065 nigel 91
2066 ph10 572 These options override the newline definition that was chosen or
2067     defaulted when the pattern was compiled. For details, see the descrip-
2068     tion of pcre_compile() above. During matching, the newline choice
2069     affects the behaviour of the dot, circumflex, and dollar metacharac-
2070     ters. It may also alter the way the match position is advanced after a
2071 ph10 227 match failure for an unanchored pattern.
2072 nigel 91
2073 ph10 572 When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is
2074     set, and a match attempt for an unanchored pattern fails when the cur-
2075     rent position is at a CRLF sequence, and the pattern contains no
2076     explicit matches for CR or LF characters, the match position is
2077 ph10 227 advanced by two characters instead of one, in other words, to after the
2078     CRLF.
2079    
2080     The above rule is a compromise that makes the most common cases work as
2081 ph10 572 expected. For example, if the pattern is .+A (and the PCRE_DOTALL
2082 ph10 227 option is not set), it does not match the string "\r\nA" because, after
2083 ph10 572 failing at the start, it skips both the CR and the LF before retrying.
2084     However, the pattern [\r\n]A does match that string, because it con-
2085 ph10 227 tains an explicit CR or LF reference, and so advances only by one char-
2086 ph10 231 acter after the first failure.
2087 ph10 227
2088 ph10 231 An explicit match for CR of LF is either a literal appearance of one of
2089 ph10 572 those characters, or one of the \r or \n escape sequences. Implicit
2090     matches such as [^X] do not count, nor does \s (which includes CR and
2091 ph10 231 LF in the characters that it matches).
2092    
2093 ph10 572 Notwithstanding the above, anomalous effects may still occur when CRLF
2094 ph10 227 is a valid newline sequence and explicit \r or \n escapes appear in the
2095     pattern.
2096    
2097 nigel 73 PCRE_NOTBOL
2098 nigel 41
2099 nigel 75 This option specifies that first character of the subject string is not
2100 ph10 572 the beginning of a line, so the circumflex metacharacter should not
2101     match before it. Setting this without PCRE_MULTILINE (at compile time)
2102     causes circumflex never to match. This option affects only the behav-
2103 nigel 77 iour of the circumflex metacharacter. It does not affect \A.
2104 nigel 41
2105 nigel 73 PCRE_NOTEOL
2106 nigel 41
2107 nigel 75 This option specifies that the end of the subject string is not the end
2108 ph10 572 of a line, so the dollar metacharacter should not match it nor (except
2109     in multiline mode) a newline immediately before it. Setting this with-
2110 nigel 75 out PCRE_MULTILINE (at compile time) causes dollar never to match. This
2111 ph10 572 option affects only the behaviour of the dollar metacharacter. It does
2112 nigel 75 not affect \Z or \z.
2113 nigel 41
2114 nigel 73 PCRE_NOTEMPTY
2115 nigel 41
2116 nigel 73 An empty string is not considered to be a valid match if this option is
2117 ph10 572 set. If there are alternatives in the pattern, they are tried. If all
2118     the alternatives match the empty string, the entire match fails. For
2119 nigel 73 example, if the pattern
2120 nigel 41
2121 nigel 73 a?b?
2122 nigel 41
2123 ph10 572 is applied to a string not beginning with "a" or "b", it matches an
2124     empty string at the start of the subject. With PCRE_NOTEMPTY set, this
2125 nigel 73 match is not valid, so PCRE searches further into the string for occur-
2126     rences of "a" or "b".
2127 nigel 41
2128 ph10 453 PCRE_NOTEMPTY_ATSTART
2129 nigel 41
2130 ph10 572 This is like PCRE_NOTEMPTY, except that an empty string match that is
2131     not at the start of the subject is permitted. If the pattern is
2132 ph10 453 anchored, such a match can occur only if the pattern contains \K.
2133    
2134 ph10 572 Perl has no direct equivalent of PCRE_NOTEMPTY or
2135     PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern
2136     match of the empty string within its split() function, and when using
2137     the /g modifier. It is possible to emulate Perl's behaviour after
2138 ph10 453 matching a null string by first trying the match again at the same off-
2139 ph10 572 set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that
2140 ph10 453 fails, by advancing the starting offset (see below) and trying an ordi-
2141 ph10 572 nary match again. There is some code that demonstrates how to do this
2142     in the pcredemo sample program. In the most general case, you have to
2143     check to see if the newline convention recognizes CRLF as a newline,
2144     and if so, and the current character is CR followed by LF, advance the
2145 ph10 567 starting offset by two characters instead of one.
2146 ph10 453
2147 ph10 392 PCRE_NO_START_OPTIMIZE
2148    
2149 ph10 572 There are a number of optimizations that pcre_exec() uses at the start
2150     of a match, in order to speed up the process. For example, if it is
2151 ph10 545 known that an unanchored match must start with a specific character, it
2152 ph10 572 searches the subject for that character, and fails immediately if it
2153     cannot find it, without actually running the main matching function.
2154 ph10 545 This means that a special item such as (*COMMIT) at the start of a pat-
2155 ph10 572 tern is not considered until after a suitable starting point for the
2156     match has been found. When callouts or (*MARK) items are in use, these
2157 ph10 548 "start-up" optimizations can cause them to be skipped if the pattern is
2158 ph10 572 never actually used. The start-up optimizations are in effect a pre-
2159 ph10 548 scan of the subject that takes place before the pattern is run.
2160 ph10 392
2161 ph10 572 The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations,
2162     possibly causing performance to suffer, but ensuring that in cases
2163     where the result is "no match", the callouts do occur, and that items
2164 ph10 548 such as (*COMMIT) and (*MARK) are considered at every possible starting
2165 ph10 572 position in the subject string. Setting PCRE_NO_START_OPTIMIZE can
2166 ph10 548 change the outcome of a matching operation. Consider the pattern
2167    
2168     (*COMMIT)ABC
2169    
2170 ph10 572 When this is compiled, PCRE records the fact that a match must start
2171     with the character "A". Suppose the subject string is "DEFABC". The
2172     start-up optimization scans along the subject, finds "A" and runs the
2173     first match attempt from there. The (*COMMIT) item means that the pat-
2174     tern must match the current starting position, which in this case, it
2175     does. However, if the same match is run with PCRE_NO_START_OPTIMIZE
2176     set, the initial scan along the subject string does not happen. The
2177     first match attempt is run starting from "D" and when this fails,
2178     (*COMMIT) prevents any further matches being tried, so the overall
2179     result is "no match". If the pattern is studied, more start-up opti-
2180     mizations may be used. For example, a minimum length for the subject
2181 ph10 548 may be recorded. Consider the pattern
2182    
2183     (*MARK:A)(X|Y)
2184    
2185 ph10 572 The minimum length for a match is one character. If the subject is
2186     "ABC", there will be attempts to match "ABC", "BC", "C", and then
2187     finally an empty string. If the pattern is studied, the final attempt
2188     does not take place, because PCRE knows that the subject is too short,
2189     and so the (*MARK) is never encountered. In this case, studying the
2190     pattern does not affect the overall match result, which is still "no
2191 ph10 548 match", but it does affect the auxiliary information that is returned.
2192    
2193 nigel 75 PCRE_NO_UTF8_CHECK
2194    
2195     When PCRE_UTF8 is set at compile time, the validity of the subject as a
2196 ph10 572 UTF-8 string is automatically checked when pcre_exec() is subsequently
2197     called. The value of startoffset is also checked to ensure that it
2198     points to the start of a UTF-8 character. There is a discussion about
2199     the validity of UTF-8 strings in the section on UTF-8 support in the
2200     main pcre page. If an invalid UTF-8 sequence of bytes is found,
2201     pcre_exec() returns the error PCRE_ERROR_BADUTF8 or, if PCRE_PAR-
2202     TIAL_HARD is set and the problem is a truncated UTF-8 character at the
2203     end of the subject, PCRE_ERROR_SHORTUTF8. If startoffset contains a
2204     value that does not point to the start of a UTF-8 character (or to the
2205     end of the subject), PCRE_ERROR_BADUTF8_OFFSET is returned.
2206 nigel 75
2207 ph10 572 If you already know that your subject is valid, and you want to skip
2208     these checks for performance reasons, you can set the
2209     PCRE_NO_UTF8_CHECK option when calling pcre_exec(). You might want to
2210     do this for the second and subsequent calls to pcre_exec() if you are
2211     making repeated calls to find all the matches in a single subject
2212     string. However, you should be sure that the value of startoffset
2213     points to the start of a UTF-8 character (or the end of the subject).
2214     When PCRE_NO_UTF8_CHECK is set, the effect of passing an invalid UTF-8
2215     string as a subject or an invalid value of startoffset is undefined.
2216 ph10 567 Your program may crash.
2217 nigel 75
2218 ph10 429 PCRE_PARTIAL_HARD
2219     PCRE_PARTIAL_SOFT
2220 nigel 75
2221 ph10 572 These options turn on the partial matching feature. For backwards com-
2222     patibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial
2223     match occurs if the end of the subject string is reached successfully,
2224     but there are not enough subject characters to complete the match. If
2225 ph10 567 this happens when PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set,
2226 ph10 572 matching continues by testing any remaining alternatives. Only if no
2227     complete match can be found is PCRE_ERROR_PARTIAL returned instead of
2228     PCRE_ERROR_NOMATCH. In other words, PCRE_PARTIAL_SOFT says that the
2229     caller is prepared to handle a partial match, but only if no complete
2230 ph10 567 match can be found.
2231 nigel 75
2232 ph10 572 If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this
2233     case, if a partial match is found, pcre_exec() immediately returns
2234     PCRE_ERROR_PARTIAL, without considering any other alternatives. In
2235     other words, when PCRE_PARTIAL_HARD is set, a partial match is consid-
2236 ph10 567 ered to be more important that an alternative complete match.
2237    
2238 ph10 572 In both cases, the portion of the string that was inspected when the
2239 ph10 567 partial match was found is set as the first matching string. There is a
2240 ph10 572 more detailed discussion of partial and multi-segment matching, with
2241 ph10 567 examples, in the pcrepartial documentation.
2242    
2243 nigel 75 The string to be matched by pcre_exec()
2244    
2245 ph10 572 The subject string is passed to pcre_exec() as a pointer in subject, a
2246 ph10 371 length (in bytes) in length, and a starting byte offset in startoffset.
2247 ph10 572 If this is negative or greater than the length of the subject,
2248     pcre_exec() returns PCRE_ERROR_BADOFFSET. When the starting offset is
2249     zero, the search for a match starts at the beginning of the subject,
2250     and this is by far the most common case. In UTF-8 mode, the byte offset
2251     must point to the start of a UTF-8 character (or the end of the sub-
2252     ject). Unlike the pattern string, the subject may contain binary zero
2253     bytes.
2254 ph10 567
2255     A non-zero starting offset is useful when searching for another match
2256     in the same subject by calling pcre_exec() again after a previous suc-
2257     cess. Setting startoffset differs from just passing over a shortened
2258     string and setting PCRE_NOTBOL in the case of a pattern that begins
2259 nigel 73 with any kind of lookbehind. For example, consider the pattern
2260 nigel 41
2261 nigel 73 \Biss\B
2262 nigel 41
2263 ph10 567 which finds occurrences of "iss" in the middle of words. (\B matches
2264     only if the current position in the subject is not a word boundary.)
2265     When applied to the string "Mississipi" the first call to pcre_exec()
2266     finds the first occurrence. If pcre_exec() is called again with just
2267     the remainder of the subject, namely "issipi", it does not match,
2268 nigel 73 because \B is always false at the start of the subject, which is deemed
2269 ph10 567 to be a word boundary. However, if pcre_exec() is passed the entire
2270 nigel 75 string again, but with startoffset set to 4, it finds the second occur-
2271 ph10 567 rence of "iss" because it is able to look behind the starting point to
2272 nigel 75 discover that it is preceded by a letter.
2273 nigel 41
2274 ph10 567 Finding all the matches in a subject is tricky when the pattern can
2275     match an empty string. It is possible to emulate Perl's /g behaviour by
2276     first trying the match again at the same offset, with the
2277     PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED options, and then if that
2278     fails, advancing the starting offset and trying an ordinary match
2279     again. There is some code that demonstrates how to do this in the pcre-
2280     demo sample program. In the most general case, you have to check to see
2281     if the newline convention recognizes CRLF as a newline, and if so, and
2282     the current character is CR followed by LF, advance the starting offset
2283     by two characters instead of one.
2284    
2285 ph10 548 If a non-zero starting offset is passed when the pattern is anchored,
2286 nigel 75 one attempt to match at the given offset is made. This can only succeed
2287 ph10 548 if the pattern does not require the match to be at the start of the
2288 nigel 75 subject.
2289 nigel 41
2290 nigel 75 How pcre_exec() returns captured substrings
2291    
2292 ph10 548 In general, a pattern matches a certain portion of the subject, and in
2293     addition, further substrings from the subject may be picked out by
2294     parts of the pattern. Following the usage in Jeffrey Friedl's book,
2295     this is called "capturing" in what follows, and the phrase "capturing
2296     subpattern" is used for a fragment of a pattern that picks out a sub-
2297     string. PCRE supports several other kinds of parenthesized subpattern
2298 nigel 73 that do not cause substrings to be captured.
2299 nigel 65
2300 ph10 371 Captured substrings are returned to the caller via a vector of integers
2301 ph10 548 whose address is passed in ovector. The number of elements in the vec-
2302     tor is passed in ovecsize, which must be a non-negative number. Note:
2303 ph10 371 this argument is NOT the size of ovector in bytes.
2304 nigel 41
2305 ph10 548 The first two-thirds of the vector is used to pass back captured sub-
2306     strings, each substring using a pair of integers. The remaining third
2307     of the vector is used as workspace by pcre_exec() while matching cap-
2308     turing subpatterns, and is not available for passing back information.
2309     The number passed in ovecsize should always be a multiple of three. If
2310 nigel 75 it is not, it is rounded down.
2311    
2312 ph10 548 When a match is successful, information about captured substrings is
2313     returned in pairs of integers, starting at the beginning of ovector,
2314     and continuing up to two-thirds of its length at the most. The first
2315     element of each pair is set to the byte offset of the first character
2316     in a substring, and the second is set to the byte offset of the first
2317     character after the end of a substring. Note: these values are always
2318 ph10 371 byte offsets, even in UTF-8 mode. They are not character counts.
2319 nigel 41
2320 ph10 548 The first pair of integers, ovector[0] and ovector[1], identify the
2321     portion of the subject string matched by the entire pattern. The next
2322     pair is used for the first capturing subpattern, and so on. The value
2323 ph10 371 returned by pcre_exec() is one more than the highest numbered pair that
2324 ph10 548 has been set. For example, if two substrings have been captured, the
2325     returned value is 3. If there are no capturing subpatterns, the return
2326 ph10 371 value from a successful match is 1, indicating that just the first pair
2327     of offsets has been set.
2328    
2329 nigel 73 If a capturing subpattern is matched repeatedly, it is the last portion
2330 nigel 75 of the string that it matched that is returned.
2331 nigel 41
2332 ph10 548 If the vector is too small to hold all the captured substring offsets,
2333 nigel 75 it is used as far as possible (up to two-thirds of its length), and the
2334 ph10 548 function returns a value of zero. If the substring offsets are not of
2335     interest, pcre_exec() may be called with ovector passed as NULL and
2336     ovecsize as zero. However, if the pattern contains back references and
2337     the ovector is not big enough to remember the related substrings, PCRE
2338     has to get additional memory for use during matching. Thus it is usu-
2339 ph10 371 ally advisable to supply an ovector.
2340 nigel 41
2341 ph10 461 The pcre_fullinfo() function can be used to find out how many capturing
2342 ph10 548 subpatterns there are in a compiled pattern. The smallest size for
2343     ovector that will allow for n captured substrings, in addition to the
2344 nigel 91 offsets of the substring matched by the whole pattern, is (n+1)*3.
2345 nigel 41
2346 ph10 548 It is possible for capturing subpattern number n+1 to match some part
2347 nigel 91 of the subject when subpattern n has not been used at all. For example,
2348 ph10 548 if the string "abc" is matched against the pattern (a|(z))(bc) the
2349 nigel 91 return from the function is 4, and subpatterns 1 and 3 are matched, but
2350 ph10 548 2 is not. When this happens, both values in the offset pairs corre-
2351 nigel 91 sponding to unused subpatterns are set to -1.
2352 nigel 75
2353 ph10 548 Offset values that correspond to unused subpatterns at the end of the
2354     expression are also set to -1. For example, if the string "abc" is
2355     matched against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not
2356     matched. The return from the function is 2, because the highest used
2357 ph10 572 capturing subpattern number is 1, and the offsets for for the second
2358     and third capturing subpatterns (assuming the vector is large enough,
2359     of course) are set to -1.
2360 nigel 91
2361 ph10 572 Note: Elements of ovector that do not correspond to capturing parenthe-
2362     ses in the pattern are never changed. That is, if a pattern contains n
2363     capturing parentheses, no more than ovector[0] to ovector[2n+1] are set
2364     by pcre_exec(). The other elements retain whatever values they previ-
2365     ously had.
2366    
2367 ph10 548 Some convenience functions are provided for extracting the captured
2368 nigel 91 substrings as separate strings. These are described below.
2369    
2370     Error return values from pcre_exec()
2371    
2372 ph10 548 If pcre_exec() fails, it returns a negative number. The following are
2373 nigel 73 defined in the header file:
2374 nigel 41
2375 nigel 73 PCRE_ERROR_NOMATCH (-1)
2376 nigel 41
2377 nigel 73 The subject string did not match the pattern.
2378 nigel 41
2379 nigel 73 PCRE_ERROR_NULL (-2)
2380 nigel 41
2381 ph10 548 Either code or subject was passed as NULL, or ovector was NULL and
2382 nigel 73 ovecsize was not zero.
2383 nigel 41
2384 nigel 73 PCRE_ERROR_BADOPTION (-3)
2385 nigel 41
2386 nigel 73 An unrecognized bit was set in the options argument.
2387 nigel 41
2388 nigel 73 PCRE_ERROR_BADMAGIC (-4)
2389 nigel 41
2390 ph10 548 PCRE stores a 4-byte "magic number" at the start of the compiled code,
2391 nigel 75 to catch the case when it is passed a junk pointer and to detect when a
2392     pattern that was compiled in an environment of one endianness is run in
2393 ph10 548 an environment with the other endianness. This is the error that PCRE
2394 nigel 75 gives when the magic number is not present.
2395 nigel 41
2396 nigel 93 PCRE_ERROR_UNKNOWN_OPCODE (-5)
2397 nigel 41
2398 nigel 73 While running the pattern match, an unknown item was encountered in the
2399 ph10 548 compiled pattern. This error could be caused by a bug in PCRE or by
2400 nigel 73 overwriting of the compiled pattern.
2401 nigel 41
2402 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2403 nigel 41
2404 ph10 548 If a pattern contains back references, but the ovector that is passed
2405 nigel 73 to pcre_exec() is not big enough to remember the referenced substrings,
2406 ph10 548 PCRE gets a block of memory at the start of matching to use for this
2407     purpose. If the call via pcre_malloc() fails, this error is given. The
2408 nigel 75 memory is automatically freed at the end of matching.
2409 nigel 41
2410 ph10 548 This error is also given if pcre_stack_malloc() fails in pcre_exec().
2411     This can happen only when PCRE has been compiled with --disable-stack-
2412 ph10 535 for-recursion.
2413    
2414 nigel 73 PCRE_ERROR_NOSUBSTRING (-7)
2415 nigel 53
2416 ph10 548 This error is used by the pcre_copy_substring(), pcre_get_substring(),
2417 nigel 73 and pcre_get_substring_list() functions (see below). It is never
2418     returned by pcre_exec().
2419 nigel 63
2420 nigel 73 PCRE_ERROR_MATCHLIMIT (-8)
2421 nigel 63
2422 ph10 548 The backtracking limit, as specified by the match_limit field in a
2423     pcre_extra structure (or defaulted) was reached. See the description
2424 nigel 87 above.
2425    
2426 nigel 73 PCRE_ERROR_CALLOUT (-9)
2427 nigel 63
2428 nigel 73 This error is never generated by pcre_exec() itself. It is provided for
2429 ph10 548 use by callout functions that want to yield a distinctive error code.
2430 nigel 73 See the pcrecallout documentation for details.
2431 nigel 71
2432 nigel 73 PCRE_ERROR_BADUTF8 (-10)
2433 nigel 71
2434 ph10 548 A string that contains an invalid UTF-8 byte sequence was passed as a
2435 ph10 572 subject. However, if PCRE_PARTIAL_HARD is set and the problem is a
2436     truncated UTF-8 character at the end of the subject, PCRE_ERROR_SHORT-
2437     UTF8 is used instead.
2438 nigel 73
2439     PCRE_ERROR_BADUTF8_OFFSET (-11)
2440    
2441     The UTF-8 byte sequence that was passed as a subject was valid, but the
2442 ph10 548 value of startoffset did not point to the beginning of a UTF-8 charac-
2443 ph10 572 ter or the end of the subject.
2444 nigel 73
2445 nigel 77 PCRE_ERROR_PARTIAL (-12)
2446 nigel 73
2447 ph10 548 The subject string did not match, but it did match partially. See the
2448 nigel 75 pcrepartial documentation for details of partial matching.
2449    
2450 nigel 77 PCRE_ERROR_BADPARTIAL (-13)
2451 nigel 75
2452 ph10 548 This code is no longer in use. It was formerly returned when the
2453     PCRE_PARTIAL option was used with a compiled pattern containing items
2454     that were not supported for partial matching. From release 8.00
2455 ph10 429 onwards, there are no restrictions on partial matching.
2456 nigel 75
2457 nigel 77 PCRE_ERROR_INTERNAL (-14)
2458 nigel 75
2459 ph10 548 An unexpected internal error has occurred. This error could be caused
2460 nigel 75 by a bug in PCRE or by overwriting of the compiled pattern.
2461    
2462 nigel 77 PCRE_ERROR_BADCOUNT (-15)
2463 nigel 75
2464 ph10 392 This error is given if the value of the ovecsize argument is negative.
2465 nigel 75
2466 nigel 93 PCRE_ERROR_RECURSIONLIMIT (-21)
2467 nigel 75
2468 nigel 93 The internal recursion limit, as specified by the match_limit_recursion
2469 ph10 548 field in a pcre_extra structure (or defaulted) was reached. See the
2470 nigel 93 description above.
2471    
2472     PCRE_ERROR_BADNEWLINE (-23)
2473    
2474     An invalid combination of PCRE_NEWLINE_xxx options was given.
2475    
2476 ph10 567 PCRE_ERROR_BADOFFSET (-24)
2477    
2478     The value of startoffset was negative or greater than the length of the
2479     subject, that is, the value in length.
2480    
2481 ph10 572 PCRE_ERROR_SHORTUTF8 (-25)
2482    
2483     The subject string ended with an incomplete (truncated) UTF-8 charac-
2484     ter, and the PCRE_PARTIAL_HARD option was set. Without this option,
2485     PCRE_ERROR_BADUTF8 is returned in this situation.
2486    
2487 ph10 197 Error numbers -16 to -20 and -22 are not used by pcre_exec().
2488 nigel 93
2489    
2490 nigel 63 EXTRACTING CAPTURED SUBSTRINGS BY NUMBER
2491    
2492 nigel 73 int pcre_copy_substring(const char *subject, int *ovector,
2493     int stringcount, int stringnumber, char *buffer,
2494     int buffersize);
2495 nigel 63
2496 nigel 73 int pcre_get_substring(const char *subject, int *ovector,
2497     int stringcount, int stringnumber,
2498     const char **stringptr);
2499 nigel 63
2500 nigel 73 int pcre_get_substring_list(const char *subject,
2501     int *ovector, int stringcount, const char ***listptr);
2502 nigel 63
2503 ph10 567 Captured substrings can be accessed directly by using the offsets
2504     returned by pcre_exec() in ovector. For convenience, the functions
2505 nigel 73 pcre_copy_substring(), pcre_get_substring(), and pcre_get_sub-
2506 ph10 567 string_list() are provided for extracting captured substrings as new,
2507     separate, zero-terminated strings. These functions identify substrings
2508     by number. The next section describes functions for extracting named
2509 nigel 91 substrings.
2510 nigel 41
2511 ph10 567 A substring that contains a binary zero is correctly extracted and has
2512     a further zero added on the end, but the result is not, of course, a C
2513     string. However, you can process such a string by referring to the
2514     length that is returned by pcre_copy_substring() and pcre_get_sub-
2515 nigel 91 string(). Unfortunately, the interface to pcre_get_substring_list() is
2516 ph10 567 not adequate for handling strings containing binary zeros, because the
2517 nigel 91 end of the final string is not independently indicated.
2518    
2519 ph10 567 The first three arguments are the same for all three of these func-
2520     tions: subject is the subject string that has just been successfully
2521 nigel 73 matched, ovector is a pointer to the vector of integer offsets that was
2522     passed to pcre_exec(), and stringcount is the number of substrings that
2523 ph10 567 were captured by the match, including the substring that matched the
2524 nigel 75 entire regular expression. This is the value returned by pcre_exec() if
2525 ph10 567 it is greater than zero. If pcre_exec() returned zero, indicating that
2526     it ran out of space in ovector, the value passed as stringcount should
2527 nigel 75 be the number of elements in the vector divided by three.
2528 nigel 41
2529 ph10 567 The functions pcre_copy_substring() and pcre_get_substring() extract a
2530     single substring, whose number is given as stringnumber. A value of
2531     zero extracts the substring that matched the entire pattern, whereas
2532     higher values extract the captured substrings. For pcre_copy_sub-
2533     string(), the string is placed in buffer, whose length is given by
2534     buffersize, while for pcre_get_substring() a new block of memory is
2535     obtained via pcre_malloc, and its address is returned via stringptr.
2536     The yield of the function is the length of the string, not including
2537 nigel 93 the terminating zero, or one of these error codes:
2538 nigel 41
2539 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2540 nigel 41
2541 ph10 567 The buffer was too small for pcre_copy_substring(), or the attempt to
2542 nigel 73 get memory failed for pcre_get_substring().
2543 nigel 41
2544 nigel 73 PCRE_ERROR_NOSUBSTRING (-7)
2545 nigel 41
2546 nigel 73 There is no substring whose number is stringnumber.
2547 nigel 41
2548 ph10 567 The pcre_get_substring_list() function extracts all available sub-
2549     strings and builds a list of pointers to them. All this is done in a
2550 nigel 75 single block of memory that is obtained via pcre_malloc. The address of
2551 ph10 567 the memory block is returned via listptr, which is also the start of
2552     the list of string pointers. The end of the list is marked by a NULL
2553     pointer. The yield of the function is zero if all went well, or the
2554 nigel 93 error code
2555 nigel 41
2556 nigel 73 PCRE_ERROR_NOMEMORY (-6)
2557 nigel 41
2558 nigel 73 if the attempt to get the memory block failed.
2559 nigel 41
2560 ph10 567 When any of these functions encounter a substring that is unset, which
2561     can happen when capturing subpattern number n+1 matches some part of
2562     the subject, but subpattern n has not been used at all, they return an
2563 nigel 73 empty string. This can be distinguished from a genuine zero-length sub-
2564 ph10 567 string by inspecting the appropriate offset in ovector, which is nega-
2565 nigel 73 tive for unset substrings.
2566 nigel 41
2567 ph10 567 The two convenience functions pcre_free_substring() and pcre_free_sub-
2568     string_list() can be used to free the memory returned by a previous
2569 nigel 75 call of pcre_get_substring() or pcre_get_substring_list(), respec-
2570 ph10 567 tively. They do nothing more than call the function pointed to by
2571     pcre_free, which of course could be called directly from a C program.
2572     However, PCRE is used in some situations where it is linked via a spe-
2573     cial interface to another programming language that cannot use
2574     pcre_free directly; it is for these cases that the functions are pro-
2575 nigel 77 vided.
2576 nigel 41
2577 nigel 73
2578 nigel 63 EXTRACTING CAPTURED SUBSTRINGS BY NAME
2579 nigel 41
2580 nigel 75 int pcre_get_stringnumber(const pcre *code,
2581     const char *name);
2582    
2583 nigel 73 int pcre_copy_named_substring(const pcre *code,
2584     const char *subject, int *ovector,
2585     int stringcount, const char *stringname,
2586     char *buffer, int buffersize);
2587 nigel 41
2588 nigel 73 int pcre_get_named_substring(const pcre *code,
2589     const char *subject, int *ovector,
2590     int stringcount, const char *stringname,
2591     const char **stringptr);
2592 nigel 41
2593 ph10 567 To extract a substring by name, you first have to find associated num-
2594 nigel 75 ber. For example, for this pattern
2595 nigel 41
2596 nigel 93 (a+)b(?<xxx>\d+)...
2597 nigel 63
2598 nigel 91 the number of the subpattern called "xxx" is 2. If the name is known to
2599     be unique (PCRE_DUPNAMES was not set), you can find the number from the
2600     name by calling pcre_get_stringnumber(). The first argument is the com-
2601     piled pattern, and the second is the name. The yield of the function is
2602 ph10 567 the subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no
2603 nigel 91 subpattern of that name.
2604 nigel 63
2605 nigel 75 Given the number, you can extract the substring directly, or use one of
2606     the functions described in the previous section. For convenience, there
2607     are also two functions that do the whole job.
2608    
2609 ph10 567 Most of the arguments of pcre_copy_named_substring() and
2610     pcre_get_named_substring() are the same as those for the similarly
2611     named functions that extract by number. As these are described in the
2612     previous section, they are not re-described here. There are just two
2613 nigel 75 differences:
2614 nigel 63
2615 ph10 567 First, instead of a substring number, a substring name is given. Sec-
2616 nigel 73 ond, there is an extra argument, given at the start, which is a pointer
2617 ph10 567 to the compiled pattern. This is needed in order to gain access to the
2618 nigel 73 name-to-number translation table.
2619 nigel 63
2620 ph10 567 These functions call pcre_get_stringnumber(), and if it succeeds, they
2621     then call pcre_copy_substring() or pcre_get_substring(), as appropri-
2622     ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the
2623 ph10 128 behaviour may not be what you want (see the next section).
2624 nigel 63
2625 ph10 461 Warning: If the pattern uses the (?| feature to set up multiple subpat-
2626 ph10 567 terns with the same number, as described in the section on duplicate
2627     subpattern numbers in the pcrepattern page, you cannot use names to
2628     distinguish the different subpatterns, because names are not included
2629     in the compiled code. The matching process uses only numbers. For this
2630     reason, the use of different names for subpatterns of the same number
2631 ph10 461 causes an error at compile time.
2632 nigel 77
2633 ph10 392
2634 nigel 91 DUPLICATE SUBPATTERN NAMES
2635    
2636     int pcre_get_stringtable_entries(const pcre *code,
2637     const char *name, char **first, char **last);
2638    
2639 ph10 567 When a pattern is compiled with the PCRE_DUPNAMES option, names for
2640     subpatterns are not required to be unique. (Duplicate names are always
2641     allowed for subpatterns with the same number, created by using the (?|
2642     feature. Indeed, if such subpatterns are named, they are required to
2643 ph10 461 use the same names.)
2644 ph10 208
2645 ph10 461 Normally, patterns with duplicate names are such that in any one match,
2646 ph10 567 only one of the named subpatterns participates. An example is shown in
2647 ph10 461 the pcrepattern documentation.
2648    
2649 ph10 567 When duplicates are present, pcre_copy_named_substring() and
2650     pcre_get_named_substring() return the first substring corresponding to
2651     the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING
2652     (-7) is returned; no data is returned. The pcre_get_stringnumber()
2653     function returns one of the numbers that are associated with the name,
2654 ph10 208 but it is not defined which it is.
2655 nigel 91
2656 ph10 567 If you want to get full details of all captured substrings for a given
2657     name, you must use the pcre_get_stringtable_entries() function. The
2658 nigel 91 first argument is the compiled pattern, and the second is the name. The
2659 ph10 567 third and fourth are pointers to variables which are updated by the
2660 nigel 91 function. After it has run, they point to the first and last entries in
2661 ph10 567 the name-to-number table for the given name. The function itself
2662     returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if
2663     there are none. The format of the table is described above in the sec-
2664     tion entitled Information about a pattern. Given all the relevant
2665     entries for the name, you can extract each of their numbers, and hence
2666 nigel 93 the captured data, if any.
2667 nigel 91
2668    
2669 nigel 77 FINDING ALL POSSIBLE MATCHES
2670    
2671 ph10 567 The traditional matching function uses a similar algorithm to Perl,
2672 nigel 77 which stops when it finds the first match, starting at a given point in
2673 ph10 567 the subject. If you want to find all possible matches, or the longest
2674     possible match, consider using the alternative matching function (see
2675     below) instead. If you cannot use the alternative function, but still
2676     need to find all possible matches, you can kludge it up by making use
2677 nigel 77 of the callout facility, which is described in the pcrecallout documen-
2678     tation.
2679    
2680     What you have to do is to insert a callout right at the end of the pat-
2681 ph10 567 tern. When your callout function is called, extract and save the cur-
2682     rent matched substring. Then return 1, which forces pcre_exec() to
2683     backtrack and try other alternatives. Ultimately, when it runs out of
2684 nigel 77 matches, pcre_exec() will yield PCRE_ERROR_NOMATCH.
2685    
2686    
2687     MATCHING A PATTERN: THE ALTERNATIVE FUNCTION
2688    
2689     int pcre_dfa_exec(const pcre *code, const pcre_extra *extra,
2690     const char *subject, int length, int startoffset,
2691     int options, int *ovector, int ovecsize,
2692     int *workspace, int wscount);
2693    
2694 ph10 567 The function pcre_dfa_exec() is called to match a subject string
2695     against a compiled pattern, using a matching algorithm that scans the
2696     subject string just once, and does not backtrack. This has different
2697     characteristics to the normal algorithm, and is not compatible with
2698     Perl. Some of the features of PCRE patterns are not supported. Never-
2699     theless, there are times when this kind of matching can be useful. For
2700     a discussion of the two matching algorithms, and a list of features
2701     that pcre_dfa_exec() does not support, see the pcrematching documenta-
2702 ph10 453 tion.
2703 nigel 77
2704 ph10 567 The arguments for the pcre_dfa_exec() function are the same as for
2705 nigel 77 pcre_exec(), plus two extras. The ovector argument is used in a differ-
2706 ph10 567 ent way, and this is described below. The other common arguments are
2707     used in the same way as for pcre_exec(), so their description is not
2708 nigel 77 repeated here.
2709    
2710 ph10 567 The two additional arguments provide workspace for the function. The
2711     workspace vector should contain at least 20 elements. It is used for
2712 nigel 77 keeping track of multiple paths through the pattern tree. More
2713 ph10 567 workspace will be needed for patterns and subjects where there are a
2714 nigel 91 lot of potential matches.
2715 nigel 77
2716 nigel 87 Here is an example of a simple call to pcre_dfa_exec():
2717 nigel 77
2718     int rc;
2719     int ovector[10];
2720     int wspace[20];
2721 nigel 87 rc = pcre_dfa_exec(
2722 nigel 77 re, /* result of pcre_compile() */
2723     NULL, /* we didn't study the pattern */
2724     "some string", /* the subject string */
2725     11, /* the length of the subject string */
2726     0, /* start at offset 0 in the subject */
2727     0, /* default options */
2728     ovector, /* vector of integers for substring information */
2729     10, /* number of elements (NOT size in bytes) */
2730     wspace, /* working space vector */
2731     20); /* number of elements (NOT size in bytes) */
2732    
2733     Option bits for pcre_dfa_exec()
2734    
2735 ph10 567 The unused bits of the options argument for pcre_dfa_exec() must be
2736     zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW-
2737 ph10 453 LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY,
2738 ph10 567 PCRE_NOTEMPTY_ATSTART, PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF,
2739     PCRE_BSR_UNICODE, PCRE_NO_START_OPTIMIZE, PCRE_PARTIAL_HARD, PCRE_PAR-
2740     TIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last
2741     four of these are exactly the same as for pcre_exec(), so their
2742 ph10 453 description is not repeated here.
2743 nigel 77
2744 ph10 429 PCRE_PARTIAL_HARD
2745     PCRE_PARTIAL_SOFT
2746 nigel 77
2747 ph10 567 These have the same general effect as they do for pcre_exec(), but the
2748     details are slightly different. When PCRE_PARTIAL_HARD is set for
2749     pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub-
2750     ject is reached and there is still at least one matching possibility
2751 ph10 429 that requires additional characters. This happens even if some complete
2752     matches have also been found. When PCRE_PARTIAL_SOFT is set, the return
2753     code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end
2754 ph10 567 of the subject is reached, there have been no complete matches, but
2755     there is still at least one matching possibility. The portion of the
2756     string that was inspected when the longest partial match was found is
2757     set as the first matching string in both cases. There is a more
2758     detailed discussion of partial and multi-segment matching, with exam-
2759     ples, in the pcrepartial documentation.
2760 nigel 77
2761     PCRE_DFA_SHORTEST
2762    
2763 ph10 567 Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to
2764 nigel 93 stop as soon as it has found one match. Because of the way the alterna-
2765 ph10 567 tive algorithm works, this is necessarily the shortest possible match
2766 nigel 93 at the first possible matching point in the subject string.
2767 nigel 77
2768     PCRE_DFA_RESTART
2769    
2770 ph10 429 When pcre_dfa_exec() returns a partial match, it is possible to call it
2771 ph10 567 again, with additional subject characters, and have it continue with
2772     the same match. The PCRE_DFA_RESTART option requests this action; when
2773     it is set, the workspace and wscount options must reference the same
2774     vector as before because data about the match so far is left in them
2775 ph10 429 after a partial match. There is more discussion of this facility in the
2776     pcrepartial documentation.
2777 nigel 77
2778     Successful returns from pcre_dfa_exec()
2779    
2780 ph10 567 When pcre_dfa_exec() succeeds, it may have matched more than one sub-
2781 nigel 77 string in the subject. Note, however, that all the matches from one run
2782 ph10 567 of the function start at the same point in the subject. The shorter
2783     matches are all initial substrings of the longer matches. For example,
2784 nigel 77 if the pattern
2785    
2786     <.*>
2787    
2788     is matched against the string
2789    
2790     This is <something> <something else> <something further> no more
2791    
2792     the three matched strings are
2793    
2794     <something>
2795     <something> <something else>
2796     <something> <something else> <something further>
2797    
2798 ph10 567 On success, the yield of the function is a number greater than zero,
2799     which is the number of matched substrings. The substrings themselves
2800     are returned in ovector. Each string uses two elements; the first is
2801     the offset to the start, and the second is the offset to the end. In
2802     fact, all the strings have the same start offset. (Space could have
2803     been saved by giving this only once, but it was decided to retain some
2804     compatibility with the way pcre_exec() returns data, even though the
2805 nigel 93 meaning of the strings is different.)
2806 nigel 77
2807     The strings are returned in reverse order of length; that is, the long-
2808 ph10 567 est matching string is given first. If there were too many matches to
2809     fit into ovector, the yield of the function is zero, and the vector is
2810 nigel 77 filled with the longest matches.
2811    
2812     Error returns from pcre_dfa_exec()
2813    
2814 ph10 567 The pcre_dfa_exec() function returns a negative number when it fails.
2815     Many of the errors are the same as for pcre_exec(), and these are
2816     described above. There are in addition the following errors that are
2817 nigel 77 specific to pcre_dfa_exec():
2818    
2819     PCRE_ERROR_DFA_UITEM (-16)
2820    
2821 ph10 567 This return is given if pcre_dfa_exec() encounters an item in the pat-
2822     tern that it does not support, for instance, the use of \C or a back
2823 nigel 77 reference.
2824    
2825     PCRE_ERROR_DFA_UCOND (-17)
2826    
2827 ph10 567 This return is given if pcre_dfa_exec() encounters a condition item
2828     that uses a back reference for the condition, or a test for recursion
2829 nigel 93 in a specific group. These are not supported.
2830 nigel 77
2831     PCRE_ERROR_DFA_UMLIMIT (-18)
2832    
2833 ph10 567 This return is given if pcre_dfa_exec() is called with an extra block
2834 nigel 77 that contains a setting of the match_limit field. This is not supported
2835     (it is meaningless).
2836    
2837     PCRE_ERROR_DFA_WSSIZE (-19)
2838    
2839 ph10 567 This return is given if pcre_dfa_exec() runs out of space in the
2840 nigel 77 workspace vector.
2841    
2842     PCRE_ERROR_DFA_RECURSE (-20)
2843    
2844 ph10 567 When a recursive subpattern is processed, the matching function calls
2845     itself recursively, using private vectors for ovector and workspace.
2846     This error is given if the output vector is not large enough. This
2847 nigel 77 should be extremely rare, as a vector of size 1000 is used.
2848    
2849 nigel 93
2850     SEE ALSO
2851    
2852 ph10 567 pcrebuild(3), pcrecallout(3), pcrecpp(3)(3), pcrematching(3), pcrepar-
2853 ph10 392 tial(3), pcreposix(3), pcreprecompile(3), pcresample(3), pcrestack(3).
2854 nigel 93
2855 nigel 63
2856 ph10 99 AUTHOR
2857 nigel 63
2858 ph10 99 Philip Hazel
2859     University Computing Service
2860     Cambridge CB2 3QH, England.
2861    
2862    
2863     REVISION
2864    
2865 ph10 572 Last updated: 13 November 2010
2866 ph10 512 Copyright (c) 1997-2010 University of Cambridge.
2867 ph10 99 ------------------------------------------------------------------------------
2868 ph10 567
2869    
2870 nigel 79 PCRECALLOUT(3) PCRECALLOUT(3)
2871 nigel 63
2872 nigel 79
2873 nigel 73 NAME
2874     PCRE - Perl-compatible regular expressions
2875    
2876 nigel 77
2877 nigel 63 PCRE CALLOUTS
2878    
2879 nigel 73 int (*pcre_callout)(pcre_callout_block *);
2880 nigel 63
2881 nigel 73 PCRE provides a feature called "callout", which is a means of temporar-
2882     ily passing control to the caller of PCRE in the middle of pattern
2883     matching. The caller of PCRE provides an external function by putting
2884     its entry point in the global variable pcre_callout. By default, this
2885     variable contains NULL, which disables all calling out.
2886 nigel 63
2887 nigel 73 Within a regular expression, (?C) indicates the points at which the
2888     external function is to be called. Different callout points can be
2889     identified by putting a number less than 256 after the letter C. The
2890     default value is zero. For example, this pattern has two callout
2891     points:
2892 nigel 63
2893 ph10 155 (?C1)abc(?C2)def
2894 nigel 63
2895 ph10 461 If the PCRE_AUTO_CALLOUT option bit is set when pcre_compile() or
2896     pcre_compile2() is called, PCRE automatically inserts callouts, all
2897     with number 255, before each item in the pattern. For example, if
2898     PCRE_AUTO_CALLOUT is used with the pattern
2899 nigel 63
2900 nigel 75 A(\d{2}|--)
2901    
2902     it is processed as if it were
2903    
2904     (?C255)A(?C255)((?C255)\d{2}(?C255)|(?C255)-(?C255)-(?C255))(?C255)
2905    
2906     Notice that there is a callout before and after each parenthesis and
2907     alternation bar. Automatic callouts can be used for tracking the
2908     progress of pattern matching. The pcretest command has an option that
2909     sets automatic callouts; when it is used, the output indicates how the
2910     pattern is matched. This is useful information when you are trying to
2911     optimize the performance of a particular pattern.
2912    
2913    
2914     MISSING CALLOUTS
2915    
2916     You should be aware that, because of optimizations in the way PCRE
2917 ph10 392 matches patterns by default, callouts sometimes do not happen. For
2918     example, if the pattern is
2919 nigel 75
2920     ab(?C4)cd
2921    
2922     PCRE knows that any matching string must contain the letter "d". If the
2923     subject string is "abyz", the lack of "d" means that matching doesn't
2924     ever start, and the callout is never reached. However, with "abyd",
2925     though the result is still no match, the callout is obeyed.
2926    
2927 ph10 461 If the pattern is studied, PCRE knows the minimum length of a matching
2928     string, and will immediately give a "no match" return without actually
2929     running a match if the subject is not long enough, or, for unanchored
2930     patterns, if it has been scanned far enough.
2931    
2932     You can disable these optimizations by passing the PCRE_NO_START_OPTI-
2933     MIZE option to pcre_exec() or pcre_dfa_exec(). This slows down the
2934     matching process, but does ensure that callouts such as the example
2935 ph10 392 above are obeyed.
2936 nigel 75
2937 ph10 392
2938 nigel 75 THE CALLOUT INTERFACE
2939    
2940 ph10 461 During matching, when PCRE reaches a callout point, the external func-
2941     tion defined by pcre_callout is called (if it is set). This applies to
2942     both the pcre_exec() and the pcre_dfa_exec() matching functions. The
2943     only argument to the callout function is a pointer to a pcre_callout
2944 nigel 77 block. This structure contains the following fields:
2945 nigel 75
2946 nigel 73 int version;
2947     int callout_number;
2948     int *offset_vector;
2949     const char *subject;
2950     int subject_length;
2951     int start_match;
2952     int current_position;
2953     int capture_top;
2954     int capture_last;
2955     void *callout_data;
2956 nigel 75 int pattern_position;
2957     int next_item_length;
2958 nigel 63
2959 ph10 461 The version field is an integer containing the version number of the
2960     block format. The initial version was 0; the current version is 1. The
2961     version number will change again in future if additional fields are
2962 nigel 75 added, but the intention is never to remove any of the existing fields.
2963 nigel 63
2964 ph10 461 The callout_number field contains the number of the callout, as com-
2965     piled into the pattern (that is, the number after ?C for manual call-
2966 nigel 75 outs, and 255 for automatically generated callouts).
2967 nigel 63
2968 ph10 461 The offset_vector field is a pointer to the vector of offsets that was
2969     passed by the caller to pcre_exec() or pcre_dfa_exec(). When
2970     pcre_exec() is used, the contents can be inspected in order to extract
2971     substrings that have been matched so far, in the same way as for
2972     extracting substrings after a match has completed. For pcre_dfa_exec()
2973 nigel 77 this field is not useful.
2974 nigel 63
2975 nigel 75 The subject and subject_length fields contain copies of the values that
2976 nigel 73 were passed to pcre_exec().
2977 nigel 63
2978 ph10 461 The start_match field normally contains the offset within the subject
2979     at which the current match attempt started. However, if the escape
2980     sequence \K has been encountered, this value is changed to reflect the
2981     modified starting point. If the pattern is not anchored, the callout
2982 ph10 172 function may be called several times from the same point in the pattern
2983     for different starting points in the subject.
2984 nigel 63
2985 ph10 461 The current_position field contains the offset within the subject of
2986 nigel 73 the current match pointer.
2987 nigel 63
2988 ph10 461 When the pcre_exec() function is used, the capture_top field contains
2989     one more than the number of the highest numbered captured substring so
2990     far. If no substrings have been captured, the value of capture_top is
2991     one. This is always the case when pcre_dfa_exec() is used, because it
2992 nigel 77 does not support captured substrings.
2993 nigel 63
2994 ph10 461 The capture_last field contains the number of the most recently cap-
2995     tured substring. If no substrings have been captured, its value is -1.
2996 nigel 77 This is always the case when pcre_dfa_exec() is used.
2997 nigel 63
2998 ph10 461 The callout_data field contains a value that is passed to pcre_exec()
2999     or pcre_dfa_exec() specifically so that it can be passed back in call-
3000     outs. It is passed in the pcre_callout field of the pcre_extra data
3001     structure. If no such data was passed, the value of callout_data in a
3002     pcre_callout block is NULL. There is a description of the pcre_extra
3003 nigel 73 structure in the pcreapi documentation.
3004 nigel 63
3005 ph10 461 The pattern_position field is present from version 1 of the pcre_call-
3006 nigel 75 out structure. It contains the offset to the next item to be matched in
3007     the pattern string.
3008 nigel 63
3009 ph10 461 The next_item_length field is present from version 1 of the pcre_call-
3010 nigel 75 out structure. It contains the length of the next item to be matched in
3011 ph10 461 the pattern string. When the callout immediately precedes an alterna-
3012     tion bar, a closing parenthesis, or the end of the pattern, the length
3013     is zero. When the callout precedes an opening parenthesis, the length
3014 nigel 75 is that of the entire subpattern.
3015 nigel 73
3016 ph10 461 The pattern_position and next_item_length fields are intended to help
3017     in distinguishing between different automatic callouts, which all have
3018 nigel 75 the same callout number. However, they are set for all callouts.
3019    
3020    
3021 nigel 63 RETURN VALUES
3022    
3023 ph10 461 The external callout function returns an integer to PCRE. If the value
3024     is zero, matching proceeds as normal. If the value is greater than
3025     zero, matching fails at the current point, but the testing of other
3026 nigel 77 matching possibilities goes ahead, just as if a lookahead assertion had
3027 ph10 461 failed. If the value is less than zero, the match is abandoned, and
3028     pcre_exec() or pcre_dfa_exec() returns the negative value.
3029 nigel 63
3030 ph10 461 Negative values should normally be chosen from the set of
3031 nigel 73 PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan-
3032 ph10 461 dard "no match" failure. The error number PCRE_ERROR_CALLOUT is
3033     reserved for use by callout functions; it will never be used by PCRE
3034 nigel 73 itself.
3035 nigel 63
3036    
3037 ph10 99 AUTHOR
3038 nigel 63
3039 ph10 99 Philip Hazel
3040     University Computing Service
3041     Cambridge CB2 3QH, England.
3042    
3043    
3044     REVISION
3045    
3046 ph10 461 Last updated: 29 September 2009
3047 ph10 392 Copyright (c) 1997-2009 University of Cambridge.
3048 ph10 99 ------------------------------------------------------------------------------
3049 ph10 567
3050    
3051 nigel 79 PCRECOMPAT(3) PCRECOMPAT(3)
3052 nigel 63
3053 nigel 79
3054 nigel 73 NAME
3055     PCRE - Perl-compatible regular expressions
3056    
3057 nigel 77
3058 nigel 75 DIFFERENCES BETWEEN PCRE AND PERL
3059 nigel 41
3060 nigel 73 This document describes the differences in the ways that PCRE and Perl
3061 ph10 461 handle regular expressions. The differences described here are with
3062 ph10 567 respect to Perl versions 5.10 and above.
3063 nigel 41
3064 ph10 461 1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details
3065     of what it does have are given in the section on UTF-8 support in the
3066 nigel 87 main pcre page.
3067 nigel 41
3068 nigel 73 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl
3069 ph10 461 permits them, but they do not mean what you might think. For example,
3070 nigel 73 (?!a){3} does not assert that the next three characters are not "a". It
3071     just asserts that the next character is not "a" three times.
3072 nigel 41
3073 ph10 461 3. Capturing subpatterns that occur inside negative lookahead asser-
3074     tions are counted, but their entries in the offsets vector are never
3075     set. Perl sets its numerical variables from any such patterns that are
3076 nigel 73 matched before the assertion fails to match something (thereby succeed-
3077 ph10 461 ing), but only if the negative lookahead assertion contains just one
3078 nigel 73 branch.
3079 nigel 41
3080 ph10 461 4. Though binary zero characters are supported in the subject string,
3081 nigel 73 they are not allowed in a pattern string because it is passed as a nor-
3082 nigel 75 mal C string, terminated by zero. The escape sequence \0 can be used in
3083     the pattern to represent a binary zero.
3084 nigel 41
3085 ph10 461 5. The following Perl escape sequences are not supported: \l, \u, \L,
3086 nigel 75 \U, and \N. In fact these are implemented by Perl's general string-han-
3087 ph10 461 dling and are not part of its pattern matching engine. If any of these
3088 nigel 75 are encountered by PCRE, an error is generated.
3089 nigel 41
3090 ph10 461 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE
3091     is built with Unicode character property support. The properties that
3092     can be tested with \p and \P are limited to the general category prop-
3093     erties such as Lu and Nd, script names such as Greek or Han, and the
3094     derived properties Any and L&. PCRE does support the Cs (surrogate)
3095     property, which Perl does not; the Perl documentation says "Because
3096 ph10 453 Perl hides the need for the user to understand the internal representa-
3097 ph10 461 tion of Unicode characters, there is no need to implement the somewhat
3098 ph10 453 messy concept of surrogates."
3099 nigel 75
3100     7. PCRE does support the \Q...\E escape for quoting substrings. Charac-
3101 ph10 461 ters in between are treated as literals. This is slightly different
3102     from Perl in that $ and @ are also handled as literals inside the
3103     quotes. In Perl, they cause variable interpolation (but of course PCRE
3104 nigel 73 does not have variables). Note the following examples:
3105 nigel 49
3106 nigel 73 Pattern PCRE matches Perl matches
3107 nigel 41
3108 nigel 73 \Qabc$xyz\E abc$xyz abc followed by the
3109     contents of $xyz
3110     \Qabc\$xyz\E abc\$xyz abc\$xyz
3111     \Qabc\E\$\Qxyz\E abc$xyz abc$xyz
3112 nigel 41
3113 ph10 461 The \Q...\E sequence is recognized both inside and outside character
3114 nigel 73 classes.
3115 nigel 41
3116 nigel 93 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code})
3117 ph10 461 constructions. However, there is support for recursive patterns. This
3118     is not available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE
3119     "callout" feature allows an external function to be called during pat-
3120 nigel 75 tern matching. See the pcrecallout documentation for details.
3121 nigel 63
3122 ph10 461 9. Subpatterns that are called recursively or as "subroutines" are
3123     always treated as atomic groups in PCRE. This is like Python, but
3124     unlike Perl. There is a discussion of an example that explains this in
3125     more detail in the section on recursion differences from Perl in the
3126     pcrepattern page.
3127 nigel 93
3128 ph10 461 10. There are some differences that are concerned with the settings of
3129     captured strings when part of a pattern is repeated. For example,
3130     matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2
3131 nigel 73 unset, but in PCRE it is set to "b".
3132 nigel 41
3133 ph10 535 11. PCRE's handling of duplicate subpattern numbers and duplicate sub-
3134 ph10 461 pattern names is not as general as Perl's. This is a consequence of the
3135     fact the PCRE works internally just with numbers, using an external ta-
3136     ble to translate between numbers and names. In particular, a pattern
3137     such as (?|(?<a>A)|(?<b)B), where the two capturing parentheses have
3138     the same number but different names, is not supported, and causes an
3139     error at compile time. If it were allowed, it would not be possible to
3140     distinguish which parentheses matched, because both names map to cap-
3141     turing subpattern number 1. To avoid this confusing situation, an error
3142     is given at compile time.
3143 nigel 41
3144 ph10 567 12. Perl recognizes comments in some places that PCRE doesn't, for
3145     example, between the ( and ? at the start of a subpattern.
3146    
3147     13. PCRE provides some extensions to the Perl regular expression facil-
3148     ities. Perl 5.10 includes new features that are not in earlier ver-
3149     sions of Perl, some of which (such as named parentheses) have been in
3150 ph10 461 PCRE for some time. This list is with respect to Perl 5.10:
3151 nigel 41
3152 ph10 567 (a) Although lookbehind assertions in PCRE must match fixed length
3153     strings, each alternative branch of a lookbehind assertion can match a
3154     different length of string. Perl requires them all to have the same
3155 ph10 461 length.
3156    
3157 ph10 567 (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $
3158 nigel 73 meta-character matches only at the very end of the string.
3159 nigel 41
3160 nigel 73 (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe-
3161 ph10 182 cial meaning is faulted. Otherwise, like Perl, the backslash is quietly
3162     ignored. (Perl can be made to issue a warning.)
3163 nigel 41
3164 ph10 567 (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti-
3165 nigel 73 fiers is inverted, that is, by default they are not greedy, but if fol-
3166     lowed by a question mark they are.
3167 nigel 41
3168 nigel 75 (e) PCRE_ANCHORED can be used at matching time to force a pattern to be
3169     tried only at the first matching position in the subject string.
3170 nigel 41
3171 ph10 453 (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
3172 ph10 567 and PCRE_NO_AUTO_CAPTURE options for pcre_exec() have no Perl equiva-
3173 ph10 453 lents.
3174 nigel 41
3175 ph10 567 (g) The \R escape sequence can be restricted to match only CR, LF, or
3176 ph10 231 CRLF by the PCRE_BSR_ANYCRLF option.
3177 nigel 41
3178 ph10 231 (h) The callout facility is PCRE-specific.
3179 nigel 43
3180 ph10 231 (i) The partial matching facility is PCRE-specific.
3181    
3182     (j) Patterns compiled by PCRE can be saved and re-used at a later time,
3183 nigel 75 even on different hosts that have the other endianness.
3184    
3185 ph10 567 (k) The alternative matching function (pcre_dfa_exec()) matches in a
3186 nigel 77 different way and is not Perl-compatible.
3187    
3188 ph10 567 (l) PCRE recognizes some special sequences such as (*CR) at the start
3189 ph10 231 of a pattern that set overall options that cannot be changed within the
3190     pattern.
3191 nigel 63
3192 ph10 231
3193 ph10 99 AUTHOR
3194 nigel 63
3195 ph10 99 Philip Hazel
3196     University Computing Service
3197     Cambridge CB2 3QH, England.
3198    
3199    
3200     REVISION
3201    
3202 ph10 567 Last updated: 31 October 2010
3203 ph10 535 Copyright (c) 1997-2010 University of Cambridge.
3204 ph10 99 ------------------------------------------------------------------------------
3205 ph10 567
3206    
3207 nigel 79 PCREPATTERN(3) PCREPATTERN(3)
3208 nigel 63
3209 nigel 79
3210 nigel 73 NAME
3211     PCRE - Perl-compatible regular expressions
3212    
3213 nigel 77
3214 nigel 63 PCRE REGULAR EXPRESSION DETAILS
3215    
3216 ph10 208 The syntax and semantics of the regular expressions that are supported
3217     by PCRE are described in detail below. There is a quick-reference syn-
3218 ph10 345 tax summary in the pcresyntax page. PCRE tries to match Perl syntax and
3219     semantics as closely as it can. PCRE also supports some alternative
3220     regular expression syntax (which does not conflict with the Perl syn-
3221     tax) in order to provide some compatibility with regular expressions in
3222     Python, .NET, and Oniguruma.
3223 nigel 49
3224 ph10 345 Perl's regular expressions are described in its own documentation, and
3225     regular expressions in general are covered in a number of books, some
3226     of which have copious examples. Jeffrey Friedl's "Mastering Regular
3227     Expressions", published by O'Reilly, covers regular expressions in
3228     great detail. This description of PCRE's regular expressions is
3229     intended as reference material.
3230    
3231 nigel 75 The original operation of PCRE was on strings of one-byte characters.
3232     However, there is now also support for UTF-8 character strings. To use
3233 ph10 461 this, PCRE must be built to include UTF-8 support, and you must call
3234     pcre_compile() or pcre_compile2() with the PCRE_UTF8 option. There is
3235     also a special sequence that can be given at the start of a pattern:
3236 nigel 41
3237 ph10 416 (*UTF8)
3238    
3239     Starting a pattern with this sequence is equivalent to setting the
3240     PCRE_UTF8 option. This feature is not Perl-compatible. How setting
3241     UTF-8 mode affects pattern matching is mentioned in several places
3242     below. There is also a summary of UTF-8 features in the section on
3243     UTF-8 support in the main pcre page.
3244    
3245 ph10 535 Another special sequence that may appear at the start of a pattern or
3246     in combination with (*UTF8) is:
3247    
3248     (*UCP)
3249    
3250     This has the same effect as setting the PCRE_UCP option: it causes
3251     sequences such as \d and \w to use Unicode properties to determine
3252     character types, instead of recognizing only characters with codes less
3253     than 128 via a lookup table.
3254    
3255 nigel 77 The remainder of this document discusses the patterns that are sup-
3256     ported by PCRE when its main matching function, pcre_exec(), is used.
3257     From release 6.0, PCRE offers a second matching function,
3258     pcre_dfa_exec(), which matches using a different algorithm that is not
3259 ph10 172 Perl-compatible. Some of the features discussed below are not available
3260     when pcre_dfa_exec() is used. The advantages and disadvantages of the
3261     alternative function, and how it differs from the normal function, are
3262     discussed in the pcrematching page.
3263 nigel 77
3264 nigel 93
3265 ph10 227 NEWLINE CONVENTIONS
3266    
3267     PCRE supports five different conventions for indicating line breaks in
3268     strings: a single CR (carriage return) character, a single LF (line-
3269     feed) character, the two-character sequence CRLF, any of the three pre-
3270     ceding, or any Unicode newline sequence. The pcreapi page has further
3271     discussion about newlines, and shows how to set the newline convention
3272     in the options arguments for the compiling and matching functions.
3273    
3274     It is also possible to specify a newline convention by starting a pat-
3275     tern string with one of the following five sequences:
3276    
3277     (*CR) carriage return
3278     (*LF) linefeed
3279     (*CRLF) carriage return, followed by linefeed
3280     (*ANYCRLF) any of the three above
3281     (*ANY) all Unicode newline sequences
3282    
3283 ph10 461 These override the default and the options given to pcre_compile() or
3284     pcre_compile2(). For example, on a Unix system where LF is the default
3285     newline sequence, the pattern
3286 ph10 227
3287     (*CR)a.b
3288    
3289     changes the convention to CR. That pattern matches "a\nb" because LF is
3290     no longer a newline. Note that these special settings, which are not
3291     Perl-compatible, are recognized only at the very start of a pattern,
3292 ph10 231 and that they must be in upper case. If more than one of them is
3293     present, the last one is used.
3294 ph10 227
3295 ph10 518 The newline convention affects the interpretation of the dot metachar-
3296     acter when PCRE_DOTALL is not set, and also the behaviour of \N. How-
3297     ever, it does not affect what the \R escape sequence matches. By
3298     default, this is any Unicode newline sequence, for Perl compatibility.
3299     However, this can be changed; see the description of \R in the section
3300     entitled "Newline sequences" below. A change of \R setting can be com-
3301     bined with a change of newline convention.
3302 ph10 227
3303 ph10 231
3304 nigel 93 CHARACTERS AND METACHARACTERS
3305    
3306 ph10 247 A regular expression is a pattern that is matched against a subject
3307     string from left to right. Most characters stand for themselves in a
3308     pattern, and match the corresponding characters in the subject. As a
3309 nigel 73 trivial example, the pattern
3310 nigel 41
3311 nigel 73 The quick brown fox
3312 nigel 41
3313 nigel 77 matches a portion of a subject string that is identical to itself. When
3314 ph10 247 caseless matching is specified (the PCRE_CASELESS option), letters are
3315     matched independently of case. In UTF-8 mode, PCRE always understands
3316     the concept of case for characters whose values are less than 128, so
3317     caseless matching is always possible. For characters with higher val-
3318     ues, the concept of case is supported if PCRE is compiled with Unicode
3319     property support, but not otherwise. If you want to use caseless
3320     matching for characters 128 and above, you must ensure that PCRE is
3321 nigel 77 compiled with Unicode property support as well as with UTF-8 support.
3322 nigel 41
3323 ph10 247 The power of regular expressions comes from the ability to include
3324     alternatives and repetitions in the pattern. These are encoded in the
3325 nigel 77 pattern by the use of metacharacters, which do not stand for themselves
3326     but instead are interpreted in some special way.
3327    
3328 ph10 247 There are two different sets of metacharacters: those that are recog-
3329     nized anywhere in the pattern except within square brackets, and those
3330     that are recognized within square brackets. Outside square brackets,
3331 nigel 93 the metacharacters are as follows:
3332 nigel 41
3333 nigel 73 \ general escape character with several uses
3334     ^ assert start of string (or line, in multiline mode)
3335     $ assert end of string (or line, in multiline mode)
3336     . match any character except newline (by default)
3337     [ start character class definition
3338     | start of alternative branch
3339     ( start subpattern
3340     ) end subpattern
3341     ? extends the meaning of (
3342     also 0 or 1 quantifier
3343     also quantifier minimizer
3344     * 0 or more quantifier
3345     + 1 or more quantifier
3346     also "possessive quantifier"
3347     { start min/max quantifier
3348 nigel 41
3349 ph10 247 Part of a pattern that is in square brackets is called a "character
3350 nigel 75 class". In a character class the only metacharacters are:
3351 nigel 41
3352 nigel 73 \ general escape character
3353     ^ negate the class, but only if the first character
3354     - indicates character range
3355     [ POSIX character class (only if followed by POSIX
3356     syntax)
3357     ] terminates the character class<