Parent Directory
|
Revision Log
|
Patch
| revision 429 by ph10, Tue Sep 1 16:10:16 2009 UTC | revision 535 by ph10, Thu Jun 3 19:18:24 2010 UTC | |
|---|---|---|
| # | Line 2 | Line 2 |
| 2 | This file contains a concatenation of the PCRE man pages, converted to plain | This file contains a concatenation of the PCRE man pages, converted to plain |
| 3 | text format for ease of searching with a text editor, or for use on systems | text format for ease of searching with a text editor, or for use on systems |
| 4 | that do not have a man page processor. The small individual files that give | that do not have a man page processor. The small individual files that give |
| 5 | synopses of each function in the library have not been included. Neither has | synopses of each function in the library have not been included. Neither has |
| 6 | the pcredemo program. There are separate text files for the pcregrep and | the pcredemo program. There are separate text files for the pcregrep and |
| 7 | pcretest commands. | pcretest commands. |
| 8 | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| # | Line 19 INTRODUCTION | Line 19 INTRODUCTION |
| 19 | ||
| 20 | The PCRE library is a set of functions that implement regular expres- | The PCRE library is a set of functions that implement regular expres- |
| 21 | sion pattern matching using the same syntax and semantics as Perl, with | sion pattern matching using the same syntax and semantics as Perl, with |
| 22 | just a few differences. Certain features that appeared in Python and | just a few differences. Some features that appeared in Python and PCRE |
| 23 | PCRE before they appeared in Perl are also available using the Python | before they appeared in Perl are also available using the Python syn- |
| 24 | syntax. There is also some support for certain .NET and Oniguruma syn- | tax, there is some support for one or two .NET and Oniguruma syntax |
| 25 | tax items, and there is an option for requesting some minor changes | items, and there is an option for requesting some minor changes that |
| 26 | that give better JavaScript compatibility. | give better JavaScript compatibility. |
| 27 | ||
| 28 | The current implementation of PCRE (release 8.xx) corresponds approxi- | The current implementation of PCRE corresponds approximately with Perl |
| 29 | mately with Perl 5.10, including support for UTF-8 encoded strings and | 5.10/5.11, including support for UTF-8 encoded strings and Unicode gen- |
| 30 | Unicode general category properties. However, UTF-8 and Unicode support | eral category properties. However, UTF-8 and Unicode support has to be |
| 31 | has to be explicitly enabled; it is not the default. The Unicode tables | explicitly enabled; it is not the default. The Unicode tables corre- |
| 32 | correspond to Unicode release 5.1. | spond to Unicode release 5.2.0. |
| 33 | ||
| 34 | In addition to the Perl-compatible matching function, PCRE contains an | In addition to the Perl-compatible matching function, PCRE contains an |
| 35 | alternative matching function that matches the same compiled patterns | alternative function that matches the same compiled patterns in a dif- |
| 36 | in a different way. In certain circumstances, the alternative function | ferent way. In certain circumstances, the alternative function has some |
| 37 | has some advantages. For a discussion of the two matching algorithms, | advantages. For a discussion of the two matching algorithms, see the |
| 38 | see the pcrematching page. | pcrematching page. |
| 39 | ||
| 40 | PCRE is written in C and released as a C library. A number of people | PCRE is written in C and released as a C library. A number of people |
| 41 | have written wrappers and interfaces of various kinds. In particular, | have written wrappers and interfaces of various kinds. In particular, |
| # | Line 55 INTRODUCTION | Line 55 INTRODUCTION |
| 55 | library is built. The pcre_config() function makes it possible for a | library is built. The pcre_config() function makes it possible for a |
| 56 | client to discover which features are available. The features them- | client to discover which features are available. The features them- |
| 57 | selves are described in the pcrebuild page. Documentation about build- | selves are described in the pcrebuild page. Documentation about build- |
| 58 | ing PCRE for various operating systems can be found in the README file | ing PCRE for various operating systems can be found in the README and |
| 59 | in the source distribution. | NON-UNIX-USE files in the source distribution. |
| 60 | ||
| 61 | The library contains a number of undocumented internal functions and | The library contains a number of undocumented internal functions and |
| 62 | data tables that are used by more than one of the exported external | data tables that are used by more than one of the exported external |
| # | Line 89 USER DOCUMENTATION | Line 89 USER DOCUMENTATION |
| 89 | pcrepartial details of the partial matching facility | pcrepartial details of the partial matching facility |
| 90 | pcrepattern syntax and semantics of supported | pcrepattern syntax and semantics of supported |
| 91 | regular expressions | regular expressions |
| pcresyntax quick syntax reference | ||
| 92 | pcreperform discussion of performance issues | pcreperform discussion of performance issues |
| 93 | pcreposix the POSIX-compatible C API | pcreposix the POSIX-compatible C API |
| 94 | pcreprecompile details of saving and re-using precompiled patterns | pcreprecompile details of saving and re-using precompiled patterns |
| 95 | pcresample discussion of the pcredemo program | pcresample discussion of the pcredemo program |
| 96 | pcrestack discussion of stack usage | pcrestack discussion of stack usage |
| 97 | pcresyntax quick syntax reference | |
| 98 | pcretest description of the pcretest testing command | pcretest description of the pcretest testing command |
| 99 | ||
| 100 | In addition, in the "man" and HTML formats, there is a short page for | In addition, in the "man" and HTML formats, there is a short page for |
| # | Line 142 UTF-8 AND UNICODE PROPERTY SUPPORT | Line 142 UTF-8 AND UNICODE PROPERTY SUPPORT |
| 142 | with the PCRE_UTF8 option flag, or the pattern must start with the | with the PCRE_UTF8 option flag, or the pattern must start with the |
| 143 | sequence (*UTF8). When either of these is the case, both the pattern | sequence (*UTF8). When either of these is the case, both the pattern |
| 144 | and any subject strings that are matched against it are treated as | and any subject strings that are matched against it are treated as |
| 145 | UTF-8 strings instead of just strings of bytes. | UTF-8 strings instead of strings of 1-byte characters. |
| 146 | ||
| 147 | If you compile PCRE with UTF-8 support, but do not use it at run time, | If you compile PCRE with UTF-8 support, but do not use it at run time, |
| 148 | the library will be a bit bigger, but the additional run time overhead | the library will be a bit bigger, but the additional run time overhead |
| # | Line 222 UTF-8 AND UNICODE PROPERTY SUPPORT | Line 222 UTF-8 AND UNICODE PROPERTY SUPPORT |
| 222 | not available in the alternative matching function, pcre_dfa_exec(). | not available in the alternative matching function, pcre_dfa_exec(). |
| 223 | ||
| 224 | 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly | 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly |
| 225 | test characters of any code value, but the characters that PCRE recog- | test characters of any code value, but, by default, the characters that |
| 226 | nizes as digits, spaces, or word characters remain the same set as | PCRE recognizes as digits, spaces, or word characters remain the same |
| 227 | before, all with values less than 256. This remains true even when PCRE | set as before, all with values less than 256. This remains true even |
| 228 | includes Unicode property support, because to do otherwise would slow | when PCRE is built to include Unicode property support, because to do |
| 229 | down PCRE in many common cases. If you really want to test for a wider | otherwise would slow down PCRE in many common cases. Note that this |
| 230 | sense of, say, "digit", you must use Unicode property tests such as | also applies to \b, because it is defined in terms of \w and \W. If you |
| 231 | \p{Nd}. Note that this also applies to \b, because it is defined in | really want to test for a wider sense of, say, "digit", you can use |
| 232 | terms of \w and \W. | explicit Unicode property tests such as \p{Nd}. Alternatively, if you |
| 233 | set the PCRE_UCP option, the way that the character escapes work is | |
| 234 | changed so that Unicode properties are used to determine which charac- | |
| 235 | ters match. There are more details in the section on generic character | |
| 236 | types in the pcrepattern documentation. | |
| 237 | ||
| 238 | 7. Similarly, characters that match the POSIX named character classes | 7. Similarly, characters that match the POSIX named character classes |
| 239 | are all low-valued characters. | are all low-valued characters, unless the PCRE_UCP option is set. |
| 240 | ||
| 241 | 8. However, the Perl 5.10 horizontal and vertical whitespace matching | 8. However, the Perl 5.10 horizontal and vertical whitespace matching |
| 242 | escapes (\h, \H, \v, and \V) do match all the appropriate Unicode char- | escapes (\h, \H, \v, and \V) do match all the appropriate Unicode char- |
| 243 | acters. | acters, whether or not PCRE_UCP is set. |
| 244 | ||
| 245 | 9. Case-insensitive matching applies only to characters whose values | 9. Case-insensitive matching applies only to characters whose values |
| 246 | are less than 128, unless PCRE is built with Unicode property support. | are less than 128, unless PCRE is built with Unicode property support. |
| # | Line 263 AUTHOR | Line 267 AUTHOR |
| 267 | ||
| 268 | REVISION | REVISION |
| 269 | ||
| 270 | Last updated: 01 September 2009 | Last updated: 12 May 2010 |
| 271 | Copyright (c) 1997-2009 University of Cambridge. | Copyright (c) 1997-2010 University of Cambridge. |
| 272 | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| 273 | ||
| 274 | ||
| 275 | PCREBUILD(3) PCREBUILD(3) | PCREBUILD(3) PCREBUILD(3) |
| 276 | ||
| 277 | ||
| # | Line 282 PCRE BUILD-TIME OPTIONS | Line 286 PCRE BUILD-TIME OPTIONS |
| 286 | script, where the optional features are selected or deselected by pro- | script, where the optional features are selected or deselected by pro- |
| 287 | viding options to configure before running the make command. However, | viding options to configure before running the make command. However, |
| 288 | the same options can be selected in both Unix-like and non-Unix-like | the same options can be selected in both Unix-like and non-Unix-like |
| 289 | environments using the GUI facility of CMakeSetup if you are using | environments using the GUI facility of cmake-gui if you are using CMake |
| 290 | CMake instead of configure to build PCRE. | instead of configure to build PCRE. |
| 291 | ||
| 292 | There is a lot more information about building PCRE in non-Unix-like | |
| 293 | environments in the file called NON_UNIX_USE, which is part of the PCRE | |
| 294 | distribution. You should consult this file as well as the README file | |
| 295 | if you are building in a non-Unix-like environment. | |
| 296 | ||
| 297 | The complete list of options for configure (which includes the standard | The complete list of options for configure (which includes the standard |
| 298 | ones such as the selection of the installation directory) can be | ones such as the selection of the installation directory) can be |
| 299 | obtained by running | obtained by running |
| 300 | ||
| 301 | ./configure --help | ./configure --help |
| 302 | ||
| 303 | The following sections include descriptions of options whose names | The following sections include descriptions of options whose names |
| 304 | begin with --enable or --disable. These settings specify changes to the | begin with --enable or --disable. These settings specify changes to the |
| 305 | defaults for the configure command. Because of the way that configure | defaults for the configure command. Because of the way that configure |
| 306 | works, --enable and --disable always come in pairs, so the complemen- | works, --enable and --disable always come in pairs, so the complemen- |
| 307 | tary option always exists as well, but as it specifies the default, it | tary option always exists as well, but as it specifies the default, it |
| 308 | is not described. | is not described. |
| 309 | ||
| 310 | ||
| # | Line 316 UTF-8 SUPPORT | Line 325 UTF-8 SUPPORT |
| 325 | ||
| 326 | --enable-utf8 | --enable-utf8 |
| 327 | ||
| 328 | to the configure command. Of itself, this does not make PCRE treat | to the configure command. Of itself, this does not make PCRE treat |
| 329 | strings as UTF-8. As well as compiling PCRE with this option, you also | strings as UTF-8. As well as compiling PCRE with this option, you also |
| 330 | have have to set the PCRE_UTF8 option when you call the pcre_compile() | have have to set the PCRE_UTF8 option when you call the pcre_compile() |
| 331 | function. | or pcre_compile2() functions. |
| 332 | ||
| 333 | If you set --enable-utf8 when compiling in an EBCDIC environment, PCRE | If you set --enable-utf8 when compiling in an EBCDIC environment, PCRE |
| 334 | expects its input to be either ASCII or UTF-8 (depending on the runtime | expects its input to be either ASCII or UTF-8 (depending on the runtime |
| 335 | option). It is not possible to support both EBCDIC and UTF-8 codes in | option). It is not possible to support both EBCDIC and UTF-8 codes in |
| 336 | the same version of the library. Consequently, --enable-utf8 and | the same version of the library. Consequently, --enable-utf8 and |
| 337 | --enable-ebcdic are mutually exclusive. | --enable-ebcdic are mutually exclusive. |
| 338 | ||
| 339 | ||
| 340 | UNICODE CHARACTER PROPERTY SUPPORT | UNICODE CHARACTER PROPERTY SUPPORT |
| 341 | ||
| 342 | UTF-8 support allows PCRE to process character values greater than 255 | UTF-8 support allows PCRE to process character values greater than 255 |
| 343 | in the strings that it handles. On its own, however, it does not pro- | in the strings that it handles. On its own, however, it does not pro- |
| 344 | vide any facilities for accessing the properties of such characters. If | vide any facilities for accessing the properties of such characters. If |
| 345 | you want to be able to use the pattern escapes \P, \p, and \X, which | you want to be able to use the pattern escapes \P, \p, and \X, which |
| 346 | refer to Unicode character properties, you must add | refer to Unicode character properties, you must add |
| 347 | ||
| 348 | --enable-unicode-properties | --enable-unicode-properties |
| 349 | ||
| 350 | to the configure command. This implies UTF-8 support, even if you have | to the configure command. This implies UTF-8 support, even if you have |
| 351 | not explicitly requested it. | not explicitly requested it. |
| 352 | ||
| 353 | Including Unicode property support adds around 30K of tables to the | Including Unicode property support adds around 30K of tables to the |
| 354 | PCRE library. Only the general category properties such as Lu and Nd | PCRE library. Only the general category properties such as Lu and Nd |
| 355 | are supported. Details are given in the pcrepattern documentation. | are supported. Details are given in the pcrepattern documentation. |
| 356 | ||
| 357 | ||
| 358 | CODE VALUE OF NEWLINE | CODE VALUE OF NEWLINE |
| 359 | ||
| 360 | By default, PCRE interprets the linefeed (LF) character as indicating | By default, PCRE interprets the linefeed (LF) character as indicating |
| 361 | the end of a line. This is the normal newline character on Unix-like | the end of a line. This is the normal newline character on Unix-like |
| 362 | systems. You can compile PCRE to use carriage return (CR) instead, by | systems. You can compile PCRE to use carriage return (CR) instead, by |
| 363 | adding | adding |
| 364 | ||
| 365 | --enable-newline-is-cr | --enable-newline-is-cr |
| 366 | ||
| 367 | to the configure command. There is also a --enable-newline-is-lf | to the configure command. There is also a --enable-newline-is-lf |
| 368 | option, which explicitly specifies linefeed as the newline character. | option, which explicitly specifies linefeed as the newline character. |
| 369 | ||
| 370 | Alternatively, you can specify that line endings are to be indicated by | Alternatively, you can specify that line endings are to be indicated by |
| # | Line 367 CODE VALUE OF NEWLINE | Line 376 CODE VALUE OF NEWLINE |
| 376 | ||
| 377 | --enable-newline-is-anycrlf | --enable-newline-is-anycrlf |
| 378 | ||
| 379 | which causes PCRE to recognize any of the three sequences CR, LF, or | which causes PCRE to recognize any of the three sequences CR, LF, or |
| 380 | CRLF as indicating a line ending. Finally, a fifth option, specified by | CRLF as indicating a line ending. Finally, a fifth option, specified by |
| 381 | ||
| 382 | --enable-newline-is-any | --enable-newline-is-any |
| 383 | ||
| 384 | causes PCRE to recognize any Unicode newline sequence. | causes PCRE to recognize any Unicode newline sequence. |
| 385 | ||
| 386 | Whatever line ending convention is selected when PCRE is built can be | Whatever line ending convention is selected when PCRE is built can be |
| 387 | overridden when the library functions are called. At build time it is | overridden when the library functions are called. At build time it is |
| 388 | conventional to use the standard for your operating system. | conventional to use the standard for your operating system. |
| 389 | ||
| 390 | ||
| 391 | WHAT \R MATCHES | WHAT \R MATCHES |
| 392 | ||
| 393 | By default, the sequence \R in a pattern matches any Unicode newline | By default, the sequence \R in a pattern matches any Unicode newline |
| 394 | sequence, whatever has been selected as the line ending sequence. If | sequence, whatever has been selected as the line ending sequence. If |
| 395 | you specify | you specify |
| 396 | ||
| 397 | --enable-bsr-anycrlf | --enable-bsr-anycrlf |
| 398 | ||
| 399 | the default is changed so that \R matches only CR, LF, or CRLF. What- | the default is changed so that \R matches only CR, LF, or CRLF. What- |
| 400 | ever is selected when PCRE is built can be overridden when the library | ever is selected when PCRE is built can be overridden when the library |
| 401 | functions are called. | functions are called. |
| 402 | ||
| 403 | ||
| 404 | BUILDING SHARED AND STATIC LIBRARIES | BUILDING SHARED AND STATIC LIBRARIES |
| 405 | ||
| 406 | The PCRE building process uses libtool to build both shared and static | The PCRE building process uses libtool to build both shared and static |
| 407 | Unix libraries by default. You can suppress one of these by adding one | Unix libraries by default. You can suppress one of these by adding one |
| 408 | of | of |
| 409 | ||
| 410 | --disable-shared | --disable-shared |
| # | Line 407 BUILDING SHARED AND STATIC LIBRARIES | Line 416 BUILDING SHARED AND STATIC LIBRARIES |
| 416 | POSIX MALLOC USAGE | POSIX MALLOC USAGE |
| 417 | ||
| 418 | When PCRE is called through the POSIX interface (see the pcreposix doc- | When PCRE is called through the POSIX interface (see the pcreposix doc- |
| 419 | umentation), additional working storage is required for holding the | umentation), additional working storage is required for holding the |
| 420 | pointers to capturing substrings, because PCRE requires three integers | pointers to capturing substrings, because PCRE requires three integers |
| 421 | per substring, whereas the POSIX interface provides only two. If the | per substring, whereas the POSIX interface provides only two. If the |
| 422 | number of expected substrings is small, the wrapper function uses space | number of expected substrings is small, the wrapper function uses space |
| 423 | on the stack, because this is faster than using malloc() for each call. | on the stack, because this is faster than using malloc() for each call. |
| 424 | The default threshold above which the stack is no longer used is 10; it | The default threshold above which the stack is no longer used is 10; it |
| # | Line 422 POSIX MALLOC USAGE | Line 431 POSIX MALLOC USAGE |
| 431 | ||
| 432 | HANDLING VERY LARGE PATTERNS | HANDLING VERY LARGE PATTERNS |
| 433 | ||
| 434 | Within a compiled pattern, offset values are used to point from one | Within a compiled pattern, offset values are used to point from one |
| 435 | part to another (for example, from an opening parenthesis to an alter- | part to another (for example, from an opening parenthesis to an alter- |
| 436 | nation metacharacter). By default, two-byte values are used for these | nation metacharacter). By default, two-byte values are used for these |
| 437 | offsets, leading to a maximum size for a compiled pattern of around | offsets, leading to a maximum size for a compiled pattern of around |
| 438 | 64K. This is sufficient to handle all but the most gigantic patterns. | 64K. This is sufficient to handle all but the most gigantic patterns. |
| 439 | Nevertheless, some people do want to process enormous patterns, so it | Nevertheless, some people do want to process truyl enormous patterns, |
| 440 | is possible to compile PCRE to use three-byte or four-byte offsets by | so it is possible to compile PCRE to use three-byte or four-byte off- |
| 441 | adding a setting such as | sets by adding a setting such as |
| 442 | ||
| 443 | --with-link-size=3 | --with-link-size=3 |
| 444 | ||
| 445 | to the configure command. The value given must be 2, 3, or 4. Using | to the configure command. The value given must be 2, 3, or 4. Using |
| 446 | longer offsets slows down the operation of PCRE because it has to load | longer offsets slows down the operation of PCRE because it has to load |
| 447 | additional bytes when handling them. | additional bytes when handling them. |
| 448 | ||
| 449 | ||
| 450 | AVOIDING EXCESSIVE STACK USAGE | AVOIDING EXCESSIVE STACK USAGE |
| 451 | ||
| 452 | When matching with the pcre_exec() function, PCRE implements backtrack- | When matching with the pcre_exec() function, PCRE implements backtrack- |
| 453 | ing by making recursive calls to an internal function called match(). | ing by making recursive calls to an internal function called match(). |
| 454 | In environments where the size of the stack is limited, this can se- | In environments where the size of the stack is limited, this can se- |
| 455 | verely limit PCRE's operation. (The Unix environment does not usually | verely limit PCRE's operation. (The Unix environment does not usually |
| 456 | suffer from this problem, but it may sometimes be necessary to increase | suffer from this problem, but it may sometimes be necessary to increase |
| 457 | the maximum stack size. There is a discussion in the pcrestack docu- | the maximum stack size. There is a discussion in the pcrestack docu- |
| 458 | mentation.) An alternative approach to recursion that uses memory from | mentation.) An alternative approach to recursion that uses memory from |
| 459 | the heap to remember data, instead of using recursive function calls, | the heap to remember data, instead of using recursive function calls, |
| 460 | has been implemented to work round the problem of limited stack size. | has been implemented to work round the problem of limited stack size. |
| 461 | If you want to build a version of PCRE that works this way, add | If you want to build a version of PCRE that works this way, add |
| 462 | ||
| 463 | --disable-stack-for-recursion | --disable-stack-for-recursion |
| 464 | ||
| 465 | to the configure command. With this configuration, PCRE will use the | to the configure command. With this configuration, PCRE will use the |
| 466 | pcre_stack_malloc and pcre_stack_free variables to call memory manage- | pcre_stack_malloc and pcre_stack_free variables to call memory manage- |
| 467 | ment functions. By default these point to malloc() and free(), but you | ment functions. By default these point to malloc() and free(), but you |
| 468 | can replace the pointers so that your own functions are used. | can replace the pointers so that your own functions are used instead. |
| 469 | ||
| 470 | Separate functions are provided rather than using pcre_malloc and | Separate functions are provided rather than using pcre_malloc and |
| 471 | pcre_free because the usage is very predictable: the block sizes | pcre_free because the usage is very predictable: the block sizes |
| 472 | requested are always the same, and the blocks are always freed in | requested are always the same, and the blocks are always freed in |
| 473 | reverse order. A calling program might be able to implement optimized | reverse order. A calling program might be able to implement optimized |
| 474 | functions that perform better than malloc() and free(). PCRE runs | functions that perform better than malloc() and free(). PCRE runs |
| 475 | noticeably more slowly when built in this way. This option affects only | noticeably more slowly when built in this way. This option affects only |
| 476 | the pcre_exec() function; it is not relevant for the the | the pcre_exec() function; it is not relevant for pcre_dfa_exec(). |
| pcre_dfa_exec() function. | ||
| 477 | ||
| 478 | ||
| 479 | LIMITING PCRE RESOURCE USAGE | LIMITING PCRE RESOURCE USAGE |
| # | Line 554 PCRETEST OPTION FOR LIBREADLINE SUPPORT | Line 562 PCRETEST OPTION FOR LIBREADLINE SUPPORT |
| 562 | to the configure command, pcretest is linked with the libreadline | to the configure command, pcretest is linked with the libreadline |
| 563 | library, and when its input is from a terminal, it reads it using the | library, and when its input is from a terminal, it reads it using the |
| 564 | readline() function. This provides line-editing and history facilities. | readline() function. This provides line-editing and history facilities. |
| 565 | Note that libreadline is GPL-licenced, so if you distribute a binary of | Note that libreadline is GPL-licensed, so if you distribute a binary of |
| 566 | pcretest linked in this way, there may be licensing issues. | pcretest linked in this way, there may be licensing issues. |
| 567 | ||
| 568 | Setting this option causes the -lreadline option to be added to the | Setting this option causes the -lreadline option to be added to the |
| # | Line 590 AUTHOR | Line 598 AUTHOR |
| 598 | ||
| 599 | REVISION | REVISION |
| 600 | ||
| 601 | Last updated: 17 March 2009 | Last updated: 29 September 2009 |
| 602 | Copyright (c) 1997-2009 University of Cambridge. | Copyright (c) 1997-2009 University of Cambridge. |
| 603 | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| 604 | ||
| 605 | ||
| 606 | PCREMATCHING(3) PCREMATCHING(3) | PCREMATCHING(3) PCREMATCHING(3) |
| 607 | ||
| 608 | ||
| # | Line 678 THE ALTERNATIVE MATCHING ALGORITHM | Line 686 THE ALTERNATIVE MATCHING ALGORITHM |
| 686 | though it is not implemented as a traditional finite state machine (it | though it is not implemented as a traditional finite state machine (it |
| 687 | keeps multiple states active simultaneously). | keeps multiple states active simultaneously). |
| 688 | ||
| 689 | Although the general principle of this matching algorithm is that it | |
| 690 | scans the subject string only once, without backtracking, there is one | |
| 691 | exception: when a lookaround assertion is encountered, the characters | |
| 692 | following or preceding the current point have to be independently | |
| 693 | inspected. | |
| 694 | ||
| 695 | The scan continues until either the end of the subject is reached, or | The scan continues until either the end of the subject is reached, or |
| 696 | there are no more unterminated paths. At this point, terminated paths | there are no more unterminated paths. At this point, terminated paths |
| 697 | represent the different matching possibilities (if there are none, the | represent the different matching possibilities (if there are none, the |
| 698 | match has failed). Thus, if there is more than one possible match, | match has failed). Thus, if there is more than one possible match, |
| 699 | this algorithm finds all of them, and in particular, it finds the long- | this algorithm finds all of them, and in particular, it finds the long- |
| 700 | est. In PCRE, there is an option to stop the algorithm after the first | est. There is an option to stop the algorithm after the first match |
| 701 | match (which is necessarily the shortest) has been found. | (which is necessarily the shortest) is found. |
| 702 | ||
| 703 | Note that all the matches that are found start at the same point in the | Note that all the matches that are found start at the same point in the |
| 704 | subject. If the pattern | subject. If the pattern |
| # | Line 757 ADVANTAGES OF THE ALTERNATIVE ALGORITHM | Line 771 ADVANTAGES OF THE ALTERNATIVE ALGORITHM |
| 771 | 2. Because the alternative algorithm scans the subject string just | 2. Because the alternative algorithm scans the subject string just |
| 772 | once, and never needs to backtrack, it is possible to pass very long | once, and never needs to backtrack, it is possible to pass very long |
| 773 | subject strings to the matching function in several pieces, checking | subject strings to the matching function in several pieces, checking |
| 774 | for partial matching each time. | for partial matching each time. The pcrepartial documentation gives |
| 775 | details of partial matching. | |
| 776 | ||
| 777 | ||
| 778 | DISADVANTAGES OF THE ALTERNATIVE ALGORITHM | DISADVANTAGES OF THE ALTERNATIVE ALGORITHM |
| 779 | ||
| 780 | The alternative algorithm suffers from a number of disadvantages: | The alternative algorithm suffers from a number of disadvantages: |
| 781 | ||
| 782 | 1. It is substantially slower than the standard algorithm. This is | 1. It is substantially slower than the standard algorithm. This is |
| 783 | partly because it has to search for all possible matches, but is also | partly because it has to search for all possible matches, but is also |
| 784 | because it is less susceptible to optimization. | because it is less susceptible to optimization. |
| 785 | ||
| 786 | 2. Capturing parentheses and back references are not supported. | 2. Capturing parentheses and back references are not supported. |
| # | Line 783 AUTHOR | Line 798 AUTHOR |
| 798 | ||
| 799 | REVISION | REVISION |
| 800 | ||
| 801 | Last updated: 25 August 2009 | Last updated: 29 September 2009 |
| 802 | Copyright (c) 1997-2009 University of Cambridge. | Copyright (c) 1997-2009 University of Cambridge. |
| 803 | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| 804 | ||
| 805 | ||
| 806 | PCREAPI(3) PCREAPI(3) | PCREAPI(3) PCREAPI(3) |
| 807 | ||
| 808 | ||
| # | Line 891 PCRE API OVERVIEW | Line 906 PCRE API OVERVIEW |
| 906 | bers for the library. Applications can use these to include support | bers for the library. Applications can use these to include support |
| 907 | for different releases of PCRE. | for different releases of PCRE. |
| 908 | ||
| 909 | In a Windows environment, if you want to statically link an application | |
| 910 | program against a non-dll pcre.a file, you must define PCRE_STATIC | |
| 911 | before including pcre.h or pcrecpp.h, because otherwise the pcre_mal- | |
| 912 | loc() and pcre_free() exported functions will be declared | |
| 913 | __declspec(dllimport), with unwanted results. | |
| 914 | ||
| 915 | The functions pcre_compile(), pcre_compile2(), pcre_study(), and | The functions pcre_compile(), pcre_compile2(), pcre_study(), and |
| 916 | pcre_exec() are used for compiling and matching regular expressions in | pcre_exec() are used for compiling and matching regular expressions in |
| 917 | a Perl-compatible manner. A sample program that demonstrates the sim- | a Perl-compatible manner. A sample program that demonstrates the sim- |
| # | Line 902 PCRE API OVERVIEW | Line 923 PCRE API OVERVIEW |
| 923 | A second matching function, pcre_dfa_exec(), which is not Perl-compati- | A second matching function, pcre_dfa_exec(), which is not Perl-compati- |
| 924 | ble, is also provided. This uses a different algorithm for the match- | ble, is also provided. This uses a different algorithm for the match- |
| 925 | ing. The alternative algorithm finds all possible matches (at a given | ing. The alternative algorithm finds all possible matches (at a given |
| 926 | point in the subject), and scans the subject just once. However, this | point in the subject), and scans the subject just once (unless there |
| 927 | algorithm does not return captured substrings. A description of the two | are lookbehind assertions). However, this algorithm does not return |
| 928 | matching algorithms and their advantages and disadvantages is given in | captured substrings. A description of the two matching algorithms and |
| 929 | the pcrematching documentation. | their advantages and disadvantages is given in the pcrematching docu- |
| 930 | mentation. | |
| 931 | ||
| 932 | In addition to the main compiling and matching functions, there are | In addition to the main compiling and matching functions, there are |
| 933 | convenience functions for extracting captured substrings from a subject | convenience functions for extracting captured substrings from a subject |
| 934 | string that is matched by pcre_exec(). They are: | string that is matched by pcre_exec(). They are: |
| 935 | ||
| # | Line 922 PCRE API OVERVIEW | Line 944 PCRE API OVERVIEW |
| 944 | pcre_free_substring() and pcre_free_substring_list() are also provided, | pcre_free_substring() and pcre_free_substring_list() are also provided, |
| 945 | to free the memory used for extracted strings. | to free the memory used for extracted strings. |
| 946 | ||
| 947 | The function pcre_maketables() is used to build a set of character | The function pcre_maketables() is used to build a set of character |
| 948 | tables in the current locale for passing to pcre_compile(), | tables in the current locale for passing to pcre_compile(), |
| 949 | pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is | pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is |
| 950 | provided for specialist use. Most commonly, no special tables are | provided for specialist use. Most commonly, no special tables are |
| 951 | passed, in which case internal tables that are generated when PCRE is | passed, in which case internal tables that are generated when PCRE is |
| 952 | built are used. | built are used. |
| 953 | ||
| 954 | The function pcre_fullinfo() is used to find out information about a | The function pcre_fullinfo() is used to find out information about a |
| 955 | compiled pattern; pcre_info() is an obsolete version that returns only | compiled pattern; pcre_info() is an obsolete version that returns only |
| 956 | some of the available information, but is retained for backwards com- | some of the available information, but is retained for backwards com- |
| 957 | patibility. The function pcre_version() returns a pointer to a string | patibility. The function pcre_version() returns a pointer to a string |
| 958 | containing the version of PCRE and its date of release. | containing the version of PCRE and its date of release. |
| 959 | ||
| 960 | The function pcre_refcount() maintains a reference count in a data | The function pcre_refcount() maintains a reference count in a data |
| 961 | block containing a compiled pattern. This is provided for the benefit | block containing a compiled pattern. This is provided for the benefit |
| 962 | of object-oriented applications. | of object-oriented applications. |
| 963 | ||
| 964 | The global variables pcre_malloc and pcre_free initially contain the | The global variables pcre_malloc and pcre_free initially contain the |
| 965 | entry points of the standard malloc() and free() functions, respec- | entry points of the standard malloc() and free() functions, respec- |
| 966 | tively. PCRE calls the memory management functions via these variables, | tively. PCRE calls the memory management functions via these variables, |
| 967 | so a calling program can replace them if it wishes to intercept the | so a calling program can replace them if it wishes to intercept the |
| 968 | calls. This should be done before calling any PCRE functions. | calls. This should be done before calling any PCRE functions. |
| 969 | ||
| 970 | The global variables pcre_stack_malloc and pcre_stack_free are also | The global variables pcre_stack_malloc and pcre_stack_free are also |
| 971 | indirections to memory management functions. These special functions | indirections to memory management functions. These special functions |
| 972 | are used only when PCRE is compiled to use the heap for remembering | are used only when PCRE is compiled to use the heap for remembering |
| 973 | data, instead of recursive function calls, when running the pcre_exec() | data, instead of recursive function calls, when running the pcre_exec() |
| 974 | function. See the pcrebuild documentation for details of how to do | function. See the pcrebuild documentation for details of how to do |
| 975 | this. It is a non-standard way of building PCRE, for use in environ- | this. It is a non-standard way of building PCRE, for use in environ- |
| 976 | ments that have limited stacks. Because of the greater use of memory | ments that have limited stacks. Because of the greater use of memory |
| 977 | management, it runs more slowly. Separate functions are provided so | management, it runs more slowly. Separate functions are provided so |
| 978 | that special-purpose external code can be used for this case. When | that special-purpose external code can be used for this case. When |
| 979 | used, these functions are always called in a stack-like manner (last | used, these functions are always called in a stack-like manner (last |
| 980 | obtained, first freed), and always for memory blocks of the same size. | obtained, first freed), and always for memory blocks of the same size. |
| 981 | There is a discussion about PCRE's stack usage in the pcrestack docu- | There is a discussion about PCRE's stack usage in the pcrestack docu- |
| 982 | mentation. | mentation. |
| 983 | ||
| 984 | The global variable pcre_callout initially contains NULL. It can be set | The global variable pcre_callout initially contains NULL. It can be set |
| 985 | by the caller to a "callout" function, which PCRE will then call at | by the caller to a "callout" function, which PCRE will then call at |
| 986 | specified points during a matching operation. Details are given in the | specified points during a matching operation. Details are given in the |
| 987 | pcrecallout documentation. | pcrecallout documentation. |
| 988 | ||
| 989 | ||
| 990 | NEWLINES | NEWLINES |
| 991 | ||
| 992 | PCRE supports five different conventions for indicating line breaks in | PCRE supports five different conventions for indicating line breaks in |
| 993 | strings: a single CR (carriage return) character, a single LF (line- | strings: a single CR (carriage return) character, a single LF (line- |
| 994 | feed) character, the two-character sequence CRLF, any of the three pre- | feed) character, the two-character sequence CRLF, any of the three pre- |
| 995 | ceding, or any Unicode newline sequence. The Unicode newline sequences | ceding, or any Unicode newline sequence. The Unicode newline sequences |
| 996 | are the three just mentioned, plus the single characters VT (vertical | are the three just mentioned, plus the single characters VT (vertical |
| 997 | tab, U+000B), FF (formfeed, U+000C), NEL (next line, U+0085), LS (line | tab, U+000B), FF (formfeed, U+000C), NEL (next line, U+0085), LS (line |
| 998 | separator, U+2028), and PS (paragraph separator, U+2029). | separator, U+2028), and PS (paragraph separator, U+2029). |
| 999 | ||
| 1000 | Each of the first three conventions is used by at least one operating | Each of the first three conventions is used by at least one operating |
| 1001 | system as its standard newline sequence. When PCRE is built, a default | system as its standard newline sequence. When PCRE is built, a default |
| 1002 | can be specified. The default default is LF, which is the Unix stan- | can be specified. The default default is LF, which is the Unix stan- |
| 1003 | dard. When PCRE is run, the default can be overridden, either when a | dard. When PCRE is run, the default can be overridden, either when a |
| 1004 | pattern is compiled, or when it is matched. | pattern is compiled, or when it is matched. |
| 1005 | ||
| 1006 | At compile time, the newline convention can be specified by the options | At compile time, the newline convention can be specified by the options |
| 1007 | argument of pcre_compile(), or it can be specified by special text at | argument of pcre_compile(), or it can be specified by special text at |
| 1008 | the start of the pattern itself; this overrides any other settings. See | the start of the pattern itself; this overrides any other settings. See |
| 1009 | the pcrepattern page for details of the special character sequences. | the pcrepattern page for details of the special character sequences. |
| 1010 | ||
| 1011 | In the PCRE documentation the word "newline" is used to mean "the char- | In the PCRE documentation the word "newline" is used to mean "the char- |
| 1012 | acter or pair of characters that indicate a line break". The choice of | acter or pair of characters that indicate a line break". The choice of |
| 1013 | newline convention affects the handling of the dot, circumflex, and | newline convention affects the handling of the dot, circumflex, and |
| 1014 | dollar metacharacters, the handling of #-comments in /x mode, and, when | dollar metacharacters, the handling of #-comments in /x mode, and, when |
| 1015 | CRLF is a recognized line ending sequence, the match position advance- | CRLF is a recognized line ending sequence, the match position advance- |
| 1016 | ment for a non-anchored pattern. There is more detail about this in the | ment for a non-anchored pattern. There is more detail about this in the |
| 1017 | section on pcre_exec() options below. | section on pcre_exec() options below. |
| 1018 | ||
| 1019 | The choice of newline convention does not affect the interpretation of | The choice of newline convention does not affect the interpretation of |
| 1020 | the \n or \r escape sequences, nor does it affect what \R matches, | the \n or \r escape sequences, nor does it affect what \R matches, |
| 1021 | which is controlled in a similar way, but by separate options. | which is controlled in a similar way, but by separate options. |
| 1022 | ||
| 1023 | ||
| 1024 | MULTITHREADING | MULTITHREADING |
| 1025 | ||
| 1026 | The PCRE functions can be used in multi-threading applications, with | The PCRE functions can be used in multi-threading applications, with |
| 1027 | the proviso that the memory management functions pointed to by | the proviso that the memory management functions pointed to by |
| 1028 | pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the | pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the |
| 1029 | callout function pointed to by pcre_callout, are shared by all threads. | callout function pointed to by pcre_callout, are shared by all threads. |
| 1030 | ||
| 1031 | The compiled form of a regular expression is not altered during match- | The compiled form of a regular expression is not altered during match- |
| 1032 | ing, so the same compiled pattern can safely be used by several threads | ing, so the same compiled pattern can safely be used by several threads |
| 1033 | at once. | at once. |
| 1034 | ||
| # | Line 1014 MULTITHREADING | Line 1036 MULTITHREADING |
| 1036 | SAVING PRECOMPILED PATTERNS FOR LATER USE | SAVING PRECOMPILED PATTERNS FOR LATER USE |
| 1037 | ||
| 1038 | The compiled form of a regular expression can be saved and re-used at a | The compiled form of a regular expression can be saved and re-used at a |
| 1039 | later time, possibly by a different program, and even on a host other | later time, possibly by a different program, and even on a host other |
| 1040 | than the one on which it was compiled. Details are given in the | than the one on which it was compiled. Details are given in the |
| 1041 | pcreprecompile documentation. However, compiling a regular expression | pcreprecompile documentation. However, compiling a regular expression |
| 1042 | with one version of PCRE for use with a different version is not guar- | with one version of PCRE for use with a different version is not guar- |
| 1043 | anteed to work and may cause crashes. | anteed to work and may cause crashes. |
| 1044 | ||
| 1045 | ||
| # | Line 1025 CHECKING BUILD-TIME OPTIONS | Line 1047 CHECKING BUILD-TIME OPTIONS |
| 1047 | ||
| 1048 | int pcre_config(int what, void *where); | int pcre_config(int what, void *where); |
| 1049 | ||
| 1050 | The function pcre_config() makes it possible for a PCRE client to dis- | The function pcre_config() makes it possible for a PCRE client to dis- |
| 1051 | cover which optional features have been compiled into the PCRE library. | cover which optional features have been compiled into the PCRE library. |
| 1052 | The pcrebuild documentation has more details about these optional fea- | The pcrebuild documentation has more details about these optional fea- |
| 1053 | tures. | tures. |
| 1054 | ||
| 1055 | The first argument for pcre_config() is an integer, specifying which | The first argument for pcre_config() is an integer, specifying which |
| 1056 | information is required; the second argument is a pointer to a variable | information is required; the second argument is a pointer to a variable |
| 1057 | into which the information is placed. The following information is | into which the information is placed. The following information is |
| 1058 | available: | available: |
| 1059 | ||
| 1060 | PCRE_CONFIG_UTF8 | PCRE_CONFIG_UTF8 |
| 1061 | ||
| 1062 | The output is an integer that is set to one if UTF-8 support is avail- | The output is an integer that is set to one if UTF-8 support is avail- |
| 1063 | able; otherwise it is set to zero. | able; otherwise it is set to zero. |
| 1064 | ||
| 1065 | PCRE_CONFIG_UNICODE_PROPERTIES | PCRE_CONFIG_UNICODE_PROPERTIES |
| 1066 | ||
| 1067 | The output is an integer that is set to one if support for Unicode | The output is an integer that is set to one if support for Unicode |
| 1068 | character properties is available; otherwise it is set to zero. | character properties is available; otherwise it is set to zero. |
| 1069 | ||
| 1070 | PCRE_CONFIG_NEWLINE | PCRE_CONFIG_NEWLINE |
| 1071 | ||
| 1072 | The output is an integer whose value specifies the default character | The output is an integer whose value specifies the default character |
| 1073 | sequence that is recognized as meaning "newline". The four values that | sequence that is recognized as meaning "newline". The four values that |
| 1074 | are supported are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF, | are supported are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF, |
| 1075 | and -1 for ANY. Though they are derived from ASCII, the same values | and -1 for ANY. Though they are derived from ASCII, the same values |
| 1076 | are returned in EBCDIC environments. The default should normally corre- | are returned in EBCDIC environments. The default should normally corre- |
| 1077 | spond to the standard sequence for your operating system. | spond to the standard sequence for your operating system. |
| 1078 | ||
| 1079 | PCRE_CONFIG_BSR | PCRE_CONFIG_BSR |
| 1080 | ||
| 1081 | The output is an integer whose value indicates what character sequences | The output is an integer whose value indicates what character sequences |
| 1082 | the \R escape sequence matches by default. A value of 0 means that \R | the \R escape sequence matches by default. A value of 0 means that \R |
| 1083 | matches any Unicode line ending sequence; a value of 1 means that \R | matches any Unicode line ending sequence; a value of 1 means that \R |
| 1084 | matches only CR, LF, or CRLF. The default can be overridden when a pat- | matches only CR, LF, or CRLF. The default can be overridden when a pat- |
| 1085 | tern is compiled or matched. | tern is compiled or matched. |
| 1086 | ||
| 1087 | PCRE_CONFIG_LINK_SIZE | PCRE_CONFIG_LINK_SIZE |
| 1088 | ||
| 1089 | The output is an integer that contains the number of bytes used for | The output is an integer that contains the number of bytes used for |
| 1090 | internal linkage in compiled regular expressions. The value is 2, 3, or | internal linkage in compiled regular expressions. The value is 2, 3, or |
| 1091 | 4. Larger values allow larger regular expressions to be compiled, at | 4. Larger values allow larger regular expressions to be compiled, at |
| 1092 | the expense of slower matching. The default value of 2 is sufficient | the expense of slower matching. The default value of 2 is sufficient |
| 1093 | for all but the most massive patterns, since it allows the compiled | for all but the most massive patterns, since it allows the compiled |
| 1094 | pattern to be up to 64K in size. | pattern to be up to 64K in size. |
| 1095 | ||
| 1096 | PCRE_CONFIG_POSIX_MALLOC_THRESHOLD | PCRE_CONFIG_POSIX_MALLOC_THRESHOLD |
| 1097 | ||
| 1098 | The output is an integer that contains the threshold above which the | The output is an integer that contains the threshold above which the |
| 1099 | POSIX interface uses malloc() for output vectors. Further details are | POSIX interface uses malloc() for output vectors. Further details are |
| 1100 | given in the pcreposix documentation. | given in the pcreposix documentation. |
| 1101 | ||
| 1102 | PCRE_CONFIG_MATCH_LIMIT | PCRE_CONFIG_MATCH_LIMIT |
| 1103 | ||
| 1104 | The output is a long integer that gives the default limit for the num- | The output is a long integer that gives the default limit for the num- |
| 1105 | ber of internal matching function calls in a pcre_exec() execution. | ber of internal matching function calls in a pcre_exec() execution. |
| 1106 | Further details are given with pcre_exec() below. | Further details are given with pcre_exec() below. |
| 1107 | ||
| 1108 | PCRE_CONFIG_MATCH_LIMIT_RECURSION | PCRE_CONFIG_MATCH_LIMIT_RECURSION |
| 1109 | ||
| 1110 | The output is a long integer that gives the default limit for the depth | The output is a long integer that gives the default limit for the depth |
| 1111 | of recursion when calling the internal matching function in a | of recursion when calling the internal matching function in a |
| 1112 | pcre_exec() execution. Further details are given with pcre_exec() | pcre_exec() execution. Further details are given with pcre_exec() |
| 1113 | below. | below. |
| 1114 | ||
| 1115 | PCRE_CONFIG_STACKRECURSE | PCRE_CONFIG_STACKRECURSE |
| 1116 | ||
| 1117 | The output is an integer that is set to one if internal recursion when | The output is an integer that is set to one if internal recursion when |
| 1118 | running pcre_exec() is implemented by recursive function calls that use | running pcre_exec() is implemented by recursive function calls that use |
| 1119 | the stack to remember their state. This is the usual way that PCRE is | the stack to remember their state. This is the usual way that PCRE is |
| 1120 | compiled. The output is zero if PCRE was compiled to use blocks of data | compiled. The output is zero if PCRE was compiled to use blocks of data |
| 1121 | on the heap instead of recursive function calls. In this case, | on the heap instead of recursive function calls. In this case, |
| 1122 | pcre_stack_malloc and pcre_stack_free are called to manage memory | pcre_stack_malloc and pcre_stack_free are called to manage memory |
| 1123 | blocks on the heap, thus avoiding the use of the stack. | blocks on the heap, thus avoiding the use of the stack. |
| 1124 | ||
| 1125 | ||
| # | Line 1114 COMPILING A PATTERN | Line 1136 COMPILING A PATTERN |
| 1136 | ||
| 1137 | Either of the functions pcre_compile() or pcre_compile2() can be called | Either of the functions pcre_compile() or pcre_compile2() can be called |
| 1138 | to compile a pattern into an internal form. The only difference between | to compile a pattern into an internal form. The only difference between |
| 1139 | the two interfaces is that pcre_compile2() has an additional argument, | the two interfaces is that pcre_compile2() has an additional argument, |
| 1140 | errorcodeptr, via which a numerical error code can be returned. | errorcodeptr, via which a numerical error code can be returned. To |
| 1141 | avoid too much repetition, we refer just to pcre_compile() below, but | |
| 1142 | the information applies equally to pcre_compile2(). | |
| 1143 | ||
| 1144 | The pattern is a C string terminated by a binary zero, and is passed in | The pattern is a C string terminated by a binary zero, and is passed in |
| 1145 | the pattern argument. A pointer to a single block of memory that is | the pattern argument. A pointer to a single block of memory that is |
| 1146 | obtained via pcre_malloc is returned. This contains the compiled code | obtained via pcre_malloc is returned. This contains the compiled code |
| 1147 | and related data. The pcre type is defined for the returned block; this | and related data. The pcre type is defined for the returned block; this |
| 1148 | is a typedef for a structure whose contents are not externally defined. | is a typedef for a structure whose contents are not externally defined. |
| 1149 | It is up to the caller to free the memory (via pcre_free) when it is no | It is up to the caller to free the memory (via pcre_free) when it is no |
| 1150 | longer required. | longer required. |
| 1151 | ||
| 1152 | Although the compiled code of a PCRE regex is relocatable, that is, it | Although the compiled code of a PCRE regex is relocatable, that is, it |
| 1153 | does not depend on memory location, the complete pcre data block is not | does not depend on memory location, the complete pcre data block is not |
| 1154 | fully relocatable, because it may contain a copy of the tableptr argu- | fully relocatable, because it may contain a copy of the tableptr argu- |
| 1155 | ment, which is an address (see below). | ment, which is an address (see below). |
| 1156 | ||
| 1157 | The options argument contains various bit settings that affect the com- | The options argument contains various bit settings that affect the com- |
| 1158 | pilation. It should be zero if no options are required. The available | pilation. It should be zero if no options are required. The available |
| 1159 | options are described below. Some of them (in particular, those that | options are described below. Some of them (in particular, those that |
| 1160 | are compatible with Perl, but also some others) can also be set and | are compatible with Perl, but some others as well) can also be set and |
| 1161 | unset from within the pattern (see the detailed description in the | unset from within the pattern (see the detailed description in the |
| 1162 | pcrepattern documentation). For those options that can be different in | pcrepattern documentation). For those options that can be different in |
| 1163 | different parts of the pattern, the contents of the options argument | different parts of the pattern, the contents of the options argument |
| 1164 | specifies their initial settings at the start of compilation and execu- | specifies their settings at the start of compilation and execution. The |
| 1165 | tion. The PCRE_ANCHORED and PCRE_NEWLINE_xxx options can be set at the | PCRE_ANCHORED, PCRE_BSR_xxx, and PCRE_NEWLINE_xxx options can be set at |
| 1166 | time of matching as well as at compile time. | the time of matching as well as at compile time. |
| 1167 | ||
| 1168 | If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise, | If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise, |
| 1169 | if compilation of a pattern fails, pcre_compile() returns NULL, and | if compilation of a pattern fails, pcre_compile() returns NULL, and |
| 1170 | sets the variable pointed to by errptr to point to a textual error mes- | sets the variable pointed to by errptr to point to a textual error mes- |
| 1171 | sage. This is a static string that is part of the library. You must not | sage. This is a static string that is part of the library. You must not |
| 1172 | try to free it. The offset from the start of the pattern to the charac- | try to free it. The byte offset from the start of the pattern to the |
| 1173 | ter where the error was discovered is placed in the variable pointed to | character that was being processed when the error was discovered is |
| 1174 | by erroffset, which must not be NULL. If it is, an immediate error is | placed in the variable pointed to by erroffset, which must not be NULL. |
| 1175 | given. | If it is, an immediate error is given. Some errors are not detected |
| 1176 | until checks are carried out when the whole pattern has been scanned; | |
| 1177 | If pcre_compile2() is used instead of pcre_compile(), and the error- | in this case the offset is set to the end of the pattern. |
| 1178 | codeptr argument is not NULL, a non-zero error code number is returned | |
| 1179 | via this argument in the event of an error. This is in addition to the | If pcre_compile2() is used instead of pcre_compile(), and the error- |
| 1180 | codeptr argument is not NULL, a non-zero error code number is returned | |
| 1181 | via this argument in the event of an error. This is in addition to the | |
| 1182 | textual error message. Error codes and messages are listed below. | textual error message. Error codes and messages are listed below. |
| 1183 | ||
| 1184 | If the final argument, tableptr, is NULL, PCRE uses a default set of | If the final argument, tableptr, is NULL, PCRE uses a default set of |
| 1185 | character tables that are built when PCRE is compiled, using the | character tables that are built when PCRE is compiled, using the |
| 1186 | default C locale. Otherwise, tableptr must be an address that is the | default C locale. Otherwise, tableptr must be an address that is the |
| 1187 | result of a call to pcre_maketables(). This value is stored with the | result of a call to pcre_maketables(). This value is stored with the |
| 1188 | compiled pattern, and used again by pcre_exec(), unless another table | compiled pattern, and used again by pcre_exec(), unless another table |
| 1189 | pointer is passed to it. For more discussion, see the section on locale | pointer is passed to it. For more discussion, see the section on locale |
| 1190 | support below. | support below. |
| 1191 | ||
| 1192 | This code fragment shows a typical straightforward call to pcre_com- | This code fragment shows a typical straightforward call to pcre_com- |
| 1193 | pile(): | pile(): |
| 1194 | ||
| 1195 | pcre *re; | pcre *re; |
| # | Line 1176 COMPILING A PATTERN | Line 1202 COMPILING A PATTERN |
| 1202 | &erroffset, /* for error offset */ | &erroffset, /* for error offset */ |
| 1203 | NULL); /* use default character tables */ | NULL); /* use default character tables */ |
| 1204 | ||
| 1205 | The following names for option bits are defined in the pcre.h header | The following names for option bits are defined in the pcre.h header |
| 1206 | file: | file: |
| 1207 | ||
| 1208 | PCRE_ANCHORED | PCRE_ANCHORED |
| 1209 | ||
| 1210 | If this bit is set, the pattern is forced to be "anchored", that is, it | If this bit is set, the pattern is forced to be "anchored", that is, it |
| 1211 | is constrained to match only at the first matching point in the string | is constrained to match only at the first matching point in the string |
| 1212 | that is being searched (the "subject string"). This effect can also be | that is being searched (the "subject string"). This effect can also be |
| 1213 | achieved by appropriate constructs in the pattern itself, which is the | achieved by appropriate constructs in the pattern itself, which is the |
| 1214 | only way to do it in Perl. | only way to do it in Perl. |
| 1215 | ||
| 1216 | PCRE_AUTO_CALLOUT | PCRE_AUTO_CALLOUT |
| 1217 | ||
| 1218 | If this bit is set, pcre_compile() automatically inserts callout items, | If this bit is set, pcre_compile() automatically inserts callout items, |
| 1219 | all with number 255, before each pattern item. For discussion of the | all with number 255, before each pattern item. For discussion of the |
| 1220 | callout facility, see the pcrecallout documentation. | callout facility, see the pcrecallout documentation. |
| 1221 | ||
| 1222 | PCRE_BSR_ANYCRLF | PCRE_BSR_ANYCRLF |
| 1223 | PCRE_BSR_UNICODE | PCRE_BSR_UNICODE |
| 1224 | ||
| 1225 | These options (which are mutually exclusive) control what the \R escape | These options (which are mutually exclusive) control what the \R escape |
| 1226 | sequence matches. The choice is either to match only CR, LF, or CRLF, | sequence matches. The choice is either to match only CR, LF, or CRLF, |
| 1227 | or to match any Unicode newline sequence. The default is specified when | or to match any Unicode newline sequence. The default is specified when |
| 1228 | PCRE is built. It can be overridden from within the pattern, or by set- | PCRE is built. It can be overridden from within the pattern, or by set- |
| 1229 | ting an option when a compiled pattern is matched. | ting an option when a compiled pattern is matched. |
| 1230 | ||
| 1231 | PCRE_CASELESS | PCRE_CASELESS |
| 1232 | ||
| 1233 | If this bit is set, letters in the pattern match both upper and lower | If this bit is set, letters in the pattern match both upper and lower |
| 1234 | case letters. It is equivalent to Perl's /i option, and it can be | case letters. It is equivalent to Perl's /i option, and it can be |
| 1235 | changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE | changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE |
| 1236 | always understands the concept of case for characters whose values are | always understands the concept of case for characters whose values are |
| 1237 | less than 128, so caseless matching is always possible. For characters | less than 128, so caseless matching is always possible. For characters |
| 1238 | with higher values, the concept of case is supported if PCRE is com- | with higher values, the concept of case is supported if PCRE is com- |
| 1239 | piled with Unicode property support, but not otherwise. If you want to | piled with Unicode property support, but not otherwise. If you want to |
| 1240 | use caseless matching for characters 128 and above, you must ensure | use caseless matching for characters 128 and above, you must ensure |
| 1241 | that PCRE is compiled with Unicode property support as well as with | that PCRE is compiled with Unicode property support as well as with |
| 1242 | UTF-8 support. | UTF-8 support. |
| 1243 | ||
| 1244 | PCRE_DOLLAR_ENDONLY | PCRE_DOLLAR_ENDONLY |
| 1245 | ||
| 1246 | If this bit is set, a dollar metacharacter in the pattern matches only | If this bit is set, a dollar metacharacter in the pattern matches only |
| 1247 | at the end of the subject string. Without this option, a dollar also | at the end of the subject string. Without this option, a dollar also |
| 1248 | matches immediately before a newline at the end of the string (but not | matches immediately before a newline at the end of the string (but not |
| 1249 | before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored | before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored |
| 1250 | if PCRE_MULTILINE is set. There is no equivalent to this option in | if PCRE_MULTILINE is set. There is no equivalent to this option in |
| 1251 | Perl, and no way to set it within a pattern. | Perl, and no way to set it within a pattern. |
| 1252 | ||
| 1253 | PCRE_DOTALL | PCRE_DOTALL |
| 1254 | ||
| 1255 | If this bit is set, a dot metacharater in the pattern matches all char- | If this bit is set, a dot metacharater in the pattern matches all char- |
| 1256 | acters, including those that indicate newline. Without it, a dot does | acters, including those that indicate newline. Without it, a dot does |
| 1257 | not match when the current position is at a newline. This option is | not match when the current position is at a newline. This option is |
| 1258 | equivalent to Perl's /s option, and it can be changed within a pattern | equivalent to Perl's /s option, and it can be changed within a pattern |
| 1259 | by a (?s) option setting. A negative class such as [^a] always matches | by a (?s) option setting. A negative class such as [^a] always matches |
| 1260 | newline characters, independent of the setting of this option. | newline characters, independent of the setting of this option. |
| 1261 | ||
| 1262 | PCRE_DUPNAMES | PCRE_DUPNAMES |
| 1263 | ||
| 1264 | If this bit is set, names used to identify capturing subpatterns need | If this bit is set, names used to identify capturing subpatterns need |
| 1265 | not be unique. This can be helpful for certain types of pattern when it | not be unique. This can be helpful for certain types of pattern when it |
| 1266 | is known that only one instance of the named subpattern can ever be | is known that only one instance of the named subpattern can ever be |
| 1267 | matched. There are more details of named subpatterns below; see also | matched. There are more details of named subpatterns below; see also |
| 1268 | the pcrepattern documentation. | the pcrepattern documentation. |
| 1269 | ||
| 1270 | PCRE_EXTENDED | PCRE_EXTENDED |
| 1271 | ||
| 1272 | If this bit is set, whitespace data characters in the pattern are | If this bit is set, whitespace data characters in the pattern are |
| 1273 | totally ignored except when escaped or inside a character class. White- | totally ignored except when escaped or inside a character class. White- |
| 1274 | space does not include the VT character (code 11). In addition, charac- | space does not include the VT character (code 11). In addition, charac- |
| 1275 | ters between an unescaped # outside a character class and the next new- | ters between an unescaped # outside a character class and the next new- |
| 1276 | line, inclusive, are also ignored. This is equivalent to Perl's /x | line, inclusive, are also ignored. This is equivalent to Perl's /x |
| 1277 | option, and it can be changed within a pattern by a (?x) option set- | option, and it can be changed within a pattern by a (?x) option set- |
| 1278 | ting. | ting. |
| 1279 | ||
| 1280 | This option makes it possible to include comments inside complicated | This option makes it possible to include comments inside complicated |
| 1281 | patterns. Note, however, that this applies only to data characters. | patterns. Note, however, that this applies only to data characters. |
| 1282 | Whitespace characters may never appear within special character | Whitespace characters may never appear within special character |
| 1283 | sequences in a pattern, for example within the sequence (?( which | sequences in a pattern, for example within the sequence (?( which |
| 1284 | introduces a conditional subpattern. | introduces a conditional subpattern. |
| 1285 | ||
| 1286 | PCRE_EXTRA | PCRE_EXTRA |
| 1287 | ||
| 1288 | This option was invented in order to turn on additional functionality | This option was invented in order to turn on additional functionality |
| 1289 | of PCRE that is incompatible with Perl, but it is currently of very | of PCRE that is incompatible with Perl, but it is currently of very |
| 1290 | little use. When set, any backslash in a pattern that is followed by a | little use. When set, any backslash in a pattern that is followed by a |
| 1291 | letter that has no special meaning causes an error, thus reserving | letter that has no special meaning causes an error, thus reserving |
| 1292 | these combinations for future expansion. By default, as in Perl, a | these combinations for future expansion. By default, as in Perl, a |
| 1293 | backslash followed by a letter with no special meaning is treated as a | backslash followed by a letter with no special meaning is treated as a |
| 1294 | literal. (Perl can, however, be persuaded to give a warning for this.) | literal. (Perl can, however, be persuaded to give an error for this, by |
| 1295 | There are at present no other features controlled by this option. It | running it with the -w option.) There are at present no other features |
| 1296 | can also be set by a (?X) option setting within a pattern. | controlled by this option. It can also be set by a (?X) option setting |
| 1297 | within a pattern. | |
| 1298 | ||
| 1299 | PCRE_FIRSTLINE | PCRE_FIRSTLINE |
| 1300 | ||
| # | Line 1355 COMPILING A PATTERN | Line 1382 COMPILING A PATTERN |
| 1382 | be used for capturing (and they acquire numbers in the usual way). | be used for capturing (and they acquire numbers in the usual way). |
| 1383 | There is no equivalent of this option in Perl. | There is no equivalent of this option in Perl. |
| 1384 | ||
| 1385 | PCRE_UCP | |
| 1386 | ||
| 1387 | This option changes the way PCRE processes \b, \d, \s, \w, and some of | |
| 1388 | the POSIX character classes. By default, only ASCII characters are rec- | |
| 1389 | ognized, but if PCRE_UCP is set, Unicode properties are used instead to | |
| 1390 | classify characters. More details are given in the section on generic | |
| 1391 | character types in the pcrepattern page. If you set PCRE_UCP, matching | |
| 1392 | one of the items it affects takes much longer. The option is available | |
| 1393 | only if PCRE has been compiled with Unicode property support. | |
| 1394 | ||
| 1395 | PCRE_UNGREEDY | PCRE_UNGREEDY |
| 1396 | ||
| 1397 | This option inverts the "greediness" of the quantifiers so that they | This option inverts the "greediness" of the quantifiers so that they |
| # | Line 1453 COMPILATION ERROR CODES | Line 1490 COMPILATION ERROR CODES |
| 1490 | 57 \g is not followed by a braced, angle-bracketed, or quoted | 57 \g is not followed by a braced, angle-bracketed, or quoted |
| 1491 | name/number or by a plain number | name/number or by a plain number |
| 1492 | 58 a numbered reference must not be zero | 58 a numbered reference must not be zero |
| 1493 | 59 (*VERB) with an argument is not supported | 59 an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT) |
| 1494 | 60 (*VERB) not recognized | 60 (*VERB) not recognized |
| 1495 | 61 number is too big | 61 number is too big |
| 1496 | 62 subpattern name expected | 62 subpattern name expected |
| 1497 | 63 digit expected after (?+ | 63 digit expected after (?+ |
| 1498 | 64 ] is an invalid data character in JavaScript compatibility mode | 64 ] is an invalid data character in JavaScript compatibility mode |
| 1499 | 65 different names for subpatterns of the same number are not | |
| 1500 | allowed | |
| 1501 | 66 (*MARK) must have an argument | |
| 1502 | 67 this version of PCRE is not compiled with PCRE_UCP support | |
| 1503 | ||
| 1504 | The numbers 32 and 10000 in errors 48 and 49 are defaults; different | The numbers 32 and 10000 in errors 48 and 49 are defaults; different |
| 1505 | values may be used if the limits were changed when PCRE was built. | values may be used if the limits were changed when PCRE was built. |
| 1506 | ||
| 1507 | ||
| # | Line 1469 STUDYING A PATTERN | Line 1510 STUDYING A PATTERN |
| 1510 | pcre_extra *pcre_study(const pcre *code, int options | pcre_extra *pcre_study(const pcre *code, int options |
| 1511 | const char **errptr); | const char **errptr); |
| 1512 | ||
| 1513 | If a compiled pattern is going to be used several times, it is worth | If a compiled pattern is going to be used several times, it is worth |
| 1514 | spending more time analyzing it in order to speed up the time taken for | spending more time analyzing it in order to speed up the time taken for |
| 1515 | matching. The function pcre_study() takes a pointer to a compiled pat- | matching. The function pcre_study() takes a pointer to a compiled pat- |
| 1516 | tern as its first argument. If studying the pattern produces additional | tern as its first argument. If studying the pattern produces additional |
| 1517 | information that will help speed up matching, pcre_study() returns a | information that will help speed up matching, pcre_study() returns a |
| 1518 | pointer to a pcre_extra block, in which the study_data field points to | pointer to a pcre_extra block, in which the study_data field points to |
| 1519 | the results of the study. | the results of the study. |
| 1520 | ||
| 1521 | The returned value from pcre_study() can be passed directly to | The returned value from pcre_study() can be passed directly to |
| 1522 | pcre_exec(). However, a pcre_extra block also contains other fields | pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con- |
| 1523 | that can be set by the caller before the block is passed; these are | tains other fields that can be set by the caller before the block is |
| 1524 | described below in the section on matching a pattern. | passed; these are described below in the section on matching a pattern. |
| 1525 | ||
| 1526 | If studying the pattern does not produce any additional information | If studying the pattern does not produce any useful information, |
| 1527 | pcre_study() returns NULL. In that circumstance, if the calling program | pcre_study() returns NULL. In that circumstance, if the calling program |
| 1528 | wants to pass any of the other fields to pcre_exec(), it must set up | wants to pass any of the other fields to pcre_exec() or |
| 1529 | its own pcre_extra block. | pcre_dfa_exec(), it must set up its own pcre_extra block. |
| 1530 | ||
| 1531 | The second argument of pcre_study() contains option bits. At present, | The second argument of pcre_study() contains option bits. At present, |
| 1532 | no options are defined, and this argument should always be zero. | no options are defined, and this argument should always be zero. |
| 1533 | ||
| 1534 | The third argument for pcre_study() is a pointer for an error message. | The third argument for pcre_study() is a pointer for an error message. |
| 1535 | If studying succeeds (even if no data is returned), the variable it | If studying succeeds (even if no data is returned), the variable it |
| 1536 | points to is set to NULL. Otherwise it is set to point to a textual | points to is set to NULL. Otherwise it is set to point to a textual |
| 1537 | error message. This is a static string that is part of the library. You | error message. This is a static string that is part of the library. You |
| 1538 | must not try to free it. You should test the error pointer for NULL | must not try to free it. You should test the error pointer for NULL |
| 1539 | after calling pcre_study(), to be sure that it has run successfully. | after calling pcre_study(), to be sure that it has run successfully. |
| 1540 | ||
| 1541 | This is a typical call to pcre_study(): | This is a typical call to pcre_study(): |
| # | Line 1505 STUDYING A PATTERN | Line 1546 STUDYING A PATTERN |
| 1546 | 0, /* no options exist */ | 0, /* no options exist */ |
| 1547 | &error); /* set to NULL or points to a message */ | &error); /* set to NULL or points to a message */ |
| 1548 | ||
| 1549 | At present, studying a pattern is useful only for non-anchored patterns | Studying a pattern does two things: first, a lower bound for the length |
| 1550 | that do not have a single fixed starting character. A bitmap of possi- | of subject string that is needed to match the pattern is computed. This |
| 1551 | ble starting bytes is created. | does not mean that there are any strings of that length that match, but |
| 1552 | it does guarantee that no shorter strings match. The value is used by | |
| 1553 | pcre_exec() and pcre_dfa_exec() to avoid wasting time by trying to | |
| 1554 | match strings that are shorter than the lower bound. You can find out | |
| 1555 | the value in a calling program via the pcre_fullinfo() function. | |
| 1556 | ||
| 1557 | Studying a pattern is also useful for non-anchored patterns that do not | |
| 1558 | have a single fixed starting character. A bitmap of possible starting | |
| 1559 | bytes is created. This speeds up finding a position in the subject at | |
| 1560 | which to start matching. | |
| 1561 | ||
| 1562 | ||
| 1563 | LOCALE SUPPORT | LOCALE SUPPORT |
| # | Line 1515 LOCALE SUPPORT | Line 1565 LOCALE SUPPORT |
| 1565 | PCRE handles caseless matching, and determines whether characters are | PCRE handles caseless matching, and determines whether characters are |
| 1566 | letters, digits, or whatever, by reference to a set of tables, indexed | letters, digits, or whatever, by reference to a set of tables, indexed |
| 1567 | by character value. When running in UTF-8 mode, this applies only to | by character value. When running in UTF-8 mode, this applies only to |
| 1568 | characters with codes less than 128. Higher-valued codes never match | characters with codes less than 128. By default, higher-valued codes |
| 1569 | escapes such as \w or \d, but can be tested with \p if PCRE is built | never match escapes such as \w or \d, but they can be tested with \p if |
| 1570 | with Unicode character property support. The use of locales with Uni- | PCRE is built with Unicode character property support. Alternatively, |
| 1571 | code is discouraged. If you are handling characters with codes greater | the PCRE_UCP option can be set at compile time; this causes \w and |
| 1572 | than 128, you should either use UTF-8 and Unicode, or use locales, but | friends to use Unicode property support instead of built-in tables. The |
| 1573 | not try to mix the two. | use of locales with Unicode is discouraged. If you are handling charac- |
| 1574 | ters with codes greater than 128, you should either use UTF-8 and Uni- | |
| 1575 | code, or use locales, but not try to mix the two. | |
| 1576 | ||
| 1577 | PCRE contains an internal set of tables that are used when the final | PCRE contains an internal set of tables that are used when the final |
| 1578 | argument of pcre_compile() is NULL. These are sufficient for many | argument of pcre_compile() is NULL. These are sufficient for many |
| # | Line 1673 INFORMATION ABOUT A PATTERN | Line 1725 INFORMATION ABOUT A PATTERN |
| 1725 | example, for the pattern /^a\d+z\d+/ the returned value is "z", but for | example, for the pattern /^a\d+z\d+/ the returned value is "z", but for |
| 1726 | /^a\dz\d/ the returned value is -1. | /^a\dz\d/ the returned value is -1. |
| 1727 | ||
| 1728 | PCRE_INFO_MINLENGTH | |
| 1729 | ||
| 1730 | If the pattern was studied and a minimum length for matching subject | |
| 1731 | strings was computed, its value is returned. Otherwise the returned | |
| 1732 | value is -1. The value is a number of characters, not bytes (this may | |
| 1733 | be relevant in UTF-8 mode). The fourth argument should point to an int | |
| 1734 | variable. A non-negative value is a lower bound to the length of any | |
| 1735 | matching string. There may not be any strings of that length that do | |
| 1736 | actually match, but every string that does match is at least that long. | |
| 1737 | ||
| 1738 | PCRE_INFO_NAMECOUNT | PCRE_INFO_NAMECOUNT |
| 1739 | PCRE_INFO_NAMEENTRYSIZE | PCRE_INFO_NAMEENTRYSIZE |
| 1740 | PCRE_INFO_NAMETABLE | PCRE_INFO_NAMETABLE |
| # | Line 1694 INFORMATION ABOUT A PATTERN | Line 1756 INFORMATION ABOUT A PATTERN |
| 1756 | a pointer to the first entry of the table (a pointer to char). The | a pointer to the first entry of the table (a pointer to char). The |
| 1757 | first two bytes of each entry are the number of the capturing parenthe- | first two bytes of each entry are the number of the capturing parenthe- |
| 1758 | sis, most significant byte first. The rest of the entry is the corre- | sis, most significant byte first. The rest of the entry is the corre- |
| 1759 | sponding name, zero terminated. The names are in alphabetical order. | sponding name, zero terminated. |
| 1760 | When PCRE_DUPNAMES is set, duplicate names are in order of their paren- | |
| 1761 | theses numbers. For example, consider the following pattern (assume | The names are in alphabetical order. Duplicate names may appear if (?| |
| 1762 | PCRE_EXTENDED is set, so white space - including newlines - is | is used to create multiple groups with the same number, as described in |
| 1763 | ignored): | the section on duplicate subpattern numbers in the pcrepattern page. |
| 1764 | Duplicate names for subpatterns with different numbers are permitted | |
| 1765 | only if PCRE_DUPNAMES is set. In all cases of duplicate names, they | |
| 1766 | appear in the table in the order in which they were found in the pat- | |
| 1767 | tern. In the absence of (?| this is the order of increasing number; | |
| 1768 | when (?| is used this is not necessarily the case because later subpat- | |
| 1769 | terns may have lower numbers. | |
| 1770 | ||
| 1771 | As a simple example of the name/number table, consider the following | |
| 1772 | pattern (assume PCRE_EXTENDED is set, so white space - including new- | |
| 1773 | lines - is ignored): | |
| 1774 | ||
| 1775 | (?<date> (?<year>(\d\d)?\d\d) - | (?<date> (?<year>(\d\d)?\d\d) - |
| 1776 | (?<month>\d\d) - (?<day>\d\d) ) | (?<month>\d\d) - (?<day>\d\d) ) |
| # | Line 1719 INFORMATION ABOUT A PATTERN | Line 1791 INFORMATION ABOUT A PATTERN |
| 1791 | ||
| 1792 | PCRE_INFO_OKPARTIAL | PCRE_INFO_OKPARTIAL |
| 1793 | ||
| 1794 | Return 1 if the pattern can be used for partial matching, otherwise 0. | Return 1 if the pattern can be used for partial matching with |
| 1795 | The fourth argument should point to an int variable. From release 8.00, | pcre_exec(), otherwise 0. The fourth argument should point to an int |
| 1796 | this always returns 1, because the restrictions that previously applied | variable. From release 8.00, this always returns 1, because the |
| 1797 | to partial matching have been lifted. The pcrepartial documentation | restrictions that previously applied to partial matching have been |
| 1798 | gives details of partial matching. | lifted. The pcrepartial documentation gives details of partial match- |
| 1799 | ing. | |
| 1800 | ||
| 1801 | PCRE_INFO_OPTIONS | PCRE_INFO_OPTIONS |
| 1802 | ||
| 1803 | Return a copy of the options with which the pattern was compiled. The | Return a copy of the options with which the pattern was compiled. The |
| 1804 | fourth argument should point to an unsigned long int variable. These | fourth argument should point to an unsigned long int variable. These |
| 1805 | option bits are those specified in the call to pcre_compile(), modified | option bits are those specified in the call to pcre_compile(), modified |
| 1806 | by any top-level option settings at the start of the pattern itself. In | by any top-level option settings at the start of the pattern itself. In |
| 1807 | other words, they are the options that will be in force when matching | other words, they are the options that will be in force when matching |
| 1808 | starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with | starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with |
| 1809 | the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, | the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, |
| 1810 | and PCRE_EXTENDED. | and PCRE_EXTENDED. |
| 1811 | ||
| 1812 | A pattern is automatically anchored by PCRE if all of its top-level | A pattern is automatically anchored by PCRE if all of its top-level |
| 1813 | alternatives begin with one of the following: | alternatives begin with one of the following: |
| 1814 | ||
| 1815 | ^ unless PCRE_MULTILINE is set | ^ unless PCRE_MULTILINE is set |
| # | Line 1750 INFORMATION ABOUT A PATTERN | Line 1823 INFORMATION ABOUT A PATTERN |
| 1823 | ||
| 1824 | PCRE_INFO_SIZE | PCRE_INFO_SIZE |
| 1825 | ||
| 1826 | Return the size of the compiled pattern, that is, the value that was | Return the size of the compiled pattern, that is, the value that was |
| 1827 | passed as the argument to pcre_malloc() when PCRE was getting memory in | passed as the argument to pcre_malloc() when PCRE was getting memory in |
| 1828 | which to place the compiled data. The fourth argument should point to a | which to place the compiled data. The fourth argument should point to a |
| 1829 | size_t variable. | size_t variable. |
| # | Line 1758 INFORMATION ABOUT A PATTERN | Line 1831 INFORMATION ABOUT A PATTERN |
| 1831 | PCRE_INFO_STUDYSIZE | PCRE_INFO_STUDYSIZE |
| 1832 | ||
| 1833 | Return the size of the data block pointed to by the study_data field in | Return the size of the data block pointed to by the study_data field in |
| 1834 | a pcre_extra block. That is, it is the value that was passed to | a pcre_extra block. That is, it is the value that was passed to |
| 1835 | pcre_malloc() when PCRE was getting memory into which to place the data | pcre_malloc() when PCRE was getting memory into which to place the data |
| 1836 | created by pcre_study(). The fourth argument should point to a size_t | created by pcre_study(). If pcre_extra is NULL, or there is no study |
| 1837 | data, zero is returned. The fourth argument should point to a size_t | |
| 1838 | variable. | variable. |
| 1839 | ||
| 1840 | ||
| # | Line 1816 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 1890 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 1890 | ||
| 1891 | The function pcre_exec() is called to match a subject string against a | The function pcre_exec() is called to match a subject string against a |
| 1892 | compiled pattern, which is passed in the code argument. If the pattern | compiled pattern, which is passed in the code argument. If the pattern |
| 1893 | has been studied, the result of the study should be passed in the extra | was studied, the result of the study should be passed in the extra |
| 1894 | argument. This function is the main matching facility of the library, | argument. This function is the main matching facility of the library, |
| 1895 | and it operates in a Perl-like manner. For specialist use there is also | and it operates in a Perl-like manner. For specialist use there is also |
| 1896 | an alternative matching function, which is described below in the sec- | an alternative matching function, which is described below in the sec- |
| # | Line 1856 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 1930 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 1930 | unsigned long int match_limit_recursion; | unsigned long int match_limit_recursion; |
| 1931 | void *callout_data; | void *callout_data; |
| 1932 | const unsigned char *tables; | const unsigned char *tables; |
| 1933 | unsigned char **mark; | |
| 1934 | ||
| 1935 | The flags field is a bitmap that specifies which of the other fields | The flags field is a bitmap that specifies which of the other fields |
| 1936 | are set. The flag bits are: | are set. The flag bits are: |
| # | Line 1865 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 1940 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 1940 | PCRE_EXTRA_MATCH_LIMIT_RECURSION | PCRE_EXTRA_MATCH_LIMIT_RECURSION |
| 1941 | PCRE_EXTRA_CALLOUT_DATA | PCRE_EXTRA_CALLOUT_DATA |
| 1942 | PCRE_EXTRA_TABLES | PCRE_EXTRA_TABLES |
| 1943 | PCRE_EXTRA_MARK | |
| 1944 | ||
| 1945 | Other flag bits should be set to zero. The study_data field is set in | Other flag bits should be set to zero. The study_data field is set in |
| 1946 | the pcre_extra block that is returned by pcre_study(), together with | the pcre_extra block that is returned by pcre_study(), together with |
| # | Line 1875 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 1951 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 1951 | The match_limit field provides a means of preventing PCRE from using up | The match_limit field provides a means of preventing PCRE from using up |
| 1952 | a vast amount of resources when running patterns that are not going to | a vast amount of resources when running patterns that are not going to |
| 1953 | match, but which have a very large number of possibilities in their | match, but which have a very large number of possibilities in their |
| 1954 | search trees. The classic example is the use of nested unlimited | search trees. The classic example is a pattern that uses nested unlim- |
| 1955 | repeats. | ited repeats. |
| 1956 | ||
| 1957 | Internally, PCRE uses a function called match() which it calls repeat- | Internally, PCRE uses a function called match() which it calls repeat- |
| 1958 | edly (sometimes recursively). The limit set by match_limit is imposed | edly (sometimes recursively). The limit set by match_limit is imposed |
| # | Line 1909 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 1985 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 1985 | PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the | PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the |
| 1986 | limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT. | limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT. |
| 1987 | ||
| 1988 | The pcre_callout field is used in conjunction with the "callout" fea- | The callout_data field is used in conjunction with the "callout" fea- |
| 1989 | ture, which is described in the pcrecallout documentation. | ture, and is described in the pcrecallout documentation. |
| 1990 | ||
| 1991 | The tables field is used to pass a character tables pointer to | The tables field is used to pass a character tables pointer to |
| 1992 | pcre_exec(); this overrides the value that is stored with the compiled | pcre_exec(); this overrides the value that is stored with the compiled |
| # | Line 1923 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 1999 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 1999 | address when pcre_exec() is called. See the pcreprecompile documenta- | address when pcre_exec() is called. See the pcreprecompile documenta- |
| 2000 | tion for a discussion of saving compiled patterns for later use. | tion for a discussion of saving compiled patterns for later use. |
| 2001 | ||
| 2002 | If PCRE_EXTRA_MARK is set in the flags field, the mark field must be | |
| 2003 | set to point to a char * variable. If the pattern contains any back- | |
| 2004 | tracking control verbs such as (*MARK:NAME), and the execution ends up | |
| 2005 | with a name to pass back, a pointer to the name string (zero termi- | |
| 2006 | nated) is placed in the variable pointed to by the mark field. The | |
| 2007 | names are within the compiled pattern; if you wish to retain such a | |
| 2008 | name you must copy it before freeing the memory of a compiled pattern. | |
| 2009 | If there is no name to pass back, the variable pointed to by the mark | |
| 2010 | field set to NULL. For details of the backtracking control verbs, see | |
| 2011 | the section entitled "Backtracking control" in the pcrepattern documen- | |
| 2012 | tation. | |
| 2013 | ||
| 2014 | Option bits for pcre_exec() | Option bits for pcre_exec() |
| 2015 | ||
| 2016 | The unused bits of the options argument for pcre_exec() must be zero. | The unused bits of the options argument for pcre_exec() must be zero. |
| 2017 | The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx, | The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx, |
| 2018 | PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_START_OPTIMIZE, | PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, |
| 2019 | PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and PCRE_PARTIAL_HARD. | PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and |
| 2020 | PCRE_PARTIAL_HARD. | |
| 2021 | ||
| 2022 | PCRE_ANCHORED | PCRE_ANCHORED |
| 2023 | ||
| 2024 | The PCRE_ANCHORED option limits pcre_exec() to matching at the first | The PCRE_ANCHORED option limits pcre_exec() to matching at the first |
| 2025 | matching position. If a pattern was compiled with PCRE_ANCHORED, or | matching position. If a pattern was compiled with PCRE_ANCHORED, or |
| 2026 | turned out to be anchored by virtue of its contents, it cannot be made | turned out to be anchored by virtue of its contents, it cannot be made |
| 2027 | unachored at matching time. | unachored at matching time. |
| 2028 | ||
| 2029 | PCRE_BSR_ANYCRLF | PCRE_BSR_ANYCRLF |
| 2030 | PCRE_BSR_UNICODE | PCRE_BSR_UNICODE |
| 2031 | ||
| 2032 | These options (which are mutually exclusive) control what the \R escape | These options (which are mutually exclusive) control what the \R escape |
| 2033 | sequence matches. The choice is either to match only CR, LF, or CRLF, | sequence matches. The choice is either to match only CR, LF, or CRLF, |
| 2034 | or to match any Unicode newline sequence. These options override the | or to match any Unicode newline sequence. These options override the |
| 2035 | choice that was made or defaulted when the pattern was compiled. | choice that was made or defaulted when the pattern was compiled. |
| 2036 | ||
| 2037 | PCRE_NEWLINE_CR | PCRE_NEWLINE_CR |
| # | Line 1951 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 2040 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 2040 | PCRE_NEWLINE_ANYCRLF | PCRE_NEWLINE_ANYCRLF |
| 2041 | PCRE_NEWLINE_ANY | PCRE_NEWLINE_ANY |
| 2042 | ||
| 2043 | These options override the newline definition that was chosen or | These options override the newline definition that was chosen or |
| 2044 | defaulted when the pattern was compiled. For details, see the descrip- | defaulted when the pattern was compiled. For details, see the descrip- |
| 2045 | tion of pcre_compile() above. During matching, the newline choice | tion of pcre_compile() above. During matching, the newline choice |
| 2046 | affects the behaviour of the dot, circumflex, and dollar metacharac- | affects the behaviour of the dot, circumflex, and dollar metacharac- |
| 2047 | ters. It may also alter the way the match position is advanced after a | ters. It may also alter the way the match position is advanced after a |
| 2048 | match failure for an unanchored pattern. | match failure for an unanchored pattern. |
| 2049 | ||
| 2050 | When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is | When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is |
| 2051 | set, and a match attempt for an unanchored pattern fails when the cur- | set, and a match attempt for an unanchored pattern fails when the cur- |
| 2052 | rent position is at a CRLF sequence, and the pattern contains no | rent position is at a CRLF sequence, and the pattern contains no |
| 2053 | explicit matches for CR or LF characters, the match position is | explicit matches for CR or LF characters, the match position is |
| 2054 | advanced by two characters instead of one, in other words, to after the | advanced by two characters instead of one, in other words, to after the |
| 2055 | CRLF. | CRLF. |
| 2056 | ||
| 2057 | The above rule is a compromise that makes the most common cases work as | The above rule is a compromise that makes the most common cases work as |
| 2058 | expected. For example, if the pattern is .+A (and the PCRE_DOTALL | expected. For example, if the pattern is .+A (and the PCRE_DOTALL |
| 2059 | option is not set), it does not match the string "\r\nA" because, after | option is not set), it does not match the string "\r\nA" because, after |
| 2060 | failing at the start, it skips both the CR and the LF before retrying. | failing at the start, it skips both the CR and the LF before retrying. |
| 2061 | However, the pattern [\r\n]A does match that string, because it con- | However, the pattern [\r\n]A does match that string, because it con- |
| 2062 | tains an explicit CR or LF reference, and so advances only by one char- | tains an explicit CR or LF reference, and so advances only by one char- |
| 2063 | acter after the first failure. | acter after the first failure. |
| 2064 | ||
| 2065 | An explicit match for CR of LF is either a literal appearance of one of | An explicit match for CR of LF is either a literal appearance of one of |
| 2066 | those characters, or one of the \r or \n escape sequences. Implicit | those characters, or one of the \r or \n escape sequences. Implicit |
| 2067 | matches such as [^X] do not count, nor does \s (which includes CR and | matches such as [^X] do not count, nor does \s (which includes CR and |
| 2068 | LF in the characters that it matches). | LF in the characters that it matches). |
| 2069 | ||
| 2070 | Notwithstanding the above, anomalous effects may still occur when CRLF | Notwithstanding the above, anomalous effects may still occur when CRLF |
| 2071 | is a valid newline sequence and explicit \r or \n escapes appear in the | is a valid newline sequence and explicit \r or \n escapes appear in the |
| 2072 | pattern. | pattern. |
| 2073 | ||
| 2074 | PCRE_NOTBOL | PCRE_NOTBOL |
| 2075 | ||
| 2076 | This option specifies that first character of the subject string is not | This option specifies that first character of the subject string is not |
| 2077 | the beginning of a line, so the circumflex metacharacter should not | the beginning of a line, so the circumflex metacharacter should not |
| 2078 | match before it. Setting this without PCRE_MULTILINE (at compile time) | match before it. Setting this without PCRE_MULTILINE (at compile time) |
| 2079 | causes circumflex never to match. This option affects only the behav- | causes circumflex never to match. This option affects only the behav- |
| 2080 | iour of the circumflex metacharacter. It does not affect \A. | iour of the circumflex metacharacter. It does not affect \A. |
| 2081 | ||
| 2082 | PCRE_NOTEOL | PCRE_NOTEOL |
| 2083 | ||
| 2084 | This option specifies that the end of the subject string is not the end | This option specifies that the end of the subject string is not the end |
| 2085 | of a line, so the dollar metacharacter should not match it nor (except | of a line, so the dollar metacharacter should not match it nor (except |
| 2086 | in multiline mode) a newline immediately before it. Setting this with- | in multiline mode) a newline immediately before it. Setting this with- |
| 2087 | out PCRE_MULTILINE (at compile time) causes dollar never to match. This | out PCRE_MULTILINE (at compile time) causes dollar never to match. This |
| 2088 | option affects only the behaviour of the dollar metacharacter. It does | option affects only the behaviour of the dollar metacharacter. It does |
| 2089 | not affect \Z or \z. | not affect \Z or \z. |
| 2090 | ||
| 2091 | PCRE_NOTEMPTY | PCRE_NOTEMPTY |
| 2092 | ||
| 2093 | An empty string is not considered to be a valid match if this option is | An empty string is not considered to be a valid match if this option is |
| 2094 | set. If there are alternatives in the pattern, they are tried. If all | set. If there are alternatives in the pattern, they are tried. If all |
| 2095 | the alternatives match the empty string, the entire match fails. For | the alternatives match the empty string, the entire match fails. For |
| 2096 | example, if the pattern | example, if the pattern |
| 2097 | ||
| 2098 | a?b? | a?b? |
| 2099 | ||
| 2100 | is applied to a string not beginning with "a" or "b", it matches the | is applied to a string not beginning with "a" or "b", it matches an |
| 2101 | empty string at the start of the subject. With PCRE_NOTEMPTY set, this | empty string at the start of the subject. With PCRE_NOTEMPTY set, this |
| 2102 | match is not valid, so PCRE searches further into the string for occur- | match is not valid, so PCRE searches further into the string for occur- |
| 2103 | rences of "a" or "b". | rences of "a" or "b". |
| 2104 | ||
| 2105 | Perl has no direct equivalent of PCRE_NOTEMPTY, but it does make a spe- | PCRE_NOTEMPTY_ATSTART |
| 2106 | cial case of a pattern match of the empty string within its split() | |
| 2107 | function, and when using the /g modifier. It is possible to emulate | This is like PCRE_NOTEMPTY, except that an empty string match that is |
| 2108 | Perl's behaviour after matching a null string by first trying the match | not at the start of the subject is permitted. If the pattern is |
| 2109 | again at the same offset with PCRE_NOTEMPTY and PCRE_ANCHORED, and then | anchored, such a match can occur only if the pattern contains \K. |
| 2110 | if that fails by advancing the starting offset (see below) and trying | |
| 2111 | an ordinary match again. There is some code that demonstrates how to do | Perl has no direct equivalent of PCRE_NOTEMPTY or |
| 2112 | this in the pcredemo sample program. | PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern |
| 2113 | match of the empty string within its split() function, and when using | |
| 2114 | the /g modifier. It is possible to emulate Perl's behaviour after | |
| 2115 | matching a null string by first trying the match again at the same off- | |
| 2116 | set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that | |
| 2117 | fails, by advancing the starting offset (see below) and trying an ordi- | |
| 2118 | nary match again. There is some code that demonstrates how to do this | |
| 2119 | in the pcredemo sample program. | |
| 2120 | ||
| 2121 | PCRE_NO_START_OPTIMIZE | PCRE_NO_START_OPTIMIZE |
| 2122 | ||
| # | Line 2066 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 2162 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 2162 | returns PCRE_ERROR_PARTIAL. Otherwise, if PCRE_PARTIAL_SOFT is set, | returns PCRE_ERROR_PARTIAL. Otherwise, if PCRE_PARTIAL_SOFT is set, |
| 2163 | matching continues by testing any other alternatives. Only if they all | matching continues by testing any other alternatives. Only if they all |
| 2164 | fail is PCRE_ERROR_PARTIAL returned (instead of PCRE_ERROR_NOMATCH). | fail is PCRE_ERROR_PARTIAL returned (instead of PCRE_ERROR_NOMATCH). |
| 2165 | The portion of the string that provided the partial match is set as the | The portion of the string that was inspected when the partial match was |
| 2166 | first matching string. There is a more detailed discussion in the | found is set as the first matching string. There is a more detailed |
| 2167 | pcrepartial documentation. | discussion in the pcrepartial documentation. |
| 2168 | ||
| 2169 | The string to be matched by pcre_exec() | The string to be matched by pcre_exec() |
| 2170 | ||
| # | Line 2155 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 2251 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 2251 | has to get additional memory for use during matching. Thus it is usu- | has to get additional memory for use during matching. Thus it is usu- |
| 2252 | ally advisable to supply an ovector. | ally advisable to supply an ovector. |
| 2253 | ||
| 2254 | The pcre_info() function can be used to find out how many capturing | The pcre_fullinfo() function can be used to find out how many capturing |
| 2255 | subpatterns there are in a compiled pattern. The smallest size for | subpatterns there are in a compiled pattern. The smallest size for |
| 2256 | ovector that will allow for n captured substrings, in addition to the | ovector that will allow for n captured substrings, in addition to the |
| 2257 | offsets of the substring matched by the whole pattern, is (n+1)*3. | offsets of the substring matched by the whole pattern, is (n+1)*3. |
| # | Line 2218 MATCHING A PATTERN: THE TRADITIONAL FUNC | Line 2314 MATCHING A PATTERN: THE TRADITIONAL FUNC |
| 2314 | purpose. If the call via pcre_malloc() fails, this error is given. The | purpose. If the call via pcre_malloc() fails, this error is given. The |
| 2315 | memory is automatically freed at the end of matching. | memory is automatically freed at the end of matching. |
| 2316 | ||
| 2317 | This error is also given if pcre_stack_malloc() fails in pcre_exec(). | |
| 2318 | This can happen only when PCRE has been compiled with --disable-stack- | |
| 2319 | for-recursion. | |
| 2320 | ||
| 2321 | PCRE_ERROR_NOSUBSTRING (-7) | PCRE_ERROR_NOSUBSTRING (-7) |
| 2322 | ||
| 2323 | This error is used by the pcre_copy_substring(), pcre_get_substring(), | This error is used by the pcre_copy_substring(), pcre_get_substring(), |
| # | Line 2416 EXTRACTING CAPTURED SUBSTRINGS BY NAME | Line 2516 EXTRACTING CAPTURED SUBSTRINGS BY NAME |
| 2516 | ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the | ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the |
| 2517 | behaviour may not be what you want (see the next section). | behaviour may not be what you want (see the next section). |
| 2518 | ||
| 2519 | Warning: If the pattern uses the "(?|" feature to set up multiple sub- | Warning: If the pattern uses the (?| feature to set up multiple subpat- |
| 2520 | patterns with the same number, you cannot use names to distinguish | terns with the same number, as described in the section on duplicate |
| 2521 | them, because names are not included in the compiled code. The matching | subpattern numbers in the pcrepattern page, you cannot use names to |
| 2522 | process uses only numbers. | distinguish the different subpatterns, because names are not included |
| 2523 | in the compiled code. The matching process uses only numbers. For this | |
| 2524 | reason, the use of different names for subpatterns of the same number | |
| 2525 | causes an error at compile time. | |
| 2526 | ||
| 2527 | ||
| 2528 | DUPLICATE SUBPATTERN NAMES | DUPLICATE SUBPATTERN NAMES |
| # | Line 2427 DUPLICATE SUBPATTERN NAMES | Line 2530 DUPLICATE SUBPATTERN NAMES |
| 2530 | int pcre_get_stringtable_entries(const pcre *code, | int pcre_get_stringtable_entries(const pcre *code, |
| 2531 | const char *name, char **first, char **last); | const char *name, char **first, char **last); |
| 2532 | ||
| 2533 | When a pattern is compiled with the PCRE_DUPNAMES option, names for | When a pattern is compiled with the PCRE_DUPNAMES option, names for |
| 2534 | subpatterns are not required to be unique. Normally, patterns with | subpatterns are not required to be unique. (Duplicate names are always |
| 2535 | duplicate names are such that in any one match, only one of the named | allowed for subpatterns with the same number, created by using the (?| |
| 2536 | subpatterns participates. An example is shown in the pcrepattern docu- | feature. Indeed, if such subpatterns are named, they are required to |
| 2537 | mentation. | use the same names.) |
| 2538 | ||
| 2539 | When duplicates are present, pcre_copy_named_substring() and | Normally, patterns with duplicate names are such that in any one match, |
| 2540 | pcre_get_named_substring() return the first substring corresponding to | only one of the named subpatterns participates. An example is shown in |
| 2541 | the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING | the pcrepattern documentation. |
| 2542 | (-7) is returned; no data is returned. The pcre_get_stringnumber() | |
| 2543 | function returns one of the numbers that are associated with the name, | When duplicates are present, pcre_copy_named_substring() and |
| 2544 | pcre_get_named_substring() return the first substring corresponding to | |
| 2545 | the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING | |
| 2546 | (-7) is returned; no data is returned. The pcre_get_stringnumber() | |
| 2547 | function returns one of the numbers that are associated with the name, | |
| 2548 | but it is not defined which it is. | but it is not defined which it is. |
| 2549 | ||
| 2550 | If you want to get full details of all captured substrings for a given | If you want to get full details of all captured substrings for a given |
| 2551 | name, you must use the pcre_get_stringtable_entries() function. The | name, you must use the pcre_get_stringtable_entries() function. The |
| 2552 | first argument is the compiled pattern, and the second is the name. The | first argument is the compiled pattern, and the second is the name. The |
| 2553 | third and fourth are pointers to variables which are updated by the | third and fourth are pointers to variables which are updated by the |
| 2554 | function. After it has run, they point to the first and last entries in | function. After it has run, they point to the first and last entries in |
| 2555 | the name-to-number table for the given name. The function itself | the name-to-number table for the given name. The function itself |
| 2556 | returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if | returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if |
| 2557 | there are none. The format of the table is described above in the sec- | there are none. The format of the table is described above in the sec- |
| 2558 | tion entitled Information about a pattern. Given all the relevant | tion entitled Information about a pattern. Given all the relevant |
| 2559 | entries for the name, you can extract each of their numbers, and hence | entries for the name, you can extract each of their numbers, and hence |
| 2560 | the captured data, if any. | the captured data, if any. |
| 2561 | ||
| 2562 | ||
| 2563 | FINDING ALL POSSIBLE MATCHES | FINDING ALL POSSIBLE MATCHES |
| 2564 | ||
| 2565 | The traditional matching function uses a similar algorithm to Perl, | The traditional matching function uses a similar algorithm to Perl, |
| 2566 | which stops when it finds the first match, starting at a given point in | which stops when it finds the first match, starting at a given point in |
| 2567 | the subject. If you want to find all possible matches, or the longest | the subject. If you want to find all possible matches, or the longest |
| 2568 | possible match, consider using the alternative matching function (see | possible match, consider using the alternative matching function (see |
| 2569 | below) instead. If you cannot use the alternative function, but still | below) instead. If you cannot use the alternative function, but still |
| 2570 | need to find all possible matches, you can kludge it up by making use | need to find all possible matches, you can kludge it up by making use |
| 2571 | of the callout facility, which is described in the pcrecallout documen- | of the callout facility, which is described in the pcrecallout documen- |
| 2572 | tation. | tation. |
| 2573 | ||
| 2574 | What you have to do is to insert a callout right at the end of the pat- | What you have to do is to insert a callout right at the end of the pat- |
| 2575 | tern. When your callout function is called, extract and save the cur- | tern. When your callout function is called, extract and save the cur- |
| 2576 | rent matched substring. Then return 1, which forces pcre_exec() to | rent matched substring. Then return 1, which forces pcre_exec() to |
| 2577 | backtrack and try other alternatives. Ultimately, when it runs out of | backtrack and try other alternatives. Ultimately, when it runs out of |
| 2578 | matches, pcre_exec() will yield PCRE_ERROR_NOMATCH. | matches, pcre_exec() will yield PCRE_ERROR_NOMATCH. |
| 2579 | ||
| 2580 | ||
| # | Line 2478 MATCHING A PATTERN: THE ALTERNATIVE FUNC | Line 2585 MATCHING A PATTERN: THE ALTERNATIVE FUNC |
| 2585 | int options, int *ovector, int ovecsize, | int options, int *ovector, int ovecsize, |
| 2586 | int *workspace, int wscount); | int *workspace, int wscount); |
| 2587 | ||
| 2588 | The function pcre_dfa_exec() is called to match a subject string | The function pcre_dfa_exec() is called to match a subject string |
| 2589 | against a compiled pattern, using a matching algorithm that scans the | against a compiled pattern, using a matching algorithm that scans the |
| 2590 | subject string just once, and does not backtrack. This has different | subject string just once, and does not backtrack. This has different |
| 2591 | characteristics to the normal algorithm, and is not compatible with | characteristics to the normal algorithm, and is not compatible with |
| 2592 | Perl. Some of the features of PCRE patterns are not supported. Never- | Perl. Some of the features of PCRE patterns are not supported. Never- |
| 2593 | theless, there are times when this kind of matching can be useful. For | theless, there are times when this kind of matching can be useful. For |
| 2594 | a discussion of the two matching algorithms, see the pcrematching docu- | a discussion of the two matching algorithms, and a list of features |
| 2595 | mentation. | that pcre_dfa_exec() does not support, see the pcrematching documenta- |
| 2596 | tion. | |
| 2597 | ||
| 2598 | The arguments for the pcre_dfa_exec() function are the same as for | The arguments for the pcre_dfa_exec() function are the same as for |
| 2599 | pcre_exec(), plus two extras. The ovector argument is used in a differ- | pcre_exec(), plus two extras. The ovector argument is used in a differ- |
| # | Line 2520 MATCHING A PATTERN: THE ALTERNATIVE FUNC | Line 2628 MATCHING A PATTERN: THE ALTERNATIVE FUNC |
| 2628 | ||
| 2629 | The unused bits of the options argument for pcre_dfa_exec() must be | The unused bits of the options argument for pcre_dfa_exec() must be |
| 2630 | zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW- | zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW- |
| 2631 | LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK, | LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, |
| 2632 | PCRE_PARTIAL_HARD, PCRE_PARTIAL_SOFT, PCRE_DFA_SHORTEST, and | PCRE_NOTEMPTY_ATSTART, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, PCRE_PAR- |
| 2633 | PCRE_DFA_RESTART. All but the last four of these are exactly the same | TIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last |
| 2634 | as for pcre_exec(), so their description is not repeated here. | four of these are exactly the same as for pcre_exec(), so their |
| 2635 | description is not repeated here. | |
| 2636 | ||
| 2637 | PCRE_PARTIAL_HARD | PCRE_PARTIAL_HARD |
| 2638 | PCRE_PARTIAL_SOFT | PCRE_PARTIAL_SOFT |
| 2639 | ||
| 2640 | These have the same general effect as they do for pcre_exec(), but the | These have the same general effect as they do for pcre_exec(), but the |
| 2641 | details are slightly different. When PCRE_PARTIAL_HARD is set for | details are slightly different. When PCRE_PARTIAL_HARD is set for |
| 2642 | pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub- | pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub- |
| 2643 | ject is reached and there is still at least one matching possibility | ject is reached and there is still at least one matching possibility |
| 2644 | that requires additional characters. This happens even if some complete | that requires additional characters. This happens even if some complete |
| 2645 | matches have also been found. When PCRE_PARTIAL_SOFT is set, the return | matches have also been found. When PCRE_PARTIAL_SOFT is set, the return |
| 2646 | code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end | code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end |
| 2647 | of the subject is reached, there have been no complete matches, but | of the subject is reached, there have been no complete matches, but |
| 2648 | there is still at least one matching possibility. The portion of the | there is still at least one matching possibility. The portion of the |
| 2649 | string that provided the longest partial match is set as the first | string that was inspected when the longest partial match was found is |
| 2650 | matching string in both cases. | set as the first matching string in both cases. |
| 2651 | ||
| 2652 | PCRE_DFA_SHORTEST | PCRE_DFA_SHORTEST |
| 2653 | ||
| 2654 | Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to | Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to |
| 2655 | stop as soon as it has found one match. Because of the way the alterna- | stop as soon as it has found one match. Because of the way the alterna- |
| 2656 | tive algorithm works, this is necessarily the shortest possible match | tive algorithm works, this is necessarily the shortest possible match |
| 2657 | at the first possible matching point in the subject string. | at the first possible matching point in the subject string. |
| 2658 | ||
| 2659 | PCRE_DFA_RESTART | PCRE_DFA_RESTART |
| 2660 | ||
| 2661 | When pcre_dfa_exec() returns a partial match, it is possible to call it | When pcre_dfa_exec() returns a partial match, it is possible to call it |
| 2662 | again, with additional subject characters, and have it continue with | again, with additional subject characters, and have it continue with |
| 2663 | the same match. The PCRE_DFA_RESTART option requests this action; when | the same match. The PCRE_DFA_RESTART option requests this action; when |
| 2664 | it is set, the workspace and wscount options must reference the same | it is set, the workspace and wscount options must reference the same |
| 2665 | vector as before because data about the match so far is left in them | vector as before because data about the match so far is left in them |
| 2666 | after a partial match. There is more discussion of this facility in the | after a partial match. There is more discussion of this facility in the |
| 2667 | pcrepartial documentation. | pcrepartial documentation. |
| 2668 | ||
| 2669 | Successful returns from pcre_dfa_exec() | Successful returns from pcre_dfa_exec() |
| 2670 | ||
| 2671 | When pcre_dfa_exec() succeeds, it may have matched more than one sub- | When pcre_dfa_exec() succeeds, it may have matched more than one sub- |
| 2672 | string in the subject. Note, however, that all the matches from one run | string in the subject. Note, however, that all the matches from one run |
| 2673 | of the function start at the same point in the subject. The shorter | of the function start at the same point in the subject. The shorter |
| 2674 | matches are all initial substrings of the longer matches. For example, | matches are all initial substrings of the longer matches. For example, |
| 2675 | if the pattern | if the pattern |
| 2676 | ||
| 2677 | <.*> | <.*> |
| # | Line 2577 MATCHING A PATTERN: THE ALTERNATIVE FUNC | Line 2686 MATCHING A PATTERN: THE ALTERNATIVE FUNC |
| 2686 | <something> <something else> | <something> <something else> |
| 2687 | <something> <something else> <something further> | <something> <something else> <something further> |
| 2688 | ||
| 2689 | On success, the yield of the function is a number greater than zero, | On success, the yield of the function is a number greater than zero, |
| 2690 | which is the number of matched substrings. The substrings themselves | which is the number of matched substrings. The substrings themselves |
| 2691 | are returned in ovector. Each string uses two elements; the first is | are returned in ovector. Each string uses two elements; the first is |
| 2692 | the offset to the start, and the second is the offset to the end. In | the offset to the start, and the second is the offset to the end. In |
| 2693 | fact, all the strings have the same start offset. (Space could have | fact, all the strings have the same start offset. (Space could have |
| 2694 | been saved by giving this only once, but it was decided to retain some | been saved by giving this only once, but it was decided to retain some |
| 2695 | compatibility with the way pcre_exec() returns data, even though the | compatibility with the way pcre_exec() returns data, even though the |
| 2696 | meaning of the strings is different.) | meaning of the strings is different.) |
| 2697 | ||
| 2698 | The strings are returned in reverse order of length; that is, the long- | The strings are returned in reverse order of length; that is, the long- |
| 2699 | est matching string is given first. If there were too many matches to | est matching string is given first. If there were too many matches to |
| 2700 | fit into ovector, the yield of the function is zero, and the vector is | fit into ovector, the yield of the function is zero, and the vector is |
| 2701 | filled with the longest matches. | filled with the longest matches. |
| 2702 | ||
| 2703 | Error returns from pcre_dfa_exec() | Error returns from pcre_dfa_exec() |
| 2704 | ||
| 2705 | The pcre_dfa_exec() function returns a negative number when it fails. | The pcre_dfa_exec() function returns a negative number when it fails. |
| 2706 | Many of the errors are the same as for pcre_exec(), and these are | Many of the errors are the same as for pcre_exec(), and these are |
| 2707 | described above. There are in addition the following errors that are | described above. There are in addition the following errors that are |
| 2708 | specific to pcre_dfa_exec(): | specific to pcre_dfa_exec(): |
| 2709 | ||
| 2710 | PCRE_ERROR_DFA_UITEM (-16) | PCRE_ERROR_DFA_UITEM (-16) |
| 2711 | ||
| 2712 | This return is given if pcre_dfa_exec() encounters an item in the pat- | This return is given if pcre_dfa_exec() encounters an item in the pat- |
| 2713 | tern that it does not support, for instance, the use of \C or a back | tern that it does not support, for instance, the use of \C or a back |
| 2714 | reference. | reference. |
| 2715 | ||
| 2716 | PCRE_ERROR_DFA_UCOND (-17) | PCRE_ERROR_DFA_UCOND (-17) |
| 2717 | ||
| 2718 | This return is given if pcre_dfa_exec() encounters a condition item | This return is given if pcre_dfa_exec() encounters a condition item |
| 2719 | that uses a back reference for the condition, or a test for recursion | that uses a back reference for the condition, or a test for recursion |
| 2720 | in a specific group. These are not supported. | in a specific group. These are not supported. |
| 2721 | ||
| 2722 | PCRE_ERROR_DFA_UMLIMIT (-18) | PCRE_ERROR_DFA_UMLIMIT (-18) |
| 2723 | ||
| 2724 | This return is given if pcre_dfa_exec() is called with an extra block | This return is given if pcre_dfa_exec() is called with an extra block |
| 2725 | that contains a setting of the match_limit field. This is not supported | that contains a setting of the match_limit field. This is not supported |
| 2726 | (it is meaningless). | (it is meaningless). |
| 2727 | ||
| 2728 | PCRE_ERROR_DFA_WSSIZE (-19) | PCRE_ERROR_DFA_WSSIZE (-19) |
| 2729 | ||
| 2730 | This return is given if pcre_dfa_exec() runs out of space in the | This return is given if pcre_dfa_exec() runs out of space in the |
| 2731 | workspace vector. | workspace vector. |
| 2732 | ||
| 2733 | PCRE_ERROR_DFA_RECURSE (-20) | PCRE_ERROR_DFA_RECURSE (-20) |
| 2734 | ||
| 2735 | When a recursive subpattern is processed, the matching function calls | When a recursive subpattern is processed, the matching function calls |
| 2736 | itself recursively, using private vectors for ovector and workspace. | itself recursively, using private vectors for ovector and workspace. |
| 2737 | This error is given if the output vector is not large enough. This | This error is given if the output vector is not large enough. This |
| 2738 | should be extremely rare, as a vector of size 1000 is used. | should be extremely rare, as a vector of size 1000 is used. |
| 2739 | ||
| 2740 | ||
| 2741 | SEE ALSO | SEE ALSO |
| 2742 | ||
| 2743 | pcrebuild(3), pcrecallout(3), pcrecpp(3)(3), pcrematching(3), pcrepar- | pcrebuild(3), pcrecallout(3), pcrecpp(3)(3), pcrematching(3), pcrepar- |
| 2744 | tial(3), pcreposix(3), pcreprecompile(3), pcresample(3), pcrestack(3). | tial(3), pcreposix(3), pcreprecompile(3), pcresample(3), pcrestack(3). |
| 2745 | ||
| 2746 | ||
| # | Line 2644 AUTHOR | Line 2753 AUTHOR |
| 2753 | ||
| 2754 | REVISION | REVISION |
| 2755 | ||
| 2756 | Last updated: 01 September 2009 | Last updated: 01 June 2010 |
| 2757 | Copyright (c) 1997-2009 University of Cambridge. | Copyright (c) 1997-2010 University of Cambridge. |
| 2758 | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| 2759 | ||
| 2760 | ||
| 2761 | PCRECALLOUT(3) PCRECALLOUT(3) | PCRECALLOUT(3) PCRECALLOUT(3) |
| 2762 | ||
| 2763 | ||
| # | Line 2674 PCRE CALLOUTS | Line 2783 PCRE CALLOUTS |
| 2783 | ||
| 2784 | (?C1)abc(?C2)def | (?C1)abc(?C2)def |
| 2785 | ||
| 2786 | If the PCRE_AUTO_CALLOUT option bit is set when pcre_compile() is | If the PCRE_AUTO_CALLOUT option bit is set when pcre_compile() or |
| 2787 | called, PCRE automatically inserts callouts, all with number 255, | pcre_compile2() is called, PCRE automatically inserts callouts, all |
| 2788 | before each item in the pattern. For example, if PCRE_AUTO_CALLOUT is | with number 255, before each item in the pattern. For example, if |
| 2789 | used with the pattern | PCRE_AUTO_CALLOUT is used with the pattern |
| 2790 | ||
| 2791 | A(\d{2}|--) | A(\d{2}|--) |
| 2792 | ||
| # | Line 2706 MISSING CALLOUTS | Line 2815 MISSING CALLOUTS |
| 2815 | ever start, and the callout is never reached. However, with "abyd", | ever start, and the callout is never reached. However, with "abyd", |
| 2816 | though the result is still no match, the callout is obeyed. | though the result is still no match, the callout is obeyed. |
| 2817 | ||
| 2818 | You can disable these optimizations by passing the PCRE_NO_START_OPTI- | If the pattern is studied, PCRE knows the minimum length of a matching |
| 2819 | MIZE option to pcre_exec() or pcre_dfa_exec(). This slows down the | string, and will immediately give a "no match" return without actually |
| 2820 | matching process, but does ensure that callouts such as the example | running a match if the subject is not long enough, or, for unanchored |
| 2821 | patterns, if it has been scanned far enough. | |
| 2822 | ||
| 2823 | You can disable these optimizations by passing the PCRE_NO_START_OPTI- | |
| 2824 | MIZE option to pcre_exec() or pcre_dfa_exec(). This slows down the | |
| 2825 | matching process, but does ensure that callouts such as the example | |
| 2826 | above are obeyed. | above are obeyed. |
| 2827 | ||
| 2828 | ||
| 2829 | THE CALLOUT INTERFACE | THE CALLOUT INTERFACE |
| 2830 | ||
| 2831 | During matching, when PCRE reaches a callout point, the external func- | During matching, when PCRE reaches a callout point, the external func- |
| 2832 | tion defined by pcre_callout is called (if it is set). This applies to | tion defined by pcre_callout is called (if it is set). This applies to |
| 2833 | both the pcre_exec() and the pcre_dfa_exec() matching functions. The | both the pcre_exec() and the pcre_dfa_exec() matching functions. The |
| 2834 | only argument to the callout function is a pointer to a pcre_callout | only argument to the callout function is a pointer to a pcre_callout |
| 2835 | block. This structure contains the following fields: | block. This structure contains the following fields: |
| 2836 | ||
| 2837 | int version; | int version; |
| # | Line 2733 THE CALLOUT INTERFACE | Line 2847 THE CALLOUT INTERFACE |
| 2847 | int pattern_position; | int pattern_position; |
| 2848 | int next_item_length; | int next_item_length; |
| 2849 | ||
| 2850 | The version field is an integer containing the version number of the | The version field is an integer containing the version number of the |
| 2851 | block format. The initial version was 0; the current version is 1. The | block format. The initial version was 0; the current version is 1. The |
| 2852 | version number will change again in future if additional fields are | version number will change again in future if additional fields are |
| 2853 | added, but the intention is never to remove any of the existing fields. | added, but the intention is never to remove any of the existing fields. |
| 2854 | ||
| 2855 | The callout_number field contains the number of the callout, as com- | The callout_number field contains the number of the callout, as com- |
| 2856 | piled into the pattern (that is, the number after ?C for manual call- | piled into the pattern (that is, the number after ?C for manual call- |
| 2857 | outs, and 255 for automatically generated callouts). | outs, and 255 for automatically generated callouts). |
| 2858 | ||
| 2859 | The offset_vector field is a pointer to the vector of offsets that was | The offset_vector field is a pointer to the vector of offsets that was |
| 2860 | passed by the caller to pcre_exec() or pcre_dfa_exec(). When | passed by the caller to pcre_exec() or pcre_dfa_exec(). When |
| 2861 | pcre_exec() is used, the contents can be inspected in order to extract | pcre_exec() is used, the contents can be inspected in order to extract |
| 2862 | substrings that have been matched so far, in the same way as for | substrings that have been matched so far, in the same way as for |
| 2863 | extracting substrings after a match has completed. For pcre_dfa_exec() | extracting substrings after a match has completed. For pcre_dfa_exec() |
| 2864 | this field is not useful. | this field is not useful. |
| 2865 | ||
| 2866 | The subject and subject_length fields contain copies of the values that | The subject and subject_length fields contain copies of the values that |
| 2867 | were passed to pcre_exec(). | were passed to pcre_exec(). |
| 2868 | ||
| 2869 | The start_match field normally contains the offset within the subject | The start_match field normally contains the offset within the subject |
| 2870 | at which the current match attempt started. However, if the escape | at which the current match attempt started. However, if the escape |
| 2871 | sequence \K has been encountered, this value is changed to reflect the | sequence \K has been encountered, this value is changed to reflect the |
| 2872 | modified starting point. If the pattern is not anchored, the callout | modified starting point. If the pattern is not anchored, the callout |
| 2873 | function may be called several times from the same point in the pattern | function may be called several times from the same point in the pattern |
| 2874 | for different starting points in the subject. | for different starting points in the subject. |
| 2875 | ||
| 2876 | The current_position field contains the offset within the subject of | The current_position field contains the offset within the subject of |
| 2877 | the current match pointer. | the current match pointer. |
| 2878 | ||
| 2879 | When the pcre_exec() function is used, the capture_top field contains | When the pcre_exec() function is used, the capture_top field contains |
| 2880 | one more than the number of the highest numbered captured substring so | one more than the number of the highest numbered captured substring so |
| 2881 | far. If no substrings have been captured, the value of capture_top is | far. If no substrings have been captured, the value of capture_top is |
| 2882 | one. This is always the case when pcre_dfa_exec() is used, because it | one. This is always the case when pcre_dfa_exec() is used, because it |
| 2883 | does not support captured substrings. | does not support captured substrings. |
| 2884 | ||
| 2885 | The capture_last field contains the number of the most recently cap- | The capture_last field contains the number of the most recently cap- |
| 2886 | tured substring. If no substrings have been captured, its value is -1. | tured substring. If no substrings have been captured, its value is -1. |
| 2887 | This is always the case when pcre_dfa_exec() is used. | This is always the case when pcre_dfa_exec() is used. |
| 2888 | ||
| 2889 | The callout_data field contains a value that is passed to pcre_exec() | The callout_data field contains a value that is passed to pcre_exec() |
| 2890 | or pcre_dfa_exec() specifically so that it can be passed back in call- | or pcre_dfa_exec() specifically so that it can be passed back in call- |
| 2891 | outs. It is passed in the pcre_callout field of the pcre_extra data | outs. It is passed in the pcre_callout field of the pcre_extra data |
| 2892 | structure. If no such data was passed, the value of callout_data in a | structure. If no such data was passed, the value of callout_data in a |
| 2893 | pcre_callout block is NULL. There is a description of the pcre_extra | pcre_callout block is NULL. There is a description of the pcre_extra |
| 2894 | structure in the pcreapi documentation. | structure in the pcreapi documentation. |
| 2895 | ||
| 2896 | The pattern_position field is present from version 1 of the pcre_call- | The pattern_position field is present from version 1 of the pcre_call- |
| 2897 | out structure. It contains the offset to the next item to be matched in | out structure. It contains the offset to the next item to be matched in |
| 2898 | the pattern string. | the pattern string. |
| 2899 | ||
| 2900 | The next_item_length field is present from version 1 of the pcre_call- | The next_item_length field is present from version 1 of the pcre_call- |
| 2901 | out structure. It contains the length of the next item to be matched in | out structure. It contains the length of the next item to be matched in |
| 2902 | the pattern string. When the callout immediately precedes an alterna- | the pattern string. When the callout immediately precedes an alterna- |
| 2903 | tion bar, a closing parenthesis, or the end of the pattern, the length | tion bar, a closing parenthesis, or the end of the pattern, the length |
| 2904 | is zero. When the callout precedes an opening parenthesis, the length | is zero. When the callout precedes an opening parenthesis, the length |
| 2905 | is that of the entire subpattern. | is that of the entire subpattern. |
| 2906 | ||
| 2907 | The pattern_position and next_item_length fields are intended to help | The pattern_position and next_item_length fields are intended to help |
| 2908 | in distinguishing between different automatic callouts, which all have | in distinguishing between different automatic callouts, which all have |
| 2909 | the same callout number. However, they are set for all callouts. | the same callout number. However, they are set for all callouts. |
| 2910 | ||
| 2911 | ||
| 2912 | RETURN VALUES | RETURN VALUES |
| 2913 | ||
| 2914 | The external callout function returns an integer to PCRE. If the value | The external callout function returns an integer to PCRE. If the value |
| 2915 | is zero, matching proceeds as normal. If the value is greater than | is zero, matching proceeds as normal. If the value is greater than |
| 2916 | zero, matching fails at the current point, but the testing of other | zero, matching fails at the current point, but the testing of other |
| 2917 | matching possibilities goes ahead, just as if a lookahead assertion had | matching possibilities goes ahead, just as if a lookahead assertion had |
| 2918 | failed. If the value is less than zero, the match is abandoned, and | failed. If the value is less than zero, the match is abandoned, and |
| 2919 | pcre_exec() (or pcre_dfa_exec()) returns the negative value. | pcre_exec() or pcre_dfa_exec() returns the negative value. |
| 2920 | ||
| 2921 | Negative values should normally be chosen from the set of | Negative values should normally be chosen from the set of |
| 2922 | PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan- | PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan- |
| 2923 | dard "no match" failure. The error number PCRE_ERROR_CALLOUT is | dard "no match" failure. The error number PCRE_ERROR_CALLOUT is |
| 2924 | reserved for use by callout functions; it will never be used by PCRE | reserved for use by callout functions; it will never be used by PCRE |
| 2925 | itself. | itself. |
| 2926 | ||
| 2927 | ||
| # | Line 2820 AUTHOR | Line 2934 AUTHOR |
| 2934 | ||
| 2935 | REVISION | REVISION |
| 2936 | ||
| 2937 | Last updated: 15 March 2009 | Last updated: 29 September 2009 |
| 2938 | Copyright (c) 1997-2009 University of Cambridge. | Copyright (c) 1997-2009 University of Cambridge. |
| 2939 | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| 2940 | ||
| 2941 | ||
| 2942 | PCRECOMPAT(3) PCRECOMPAT(3) | PCRECOMPAT(3) PCRECOMPAT(3) |
| 2943 | ||
| 2944 | ||
| # | Line 2835 NAME | Line 2949 NAME |
| 2949 | DIFFERENCES BETWEEN PCRE AND PERL | DIFFERENCES BETWEEN PCRE AND PERL |
| 2950 | ||
| 2951 | This document describes the differences in the ways that PCRE and Perl | This document describes the differences in the ways that PCRE and Perl |
| 2952 | handle regular expressions. The differences described here are mainly | handle regular expressions. The differences described here are with |
| 2953 | with respect to Perl 5.8, though PCRE versions 7.0 and later contain | respect to Perl 5.10/5.11. |
| some features that are in Perl 5.10. | ||
| 2954 | ||
| 2955 | 1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details | 1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details |
| 2956 | of what it does have are given in the section on UTF-8 support in the | of what it does have are given in the section on UTF-8 support in the |
| 2957 | main pcre page. | main pcre page. |
| 2958 | ||
| 2959 | 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl | 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl |
| 2960 | permits them, but they do not mean what you might think. For example, | permits them, but they do not mean what you might think. For example, |
| 2961 | (?!a){3} does not assert that the next three characters are not "a". It | (?!a){3} does not assert that the next three characters are not "a". It |
| 2962 | just asserts that the next character is not "a" three times. | just asserts that the next character is not "a" three times. |
| 2963 | ||
| 2964 | 3. Capturing subpatterns that occur inside negative lookahead asser- | 3. Capturing subpatterns that occur inside negative lookahead asser- |
| 2965 | tions are counted, but their entries in the offsets vector are never | tions are counted, but their entries in the offsets vector are never |
| 2966 | set. Perl sets its numerical variables from any such patterns that are | set. Perl sets its numerical variables from any such patterns that are |
| 2967 | matched before the assertion fails to match something (thereby succeed- | matched before the assertion fails to match something (thereby succeed- |
| 2968 | ing), but only if the negative lookahead assertion contains just one | ing), but only if the negative lookahead assertion contains just one |
| 2969 | branch. | branch. |
| 2970 | ||
| 2971 | 4. Though binary zero characters are supported in the subject string, | 4. Though binary zero characters are supported in the subject string, |
| 2972 | they are not allowed in a pattern string because it is passed as a nor- | they are not allowed in a pattern string because it is passed as a nor- |
| 2973 | mal C string, terminated by zero. The escape sequence \0 can be used in | mal C string, terminated by zero. The escape sequence \0 can be used in |
| 2974 | the pattern to represent a binary zero. | the pattern to represent a binary zero. |
| 2975 | ||
| 2976 | 5. The following Perl escape sequences are not supported: \l, \u, \L, | 5. The following Perl escape sequences are not supported: \l, \u, \L, |
| 2977 | \U, and \N. In fact these are implemented by Perl's general string-han- | \U, and \N. In fact these are implemented by Perl's general string-han- |
| 2978 | dling and are not part of its pattern matching engine. If any of these | dling and are not part of its pattern matching engine. If any of these |
| 2979 | are encountered by PCRE, an error is generated. | are encountered by PCRE, an error is generated. |
| 2980 | ||
| 2981 | 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE | 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE |
| 2982 | is built with Unicode character property support. The properties that | is built with Unicode character property support. The properties that |
| 2983 | can be tested with \p and \P are limited to the general category prop- | can be tested with \p and \P are limited to the general category prop- |
| 2984 | erties such as Lu and Nd, script names such as Greek or Han, and the | erties such as Lu and Nd, script names such as Greek or Han, and the |
| 2985 | derived properties Any and L&. | derived properties Any and L&. PCRE does support the Cs (surrogate) |
| 2986 | property, which Perl does not; the Perl documentation says "Because | |
| 2987 | Perl hides the need for the user to understand the internal representa- | |
| 2988 | tion of Unicode characters, there is no need to implement the somewhat | |
| 2989 | messy concept of surrogates." | |
| 2990 | ||
| 2991 | 7. PCRE does support the \Q...\E escape for quoting substrings. Charac- | 7. PCRE does support the \Q...\E escape for quoting substrings. Charac- |
| 2992 | ters in between are treated as literals. This is slightly different | ters in between are treated as literals. This is slightly different |
| 2993 | from Perl in that $ and @ are also handled as literals inside the | from Perl in that $ and @ are also handled as literals inside the |
| 2994 | quotes. In Perl, they cause variable interpolation (but of course PCRE | quotes. In Perl, they cause variable interpolation (but of course PCRE |
| 2995 | does not have variables). Note the following examples: | does not have variables). Note the following examples: |
| 2996 | ||
| 2997 | Pattern PCRE matches Perl matches | Pattern PCRE matches Perl matches |
| # | Line 2884 DIFFERENCES BETWEEN PCRE AND PERL | Line 3001 DIFFERENCES BETWEEN PCRE AND PERL |
| 3001 | \Qabc\$xyz\E abc\$xyz abc\$xyz | \Qabc\$xyz\E abc\$xyz abc\$xyz |
| 3002 | \Qabc\E\$\Qxyz\E abc$xyz abc$xyz | \Qabc\E\$\Qxyz\E abc$xyz abc$xyz |
| 3003 | ||
| 3004 | The \Q...\E sequence is recognized both inside and outside character | The \Q...\E sequence is recognized both inside and outside character |
| 3005 | classes. | classes. |
| 3006 | ||
| 3007 | 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code}) | 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code}) |
| 3008 | constructions. However, there is support for recursive patterns. This | constructions. However, there is support for recursive patterns. This |
| 3009 | is not available in Perl 5.8, but will be in Perl 5.10. Also, the PCRE | is not available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE |
| 3010 | "callout" feature allows an external function to be called during pat- | "callout" feature allows an external function to be called during pat- |
| 3011 | tern matching. See the pcrecallout documentation for details. | tern matching. See the pcrecallout documentation for details. |
| 3012 | ||
| 3013 | 9. Subpatterns that are called recursively or as "subroutines" are | 9. Subpatterns that are called recursively or as "subroutines" are |
| 3014 | always treated as atomic groups in PCRE. This is like Python, but | always treated as atomic groups in PCRE. This is like Python, but |
| 3015 | unlike Perl. | unlike Perl. There is a discussion of an example that explains this in |
| 3016 | more detail in the section on recursion differences from Perl in the | |
| 3017 | 10. There are some differences that are concerned with the settings of | pcrepattern page. |
| 3018 | captured strings when part of a pattern is repeated. For example, | |
| 3019 | matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2 | 10. There are some differences that are concerned with the settings of |
| 3020 | captured strings when part of a pattern is repeated. For example, | |
| 3021 | matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2 | |
| 3022 | unset, but in PCRE it is set to "b". | unset, but in PCRE it is set to "b". |
| 3023 | ||
| 3024 | 11. PCRE does support Perl 5.10's backtracking verbs (*ACCEPT), | 11. PCRE's handling of duplicate subpattern numbers and duplicate sub- |
| 3025 | (*FAIL), (*F), (*COMMIT), (*PRUNE), (*SKIP), and (*THEN), but only in | pattern names is not as general as Perl's. This is a consequence of the |
| 3026 | the forms without an argument. PCRE does not support (*MARK). If | fact the PCRE works internally just with numbers, using an external ta- |
| 3027 | (*ACCEPT) is within capturing parentheses, PCRE does not set that cap- | ble to translate between numbers and names. In particular, a pattern |
| 3028 | ture group; this is different to Perl. | such as (?|(?<a>A)|(?<b)B), where the two capturing parentheses have |
| 3029 | the same number but different names, is not supported, and causes an | |
| 3030 | error at compile time. If it were allowed, it would not be possible to | |
| 3031 | distinguish which parentheses matched, because both names map to cap- | |
| 3032 | turing subpattern number 1. To avoid this confusing situation, an error | |
| 3033 | is given at compile time. | |
| 3034 | ||
| 3035 | 12. PCRE provides some extensions to the Perl regular expression facil- | 12. PCRE provides some extensions to the Perl regular expression facil- |
| 3036 | ities. Perl 5.10 will include new features that are not in earlier | ities. Perl 5.10 includes new features that are not in earlier ver- |
| 3037 | versions, some of which (such as named parentheses) have been in PCRE | sions of Perl, some of which (such as named parentheses) have been in |
| 3038 | for some time. This list is with respect to Perl 5.10: | PCRE for some time. This list is with respect to Perl 5.10: |
| 3039 | ||
| 3040 | (a) Although lookbehind assertions must match fixed length strings, | (a) Although lookbehind assertions in PCRE must match fixed length |
| 3041 | each alternative branch of a lookbehind assertion can match a different | strings, each alternative branch of a lookbehind assertion can match a |
| 3042 | length of string. Perl requires them all to have the same length. | different length of string. Perl requires them all to have the same |
| 3043 | length. | |
| 3044 | ||
| 3045 | (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ | (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ |
| 3046 | meta-character matches only at the very end of the string. | meta-character matches only at the very end of the string. |
| 3047 | ||
| 3048 | (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe- | (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe- |
| 3049 | cial meaning is faulted. Otherwise, like Perl, the backslash is quietly | cial meaning is faulted. Otherwise, like Perl, the backslash is quietly |
| 3050 | ignored. (Perl can be made to issue a warning.) | ignored. (Perl can be made to issue a warning.) |
| 3051 | ||
| 3052 | (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti- | (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti- |
| 3053 | fiers is inverted, that is, by default they are not greedy, but if fol- | fiers is inverted, that is, by default they are not greedy, but if fol- |
| 3054 | lowed by a question mark they are. | lowed by a question mark they are. |
| 3055 | ||
| 3056 | (e) PCRE_ANCHORED can be used at matching time to force a pattern to be | (e) PCRE_ANCHORED can be used at matching time to force a pattern to be |
| 3057 | tried only at the first matching position in the subject string. | tried only at the first matching position in the subject string. |
| 3058 | ||
| 3059 | (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and PCRE_NO_AUTO_CAP- | (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, |
| 3060 | TURE options for pcre_exec() have no Perl equivalents. | and PCRE_NO_AUTO_CAPTURE options for pcre_exec() have no Perl equiva- |
| 3061 | lents. | |
| 3062 | ||
| 3063 | (g) The \R escape sequence can be restricted to match only CR, LF, or | (g) The \R escape sequence can be restricted to match only CR, LF, or |
| 3064 | CRLF by the PCRE_BSR_ANYCRLF option. | CRLF by the PCRE_BSR_ANYCRLF option. |
| # | Line 2961 AUTHOR | Line 3087 AUTHOR |
| 3087 | ||
| 3088 | REVISION | REVISION |
| 3089 | ||
| 3090 | Last updated: 25 August 2009 | Last updated: 12 May 2010 |
| 3091 | Copyright (c) 1997-2009 University of Cambridge. | Copyright (c) 1997-2010 University of Cambridge. |
| 3092 | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| 3093 | ||
| 3094 | ||
| 3095 | PCREPATTERN(3) PCREPATTERN(3) | PCREPATTERN(3) PCREPATTERN(3) |
| 3096 | ||
| 3097 | ||
| # | Line 2992 PCRE REGULAR EXPRESSION DETAILS | Line 3118 PCRE REGULAR EXPRESSION DETAILS |
| 3118 | ||
| 3119 | The original operation of PCRE was on strings of one-byte characters. | The original operation of PCRE was on strings of one-byte characters. |
| 3120 | However, there is now also support for UTF-8 character strings. To use | However, there is now also support for UTF-8 character strings. To use |
| 3121 | this, you must build PCRE to include UTF-8 support, and then call | this, PCRE must be built to include UTF-8 support, and you must call |
| 3122 | pcre_compile() with the PCRE_UTF8 option. There is also a special | pcre_compile() or pcre_compile2() with the PCRE_UTF8 option. There is |
| 3123 | sequence that can be given at the start of a pattern: | also a special sequence that can be given at the start of a pattern: |
| 3124 | ||
| 3125 | (*UTF8) | (*UTF8) |
| 3126 | ||
| # | Line 3004 PCRE REGULAR EXPRESSION DETAILS | Line 3130 PCRE REGULAR EXPRESSION DETAILS |
| 3130 | below. There is also a summary of UTF-8 features in the section on | below. There is also a summary of UTF-8 features in the section on |
| 3131 | UTF-8 support in the main pcre page. | UTF-8 support in the main pcre page. |
| 3132 | ||
| 3133 | Another special sequence that may appear at the start of a pattern or | |
| 3134 | in combination with (*UTF8) is: | |
| 3135 | ||
| 3136 | (*UCP) | |
| 3137 | ||
| 3138 | This has the same effect as setting the PCRE_UCP option: it causes | |
| 3139 | sequences such as \d and \w to use Unicode properties to determine | |
| 3140 | character types, instead of recognizing only characters with codes less | |
| 3141 | than 128 via a lookup table. | |
| 3142 | ||
| 3143 | The remainder of this document discusses the patterns that are sup- | The remainder of this document discusses the patterns that are sup- |
| 3144 | ported by PCRE when its main matching function, pcre_exec(), is used. | ported by PCRE when its main matching function, pcre_exec(), is used. |
| 3145 | From release 6.0, PCRE offers a second matching function, | From release 6.0, PCRE offers a second matching function, |
| # | Line 3032 NEWLINE CONVENTIONS | Line 3168 NEWLINE CONVENTIONS |
| 3168 | (*ANYCRLF) any of the three above | (*ANYCRLF) any of the three above |
| 3169 | (*ANY) all Unicode newline sequences | (*ANY) all Unicode newline sequences |
| 3170 | ||
| 3171 | These override the default and the options given to pcre_compile(). For | These override the default and the options given to pcre_compile() or |
| 3172 | example, on a Unix system where LF is the default newline sequence, the | pcre_compile2(). For example, on a Unix system where LF is the default |
| 3173 | pattern | newline sequence, the pattern |
| 3174 | ||
| 3175 | (*CR)a.b | (*CR)a.b |
| 3176 | ||
| # | Line 3044 NEWLINE CONVENTIONS | Line 3180 NEWLINE CONVENTIONS |
| 3180 | and that they must be in upper case. If more than one of them is | and that they must be in upper case. If more than one of them is |
| 3181 | present, the last one is used. | present, the last one is used. |
| 3182 | ||
| 3183 | The newline convention does not affect what the \R escape sequence | The newline convention affects the interpretation of the dot metachar- |
| 3184 | matches. By default, this is any Unicode newline sequence, for Perl | acter when PCRE_DOTALL is not set, and also the behaviour of \N. How- |
| 3185 | compatibility. However, this can be changed; see the description of \R | ever, it does not affect what the \R escape sequence matches. By |
| 3186 | in the section entitled "Newline sequences" below. A change of \R set- | default, this is any Unicode newline sequence, for Perl compatibility. |
| 3187 | ting can be combined with a change of newline convention. | However, this can be changed; see the description of \R in the section |
| 3188 | entitled "Newline sequences" below. A change of \R setting can be com- | |
| 3189 | bined with a change of newline convention. | |
| 3190 | ||
| 3191 | ||
| 3192 | CHARACTERS AND METACHARACTERS | CHARACTERS AND METACHARACTERS |
| # | Line 3151 BACKSLASH | Line 3289 BACKSLASH |
| 3289 | acters in patterns in a visible manner. There is no restriction on the | acters in patterns in a visible manner. There is no restriction on the |
| 3290 | appearance of non-printing characters, apart from the binary zero that | appearance of non-printing characters, apart from the binary zero that |
| 3291 | terminates a pattern, but when a pattern is being prepared by text | terminates a pattern, but when a pattern is being prepared by text |
| 3292 | editing, it is usually easier to use one of the following escape | editing, it is often easier to use one of the following escape |
| 3293 | sequences than the binary character it represents: | sequences than the binary character it represents: |
| 3294 | ||
| 3295 | \a alarm, that is, the BEL character (hex 07) | \a alarm, that is, the BEL character (hex 07) |
| # | Line 3161 BACKSLASH | Line 3299 BACKSLASH |
| 3299 | \n linefeed (hex 0A) | \n linefeed (hex 0A) |
| 3300 | \r carriage return (hex 0D) | \r carriage return (hex 0D) |
| 3301 | \t tab (hex 09) | \t tab (hex 09) |
| 3302 | \ddd character with octal code ddd, or backreference | \ddd character with octal code ddd, or back reference |
| 3303 | \xhh character with hex code hh | \xhh character with hex code hh |
| 3304 | \x{hhh..} character with hex code hhh.. | \x{hhh..} character with hex code hhh.. |
| 3305 | ||
| # | Line 3230 BACKSLASH | Line 3368 BACKSLASH |
| 3368 | All the sequences that define a single character value can be used both | All the sequences that define a single character value can be used both |
| 3369 | inside and outside character classes. In addition, inside a character | inside and outside character classes. In addition, inside a character |
| 3370 | class, the sequence \b is interpreted as the backspace character (hex | class, the sequence \b is interpreted as the backspace character (hex |
| 3371 | 08), and the sequences \R and \X are interpreted as the characters "R" | 08). The sequences \B, \N, \R, and \X are not special inside a charac- |
| 3372 | and "X", respectively. Outside a character class, these sequences have | ter class. Like any other unrecognized escape sequences, they are |
| 3373 | different meanings (see below). | treated as the literal characters "B", "N", "R", and "X" by default, |
| 3374 | but cause an error if the PCRE_EXTRA option is set. Outside a character | |
| 3375 | class, these sequences have different meanings. | |
| 3376 | ||
| 3377 | Absolute and relative back references | Absolute and relative back references |
| 3378 | ||
| # | Line 3252 BACKSLASH | Line 3392 BACKSLASH |
| 3392 | ||
| 3393 | Generic character types | Generic character types |
| 3394 | ||
| 3395 | Another use of backslash is for specifying generic character types. The | Another use of backslash is for specifying generic character types: |
| following are always recognized: | ||
| 3396 | ||
| 3397 | \d any decimal digit | \d any decimal digit |
| 3398 | \D any character that is not a decimal digit | \D any character that is not a decimal digit |
| # | Line 3266 BACKSLASH | Line 3405 BACKSLASH |
| 3405 | \w any "word" character | \w any "word" character |
| 3406 | \W any "non-word" character | \W any "non-word" character |
| 3407 | ||
| 3408 | Each pair of escape sequences partitions the complete set of characters | There is also the single sequence \N, which matches a non-newline char- |
| 3409 | into two disjoint sets. Any given character matches one, and only one, | acter. This is the same as the "." metacharacter when PCRE_DOTALL is |
| 3410 | of each pair. | not set. |
| 3411 | ||
| 3412 | These character type sequences can appear both inside and outside char- | Each pair of lower and upper case escape sequences partitions the com- |
| 3413 | acter classes. They each match one character of the appropriate type. | plete set of characters into two disjoint sets. Any given character |
| 3414 | If the current matching point is at the end of the subject string, all | matches one, and only one, of each pair. The sequences can appear both |
| 3415 | of them fail, since there is no character to match. | inside and outside character classes. They each match one character of |
| 3416 | the appropriate type. If the current matching point is at the end of | |
| 3417 | the subject string, all of them fail, because there is no character to | |
| 3418 | match. | |
| 3419 | ||
| 3420 | For compatibility with Perl, \s does not match the VT character (code | For compatibility with Perl, \s does not match the VT character (code |
| 3421 | 11). This makes it different from the the POSIX "space" class. The \s | 11). This makes it different from the the POSIX "space" class. The \s |
| # | Line 3281 BACKSLASH | Line 3423 BACKSLASH |
| 3423 | "use locale;" is included in a Perl script, \s may match the VT charac- | "use locale;" is included in a Perl script, \s may match the VT charac- |
| 3424 | ter. In PCRE, it never does. | ter. In PCRE, it never does. |
| 3425 | ||
| 3426 | In UTF-8 mode, characters with values greater than 128 never match \d, | A "word" character is an underscore or any character that is a letter |
| 3427 | \s, or \w, and always match \D, \S, and \W. This is true even when Uni- | or digit. By default, the definition of letters and digits is con- |
| 3428 | code character property support is available. These sequences retain | trolled by PCRE's low-valued character tables, and may vary if locale- |
| 3429 | their original meanings from before UTF-8 support was available, mainly | specific matching is taking place (see "Locale support" in the pcreapi |
| 3430 | for efficiency reasons. Note that this also affects \b, because it is | page). For example, in a French locale such as "fr_FR" in Unix-like |
| 3431 | defined in terms of \w and \W. | systems, or "french" in Windows, some character codes greater than 128 |
| 3432 | are used for accented letters, and these are then matched by \w. The | |
| 3433 | use of locales with Unicode is discouraged. | |
| 3434 | ||
| 3435 | By default, in UTF-8 mode, characters with values greater than 128 | |
| 3436 | never match \d, \s, or \w, and always match \D, \S, and \W. These | |
| 3437 | sequences retain their original meanings from before UTF-8 support was | |
| 3438 | available, mainly for efficiency reasons. However, if PCRE is compiled | |
| 3439 | with Unicode property support, and the PCRE_UCP option is set, the be- | |
| 3440 | haviour is changed so that Unicode properties are used to determine | |
| 3441 | character types, as follows: | |
| 3442 | ||
| 3443 | \d any character that \p{Nd} matches (decimal digit) | |
| 3444 | \s any character that \p{Z} matches, plus HT, LF, FF, CR | |
| 3445 | \w any character that \p{L} or \p{N} matches, plus underscore | |
| 3446 | ||
| 3447 | The upper case escapes match the inverse sets of characters. Note that | |
| 3448 | \d matches only decimal digits, whereas \w matches any Unicode digit, | |
| 3449 | as well as any Unicode letter, and underscore. Note also that PCRE_UCP | |
| 3450 | affects \b, and \B because they are defined in terms of \w and \W. | |
| 3451 | Matching these sequences is noticeably slower when PCRE_UCP is set. | |
| 3452 | ||
| 3453 | The sequences \h, \H, \v, and \V are Perl 5.10 features. In contrast to | The sequences \h, \H, \v, and \V are Perl 5.10 features. In contrast to |
| 3454 | the other sequences, these do match certain high-valued codepoints in | the other sequences, which match only ASCII characters by default, |
| 3455 | UTF-8 mode. The horizontal space characters are: | these always match certain high-valued codepoints in UTF-8 mode, |
| 3456 | whether or not PCRE_UCP is set. The horizontal space characters are: | |
| 3457 | ||
| 3458 | U+0009 Horizontal tab | U+0009 Horizontal tab |
| 3459 | U+0020 Space | U+0020 Space |
| # | Line 3322 BACKSLASH | Line 3485 BACKSLASH |
| 3485 | U+2028 Line separator | U+2028 Line separator |
| 3486 | U+2029 Paragraph separator | U+2029 Paragraph separator |
| 3487 | ||
| A "word" character is an underscore or any character less than 256 that | ||
| is a letter or digit. The definition of letters and digits is con- | ||
| trolled by PCRE's low-valued character tables, and may vary if locale- | ||
| specific matching is taking place (see "Locale support" in the pcreapi | ||
| page). For example, in a French locale such as "fr_FR" in Unix-like | ||
| systems, or "french" in Windows, some character codes greater than 128 | ||
| are used for accented letters, and these are matched by \w. The use of | ||
| locales with Unicode is discouraged. | ||
| 3488 | Newline sequences | Newline sequences |
| 3489 | ||
| 3490 | Outside a character class, by default, the escape sequence \R matches | Outside a character class, by default, the escape sequence \R matches |
| # | Line 3363 BACKSLASH | Line 3517 BACKSLASH |
| 3517 | (*BSR_ANYCRLF) CR, LF, or CRLF only | (*BSR_ANYCRLF) CR, LF, or CRLF only |
| 3518 | (*BSR_UNICODE) any Unicode newline sequence | (*BSR_UNICODE) any Unicode newline sequence |
| 3519 | ||
| 3520 | These override the default and the options given to pcre_compile(), but | These override the default and the options given to pcre_compile() or |
| 3521 | they can be overridden by options given to pcre_exec(). Note that these | pcre_compile2(), but they can be overridden by options given to |
| 3522 | special settings, which are not Perl-compatible, are recognized only at | pcre_exec() or pcre_dfa_exec(). Note that these special settings, which |
| 3523 | the very start of a pattern, and that they must be in upper case. If | are not Perl-compatible, are recognized only at the very start of a |
| 3524 | more than one of them is present, the last one is used. They can be | pattern, and that they must be in upper case. If more than one of them |
| 3525 | combined with a change of newline convention, for example, a pattern | is present, the last one is used. They can be combined with a change of |
| 3526 | can start with: | newline convention; for example, a pattern can start with: |
| 3527 | ||
| 3528 | (*ANY)(*BSR_ANYCRLF) | (*ANY)(*BSR_ANYCRLF) |
| 3529 | ||
| 3530 | Inside a character class, \R matches the letter "R". | They can also be combined with the (*UTF8) or (*UCP) special sequences. |
| 3531 | Inside a character class, \R is treated as an unrecognized escape | |
| 3532 | sequence, and so matches the letter "R" by default, but causes an error | |
| 3533 | if PCRE_EXTRA is set. | |
| 3534 | ||
| 3535 | Unicode character properties | Unicode character properties |
| 3536 | ||
| 3537 | When PCRE is built with Unicode character property support, three addi- | When PCRE is built with Unicode character property support, three addi- |
| 3538 | tional escape sequences that match characters with specific properties | tional escape sequences that match characters with specific properties |
| 3539 | are available. When not in UTF-8 mode, these sequences are of course | are available. When not in UTF-8 mode, these sequences are of course |
| 3540 | limited to testing characters whose codepoints are less than 256, but | limited to testing characters whose codepoints are less than 256, but |
| 3541 | they do work in this mode. The extra escape sequences are: | they do work in this mode. The extra escape sequences are: |
| 3542 | ||
| 3543 | \p{xx} a character with the xx property | \p{xx} a character with the xx property |
| 3544 | \P{xx} a character without the xx property | \P{xx} a character without the xx property |
| 3545 | \X an extended Unicode sequence | \X an extended Unicode sequence |
| 3546 | ||
| 3547 | The property names represented by xx above are limited to the Unicode | The property names represented by xx above are limited to the Unicode |
| 3548 | script names, the general category properties, and "Any", which matches | script names, the general category properties, "Any", which matches any |
| 3549 | any character (including newline). Other properties such as "InMusical- | character (including newline), and some special PCRE properties |
| 3550 | Symbols" are not currently supported by PCRE. Note that \P{Any} does | (described in the next section). Other Perl properties such as "InMu- |
| 3551 | not match any characters, so always causes a match failure. | sicalSymbols" are not currently supported by PCRE. Note that \P{Any} |
| 3552 | does not match any characters, so always causes a match failure. | |
| 3553 | ||
| 3554 | Sets of Unicode characters are defined as belonging to certain scripts. | Sets of Unicode characters are defined as belonging to certain scripts. |
| 3555 | A character from one of these sets can be matched using a script name. | A character from one of these sets can be matched using a script name. |
| # | Line 3403 BACKSLASH | Line 3561 BACKSLASH |
| 3561 | Those that are not part of an identified script are lumped together as | Those that are not part of an identified script are lumped together as |
| 3562 | "Common". The current list of scripts is: | "Common". The current list of scripts is: |
| 3563 | ||
| 3564 | Arabic, Armenian, Balinese, Bengali, Bopomofo, Braille, Buginese, | Arabic, Armenian, Avestan, Balinese, Bamum, Bengali, Bopomofo, Braille, |
| 3565 | Buhid, Canadian_Aboriginal, Cherokee, Common, Coptic, Cuneiform, | Buginese, Buhid, Canadian_Aboriginal, Carian, Cham, Cherokee, Common, |
| 3566 | Cypriot, Cyrillic, Deseret, Devanagari, Ethiopic, Georgian, Glagolitic, | Coptic, Cuneiform, Cypriot, Cyrillic, Deseret, Devanagari, Egyp- |
| 3567 | Gothic, Greek, Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hira- | tian_Hieroglyphs, Ethiopic, Georgian, Glagolitic, Gothic, Greek, |
| 3568 | gana, Inherited, Kannada, Katakana, Kharoshthi, Khmer, Lao, Latin, | Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hiragana, Impe- |
| 3569 | Limbu, Linear_B, Malayalam, Mongolian, Myanmar, New_Tai_Lue, Nko, | rial_Aramaic, Inherited, Inscriptional_Pahlavi, Inscriptional_Parthian, |
| 3570 | Ogham, Old_Italic, Old_Persian, Oriya, Osmanya, Phags_Pa, Phoenician, | Javanese, Kaithi, Kannada, Katakana, Kayah_Li, Kharoshthi, Khmer, Lao, |
| 3571 | Runic, Shavian, Sinhala, Syloti_Nagri, Syriac, Tagalog, Tagbanwa, | Latin, Lepcha, Limbu, Linear_B, Lisu, Lycian, Lydian, Malayalam, |
| 3572 | Tai_Le, Tamil, Telugu, Thaana, Thai, Tibetan, Tifinagh, Ugaritic, Yi. | Meetei_Mayek, Mongolian, Myanmar, New_Tai_Lue, Nko, Ogham, Old_Italic, |
| 3573 | Old_Persian, Old_South_Arabian, Old_Turkic, Ol_Chiki, Oriya, Osmanya, | |
| 3574 | Each character has exactly one general category property, specified by | Phags_Pa, Phoenician, Rejang, Runic, Samaritan, Saurashtra, Shavian, |
| 3575 | a two-letter abbreviation. For compatibility with Perl, negation can be | Sinhala, Sundanese, Syloti_Nagri, Syriac, Tagalog, Tagbanwa, Tai_Le, |
| 3576 | specified by including a circumflex between the opening brace and the | Tai_Tham, Tai_Viet, Tamil, Telugu, Thaana, Thai, Tibetan, Tifinagh, |
| 3577 | property name. For example, \p{^Lu} is the same as \P{Lu}. | Ugaritic, Vai, Yi. |
| 3578 | ||
| 3579 | Each character has exactly one Unicode general category property, spec- | |
| 3580 | ified by a two-letter abbreviation. For compatibility with Perl, nega- | |
| 3581 | tion can be specified by including a circumflex between the opening | |
| 3582 | brace and the property name. For example, \p{^Lu} is the same as | |
| 3583 | \P{Lu}. | |
| 3584 | ||
| 3585 | If only one letter is specified with \p or \P, it includes all the gen- | If only one letter is specified with \p or \P, it includes all the gen- |
| 3586 | eral category properties that start with that letter. In this case, in | eral category properties that start with that letter. In this case, in |
| # | Line 3480 BACKSLASH | Line 3644 BACKSLASH |
| 3644 | U+D800 to U+DFFF. Such characters are not valid in UTF-8 strings (see | U+D800 to U+DFFF. Such characters are not valid in UTF-8 strings (see |
| 3645 | RFC 3629) and so cannot be tested by PCRE, unless UTF-8 validity check- | RFC 3629) and so cannot be tested by PCRE, unless UTF-8 validity check- |
| 3646 | ing has been turned off (see the discussion of PCRE_NO_UTF8_CHECK in | ing has been turned off (see the discussion of PCRE_NO_UTF8_CHECK in |
| 3647 | the pcreapi page). | the pcreapi page). Perl does not support the Cs property. |
| 3648 | ||
| 3649 | The long synonyms for these properties that Perl supports (such as | The long synonyms for property names that Perl supports (such as |
| 3650 | \p{Letter}) are not supported by PCRE, nor is it permitted to prefix | \p{Letter}) are not supported by PCRE, nor is it permitted to prefix |
| 3651 | any of these properties with "Is". | any of these properties with "Is". |
| 3652 | ||
| # | Line 3508 BACKSLASH | Line 3672 BACKSLASH |
| 3672 | Matching characters by Unicode property is not fast, because PCRE has | Matching characters by Unicode property is not fast, because PCRE has |
| 3673 | to search a structure that contains data for over fifteen thousand | to search a structure that contains data for over fifteen thousand |
| 3674 | characters. That is why the traditional escape sequences such as \d and | characters. That is why the traditional escape sequences such as \d and |
| 3675 | \w do not use Unicode properties in PCRE. | \w do not use Unicode properties in PCRE by default, though you can |
| 3676 | make them do so by setting the PCRE_UCP option for pcre_compile() or by | |
| 3677 | starting the pattern with (*UCP). | |
| 3678 | ||
| 3679 | PCRE's additional properties | |
| 3680 | ||
| 3681 | As well as the standard Unicode properties described in the previous | |
| 3682 | section, PCRE supports four more that make it possible to convert tra- | |
| 3683 | ditional escape sequences such as \w and \s and POSIX character classes | |
| 3684 | to use Unicode properties. PCRE uses these non-standard, non-Perl prop- | |
| 3685 | erties internally when PCRE_UCP is set. They are: | |
| 3686 | ||
| 3687 | Xan Any alphanumeric character | |
| 3688 | Xps Any POSIX space character | |
| 3689 | Xsp Any Perl space character | |
| 3690 | Xwd Any Perl "word" character | |
| 3691 | ||
| 3692 | Xan matches characters that have either the L (letter) or the N (num- | |
| 3693 | ber) property. Xps matches the characters tab, linefeed, vertical tab, | |
| 3694 | formfeed, or carriage return, and any other character that has the Z | |
| 3695 | (separator) property. Xsp is the same as Xps, except that vertical tab | |
| 3696 | is excluded. Xwd matches the same characters as Xan, plus underscore. | |
| 3697 | ||
| 3698 | Resetting the match start | Resetting the match start |
| 3699 | ||
| # | Line 3529 BACKSLASH | Line 3714 BACKSLASH |
| 3714 | ||
| 3715 | matches "foobar", the first substring is still set to "foo". | matches "foobar", the first substring is still set to "foo". |
| 3716 | ||
| 3717 | Perl documents that the use of \K within assertions is "not well | |
| 3718 | defined". In PCRE, \K is acted upon when it occurs inside positive | |
| 3719 | assertions, but is ignored in negative assertions. | |
| 3720 | ||
| 3721 | Simple assertions | Simple assertions |
| 3722 | ||
| 3723 | The final use of backslash is for certain simple assertions. An asser- | The final use of backslash is for certain simple assertions. An asser- |
| # | Line 3545 BACKSLASH | Line 3734 BACKSLASH |
| 3734 | \z matches only at the end of the subject | \z matches only at the end of the subject |
| 3735 | \G matches at the first matching position in the subject | \G matches at the first matching position in the subject |
| 3736 | ||
| 3737 | These assertions may not appear in character classes (but note that \b | Inside a character class, \b has a different meaning; it matches the |
| 3738 | has a different meaning, namely the backspace character, inside a char- | backspace character. If any other of these assertions appears in a |
| 3739 | acter class). | character class, by default it matches the corresponding literal char- |
| 3740 | acter (for example, \B matches the letter B). However, if the | |
| 3741 | A word boundary is a position in the subject string where the current | PCRE_EXTRA option is set, an "invalid escape sequence" error is gener- |
| 3742 | character and the previous character do not both match \w or \W (i.e. | ated instead. |
| 3743 | one matches \w and the other matches \W), or the start or end of the | |
| 3744 | string if the first or last character matches \w, respectively. | A word boundary is a position in the subject string where the current |
| 3745 | character and the previous character do not both match \w or \W (i.e. | |
| 3746 | one matches \w and the other matches \W), or the start or end of the | |
| 3747 | string if the first or last character matches \w, respectively. In | |
| 3748 | UTF-8 mode, the meanings of \w and \W can be changed by setting the | |
| 3749 | PCRE_UCP option. When this is done, it also affects \b and \B. Neither | |
| 3750 | PCRE nor Perl has a separate "start of word" or "end of word" metase- | |
| 3751 | quence. However, whatever follows \b normally determines which it is. | |
| 3752 | For example, the fragment \ba matches "a" at the start of a word. | |
| 3753 | ||
| 3754 | The \A, \Z, and \z assertions differ from the traditional circumflex | The \A, \Z, and \z assertions differ from the traditional circumflex |
| 3755 | and dollar (described in the next section) in that they only ever match | and dollar (described in the next section) in that they only ever match |
| # | Line 3635 CIRCUMFLEX AND DOLLAR | Line 3832 CIRCUMFLEX AND DOLLAR |
| 3832 | set. | set. |
| 3833 | ||
| 3834 | ||
| 3835 | FULL STOP (PERIOD, DOT) | FULL STOP (PERIOD, DOT) AND \N |
| 3836 | ||
| 3837 | Outside a character class, a dot in the pattern matches any one charac- | Outside a character class, a dot in the pattern matches any one charac- |
| 3838 | ter in the subject string except (by default) a character that signi- | ter in the subject string except (by default) a character that signi- |
| # | Line 3658 FULL STOP (PERIOD, DOT) | Line 3855 FULL STOP (PERIOD, DOT) |
| 3855 | flex and dollar, the only relationship being that they both involve | flex and dollar, the only relationship being that they both involve |
| 3856 | newlines. Dot has no special meaning in a character class. | newlines. Dot has no special meaning in a character class. |
| 3857 | ||
| 3858 | The escape sequence \N always behaves as a dot does when PCRE_DOTALL is | |
| 3859 | not set. In other words, it matches any one character except one that | |
| 3860 | signifies the end of a line. | |
| 3861 | ||
| 3862 | ||
| 3863 | MATCHING A SINGLE BYTE | MATCHING A SINGLE BYTE |
| 3864 | ||
| # | Line 3678 SQUARE BRACKETS AND CHARACTER CLASSES | Line 3879 SQUARE BRACKETS AND CHARACTER CLASSES |
| 3879 | ||
| 3880 | An opening square bracket introduces a character class, terminated by a | An opening square bracket introduces a character class, terminated by a |
| 3881 | closing square bracket. A closing square bracket on its own is not spe- | closing square bracket. A closing square bracket on its own is not spe- |
| 3882 | cial. If a closing square bracket is required as a member of the class, | cial by default. However, if the PCRE_JAVASCRIPT_COMPAT option is set, |
| 3883 | it should be the first data character in the class (after an initial | a lone closing square bracket causes a compile-time error. If a closing |
| 3884 | circumflex, if present) or escaped with a backslash. | square bracket is required as a member of the class, it should be the |
| 3885 | first data character in the class (after an initial circumflex, if | |
| 3886 | present) or escaped with a backslash. | |
| 3887 | ||
| 3888 | A character class matches a single character in the subject. In UTF-8 | A character class matches a single character in the subject. In UTF-8 |
| 3889 | mode, the character may occupy more than one byte. A matched character | mode, the character may be more than one byte long. A matched character |
| 3890 | must be in the set of characters defined by the class, unless the first | must be in the set of characters defined by the class, unless the first |
| 3891 | character in the class definition is a circumflex, in which case the | character in the class definition is a circumflex, in which case the |
| 3892 | subject character must not be in the set defined by the class. If a | subject character must not be in the set defined by the class. If a |
| # | Line 3694 SQUARE BRACKETS AND CHARACTER CLASSES | Line 3897 SQUARE BRACKETS AND CHARACTER CLASSES |
| 3897 | while [^aeiou] matches any character that is not a lower case vowel. | while [^aeiou] matches any character that is not a lower case vowel. |
| 3898 | Note that a circumflex is just a convenient notation for specifying the | Note that a circumflex is just a convenient notation for specifying the |
| 3899 | characters that are in the class by enumerating those that are not. A | characters that are in the class by enumerating those that are not. A |
| 3900 | class that starts with a circumflex is not an assertion: it still con- | class that starts with a circumflex is not an assertion; it still con- |
| 3901 | sumes a character from the subject string, and therefore it fails if | sumes a character from the subject string, and therefore it fails if |
| 3902 | the current pointer is at the end of the string. | the current pointer is at the end of the string. |
| 3903 | ||
| # | Line 3710 SQUARE BRACKETS AND CHARACTER CLASSES | Line 3913 SQUARE BRACKETS AND CHARACTER CLASSES |
| 3913 | than 128, so caseless matching is always possible. For characters with | than 128, so caseless matching is always possible. For characters with |
| 3914 | higher values, the concept of case is supported if PCRE is compiled | higher values, the concept of case is supported if PCRE is compiled |
| 3915 | with Unicode property support, but not otherwise. If you want to use | with Unicode property support, but not otherwise. If you want to use |
| 3916 | caseless matching for characters 128 and above, you must ensure that | caseless matching in UTF8-mode for characters 128 and above, you must |
| 3917 | PCRE is compiled with Unicode property support as well as with UTF-8 | ensure that PCRE is compiled with Unicode property support as well as |
| 3918 | support. | with UTF-8 support. |
| 3919 | ||
| 3920 | Characters that might indicate line breaks are never treated in any | Characters that might indicate line breaks are never treated in any |
| 3921 | special way when matching character classes, whatever line-ending | special way when matching character classes, whatever line-ending |
| # | Line 3749 SQUARE BRACKETS AND CHARACTER CLASSES | Line 3952 SQUARE BRACKETS AND CHARACTER CLASSES |
| 3952 | concept of case for characters with values greater than 128 only when | concept of case for characters with values greater than 128 only when |
| 3953 | it is compiled with Unicode property support. | it is compiled with Unicode property support. |
| 3954 | ||
| 3955 | The character types \d, \D, \p, \P, \s, \S, \w, and \W may also appear | The character types \d, \D, \h, \H, \p, \P, \s, \S, \v, \V, \w, and \W |
| 3956 | in a character class, and add the characters that they match to the | may also appear in a character class, and add the characters that they |
| 3957 | class. For example, [\dABCDEF] matches any hexadecimal digit. A circum- | match to the class. For example, [\dABCDEF] matches any hexadecimal |
| 3958 | flex can conveniently be used with the upper case character types to | digit. A circumflex can conveniently be used with the upper case char- |
| 3959 | specify a more restricted set of characters than the matching lower | acter types to specify a more restricted set of characters than the |
| 3960 | case type. For example, the class [^\W_] matches any letter or digit, | matching lower case type. For example, the class [^\W_] matches any |
| 3961 | but not underscore. | letter or digit, but not underscore. |
| 3962 | ||
| 3963 | The only metacharacters that are recognized in character classes are | The only metacharacters that are recognized in character classes are |
| 3964 | backslash, hyphen (only where it can be interpreted as specifying a | backslash, hyphen (only where it can be interpreted as specifying a |
| # | Line 3774 POSIX CHARACTER CLASSES | Line 3977 POSIX CHARACTER CLASSES |
| 3977 | [01[:alpha:]%] | [01[:alpha:]%] |
| 3978 | ||
| 3979 | matches "0", "1", any alphabetic character, or "%". The supported class | matches "0", "1", any alphabetic character, or "%". The supported class |
| 3980 | names are | names are: |
| 3981 | ||
| 3982 | alnum letters and digits | alnum letters and digits |
| 3983 | alpha letters | alpha letters |
| # | Line 3785 POSIX CHARACTER CLASSES | Line 3988 POSIX CHARACTER CLASSES |
| 3988 | graph printing characters, excluding space | graph printing characters, excluding space |
| 3989 | lower lower case letters | lower lower case letters |
| 3990 | print printing characters, including space | print printing characters, including space |
| 3991 | punct printing characters, excluding letters and digits | punct printing characters, excluding letters and digits and space |
| 3992 | space white space (not quite the same as \s) | space white space (not quite the same as \s) |
| 3993 | upper upper case letters | upper upper case letters |
| 3994 | word "word" characters (same as \w) | word "word" characters (same as \w) |
| # | Line 3806 POSIX CHARACTER CLASSES | Line 4009 POSIX CHARACTER CLASSES |
| 4009 | POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but | POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but |
| 4010 | these are not supported, and an error is given if they are encountered. | these are not supported, and an error is given if they are encountered. |
| 4011 | ||
| 4012 | In UTF-8 mode, characters with values greater than 128 do not match any | By default, in UTF-8 mode, characters with values greater than 128 do |
| 4013 | of the POSIX character classes. | not match any of the POSIX character classes. However, if the PCRE_UCP |
| 4014 | option is passed to pcre_compile(), some of the classes are changed so | |
| 4015 | that Unicode character properties are used. This is achieved by replac- | |
| 4016 | ing the POSIX classes by other sequences, as follows: | |
| 4017 | ||
| 4018 | [:alnum:] becomes \p{Xan} | |
| 4019 | [:alpha:] becomes \p{L} | |
| 4020 | [:blank:] becomes \h | |
| 4021 | [:digit:] becomes \p{Nd} | |
| 4022 | [:lower:] becomes \p{Ll} | |
| 4023 | [:space:] becomes \p{Xps} | |
| 4024 | [:upper:] becomes \p{Lu} | |
| 4025 | [:word:] becomes \p{Xwd} | |
| 4026 | ||
| 4027 | Negated versions, such as [:^alpha:] use \P instead of \p. The other | |
| 4028 | POSIX classes are unchanged, and match only characters with code points | |
| 4029 | less than 128. | |
| 4030 | ||
| 4031 | ||
| 4032 | VERTICAL BAR | VERTICAL BAR |
| 4033 | ||
| 4034 | Vertical bar characters are used to separate alternative patterns. For | Vertical bar characters are used to separate alternative patterns. For |
| 4035 | example, the pattern | example, the pattern |
| 4036 | ||
| 4037 | gilbert|sullivan | gilbert|sullivan |
| 4038 | ||
| 4039 | matches either "gilbert" or "sullivan". Any number of alternatives may | matches either "gilbert" or "sullivan". Any number of alternatives may |
| 4040 | appear, and an empty alternative is permitted (matching the empty | appear, and an empty alternative is permitted (matching the empty |
| 4041 | string). The matching process tries each alternative in turn, from left | string). The matching process tries each alternative in turn, from left |
| 4042 | to right, and the first one that succeeds is used. If the alternatives | to right, and the first one that succeeds is used. If the alternatives |
| 4043 | are within a subpattern (defined below), "succeeds" means matching the | are within a subpattern (defined below), "succeeds" means matching the |
| 4044 | rest of the main pattern as well as the alternative in the subpattern. | rest of the main pattern as well as the alternative in the subpattern. |
| 4045 | ||
| 4046 | ||
| 4047 | INTERNAL OPTION SETTING | INTERNAL OPTION SETTING |
| 4048 | ||
| 4049 | The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and | The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
| 4050 | PCRE_EXTENDED options (which are Perl-compatible) can be changed from | PCRE_EXTENDED options (which are Perl-compatible) can be changed from |
| 4051 | within the pattern by a sequence of Perl option letters enclosed | within the pattern by a sequence of Perl option letters enclosed |
| 4052 | between "(?" and ")". The option letters are | between "(?" and ")". The option letters are |
| 4053 | ||
| 4054 | i for PCRE_CASELESS | i for PCRE_CASELESS |
| # | Line 3839 INTERNAL OPTION SETTING | Line 4058 INTERNAL OPTION SETTING |
| 4058 | ||
| 4059 | For example, (?im) sets caseless, multiline matching. It is also possi- | For example, (?im) sets caseless, multiline matching. It is also possi- |
| 4060 | ble to unset these options by preceding the letter with a hyphen, and a | ble to unset these options by preceding the letter with a hyphen, and a |
| 4061 | combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- | combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- |
| 4062 | LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, | LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, |
| 4063 | is also permitted. If a letter appears both before and after the | is also permitted. If a letter appears both before and after the |
| 4064 | hyphen, the option is unset. | hyphen, the option is unset. |
| 4065 | ||
| 4066 | The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA | The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA |
| 4067 | can be changed in the same way as the Perl-compatible options by using | can be changed in the same way as the Perl-compatible options by using |
| 4068 | the characters J, U and X respectively. | the characters J, U and X respectively. |
| 4069 | ||
| 4070 | When one of these option changes occurs at top level (that is, not | When one of these option changes occurs at top level (that is, not |
| 4071 | inside subpattern parentheses), the change applies to the remainder of | inside subpattern parentheses), the change applies to the remainder of |
| 4072 | the pattern that follows. If the change is placed right at the start of | the pattern that follows. If the change is placed right at the start of |
| 4073 | a pattern, PCRE extracts it into the global options (and it will there- | a pattern, PCRE extracts it into the global options (and it will there- |
| 4074 | fore show up in data extracted by the pcre_fullinfo() function). | fore show up in data extracted by the pcre_fullinfo() function). |
| 4075 | ||
| 4076 | An option change within a subpattern (see below for a description of | An option change within a subpattern (see below for a description of |
| 4077 | subpatterns) affects only that part of the current pattern that follows | subpatterns) affects only that part of the current pattern that follows |
| 4078 | it, so | it, so |
| 4079 | ||
| 4080 | (a(?i)b)c | (a(?i)b)c |
| 4081 | ||
| 4082 | matches abc and aBc and no other strings (assuming PCRE_CASELESS is not | matches abc and aBc and no other strings (assuming PCRE_CASELESS is not |
| 4083 | used). By this means, options can be made to have different settings | used). By this means, options can be made to have different settings |
| 4084 | in different parts of the pattern. Any changes made in one alternative | in different parts of the pattern. Any changes made in one alternative |
| 4085 | do carry on into subsequent branches within the same subpattern. For | do carry on into subsequent branches within the same subpattern. For |
| 4086 | example, | example, |
| 4087 | ||
| 4088 | (a(?i)b|c) | (a(?i)b|c) |
| 4089 | ||
| 4090 | matches "ab", "aB", "c", and "C", even though when matching "C" the | matches "ab", "aB", "c", and "C", even though when matching "C" the |
| 4091 | first branch is abandoned before the option setting. This is because | first branch is abandoned before the option setting. This is because |
| 4092 | the effects of option settings happen at compile time. There would be | the effects of option settings happen at compile time. There would be |
| 4093 | some very weird behaviour otherwise. | some very weird behaviour otherwise. |
| 4094 | ||
| 4095 | Note: There are other PCRE-specific options that can be set by the | Note: There are other PCRE-specific options that can be set by the |
| 4096 | application when the compile or match functions are called. In some | application when the compile or match functions are called. In some |
| 4097 | cases the pattern can contain special leading sequences such as (*CRLF) | cases the pattern can contain special leading sequences such as (*CRLF) |
| 4098 | to override what the application has set or what has been defaulted. | to override what the application has set or what has been defaulted. |
| 4099 | Details are given in the section entitled "Newline sequences" above. | Details are given in the section entitled "Newline sequences" above. |
| 4100 | There is also the (*UTF8) leading sequence that can be used to set | There are also the (*UTF8) and (*UCP) leading sequences that can be |
| 4101 | UTF-8 mode; this is equivalent to setting the PCRE_UTF8 option. | used to set UTF-8 and Unicode property modes; they are equivalent to |
| 4102 | setting the PCRE_UTF8 and the PCRE_UCP options, respectively. | |
| 4103 | ||
| 4104 | ||
| 4105 | SUBPATTERNS | SUBPATTERNS |
| # | Line 3961 DUPLICATE SUBPATTERN NUMBERS | Line 4181 DUPLICATE SUBPATTERN NUMBERS |
| 4181 | / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x | / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x |
| 4182 | # 1 2 2 3 2 3 4 | # 1 2 2 3 2 3 4 |
| 4183 | ||
| 4184 | A backreference or a recursive call to a numbered subpattern always | A back reference to a numbered subpattern uses the most recent value |
| 4185 | refers to the first one in the pattern with the given number. | that is set for that number by any subpattern. The following pattern |
| 4186 | matches "abcabc" or "defdef": | |
| 4187 | ||
| 4188 | /(?|(abc)|(def))\1/ | |
| 4189 | ||
| 4190 | In contrast, a recursive or "subroutine" call to a numbered subpattern | |
| 4191 | always refers to the first one in the pattern with the given number. | |
| 4192 | The following pattern matches "abcabc" or "defabc": | |
| 4193 | ||
| 4194 | /(?|(abc)|(def))(?1)/ | |
| 4195 | ||
| 4196 | If a condition test for a subpattern's having matched refers to a non- | |
| 4197 | unique number, the test is true if any of the subpatterns of that num- | |
| 4198 | ber have matched. | |
| 4199 | ||
| 4200 | An alternative approach to using this "branch reset" feature is to use | An alternative approach to using this "branch reset" feature is to use |
| 4201 | duplicate named subpatterns, as described in the next section. | duplicate named subpatterns, as described in the next section. |
| 4202 | ||
| 4203 | ||
| 4204 | NAMED SUBPATTERNS | NAMED SUBPATTERNS |
| 4205 | ||
| 4206 | Identifying capturing parentheses by number is simple, but it can be | Identifying capturing parentheses by number is simple, but it can be |
| 4207 | very hard to keep track of the numbers in complicated regular expres- | very hard to keep track of the numbers in complicated regular expres- |
| 4208 | sions. Furthermore, if an expression is modified, the numbers may | sions. Furthermore, if an expression is modified, the numbers may |
| 4209 | change. To help with this difficulty, PCRE supports the naming of sub- | change. To help with this difficulty, PCRE supports the naming of sub- |
| 4210 | patterns. This feature was not added to Perl until release 5.10. Python | patterns. This feature was not added to Perl until release 5.10. Python |
| 4211 | had the feature earlier, and PCRE introduced it at release 4.0, using | had the feature earlier, and PCRE introduced it at release 4.0, using |
| 4212 | the Python syntax. PCRE now supports both the Perl and the Python syn- | the Python syntax. PCRE now supports both the Perl and the Python syn- |
| 4213 | tax. | tax. Perl allows identically numbered subpatterns to have different |
| 4214 | names, but PCRE does not. | |
| 4215 | ||
| 4216 | In PCRE, a subpattern can be named in one of three ways: (?<name>...) | In PCRE, a subpattern can be named in one of three ways: (?<name>...) |
| 4217 | or (?'name'...) as in Perl, or (?P<name>...) as in Python. References | or (?'name'...) as in Perl, or (?P<name>...) as in Python. References |
| 4218 | to capturing parentheses from other parts of the pattern, such as back- | to capturing parentheses from other parts of the pattern, such as back |
| 4219 | references, recursion, and conditions, can be made by name as well as | references, recursion, and conditions, can be made by name as well as |
| 4220 | by number. | by number. |
| 4221 | ||
| # | Line 3994 NAMED SUBPATTERNS | Line 4228 NAMED SUBPATTERNS |
| 4228 | ||
| 4229 | By default, a name must be unique within a pattern, but it is possible | By default, a name must be unique within a pattern, but it is possible |
| 4230 | to relax this constraint by setting the PCRE_DUPNAMES option at compile | to relax this constraint by setting the PCRE_DUPNAMES option at compile |
| 4231 | time. This can be useful for patterns where only one instance of the | time. (Duplicate names are also always permitted for subpatterns with |
| 4232 | the same number, set up as described in the previous section.) Dupli- | |
| 4233 | cate names can be useful for patterns where only one instance of the | |
| 4234 | named parentheses can match. Suppose you want to match the name of a | named parentheses can match. Suppose you want to match the name of a |
| 4235 | weekday, either as a 3-letter abbreviation or as the full name, and in | weekday, either as a 3-letter abbreviation or as the full name, and in |
| 4236 | both cases you want to extract the abbreviation. This pattern (ignoring | both cases you want to extract the abbreviation. This pattern (ignoring |
| # | Line 4013 NAMED SUBPATTERNS | Line 4249 NAMED SUBPATTERNS |
| 4249 | The convenience function for extracting the data by name returns the | The convenience function for extracting the data by name returns the |
| 4250 | substring for the first (and in this example, the only) subpattern of | substring for the first (and in this example, the only) subpattern of |
| 4251 | that name that matched. This saves searching to find which numbered | that name that matched. This saves searching to find which numbered |
| 4252 | subpattern it was. If you make a reference to a non-unique named sub- | subpattern it was. |
| 4253 | pattern from elsewhere in the pattern, the one that corresponds to the | |
| 4254 | lowest number is used. For further details of the interfaces for han- | If you make a back reference to a non-unique named subpattern from |
| 4255 | dling named subpatterns, see the pcreapi documentation. | elsewhere in the pattern, the one that corresponds to the first occur- |
| 4256 | rence of the name is used. In the absence of duplicate numbers (see the | |
| 4257 | previous section) this is the one with the lowest number. If you use a | |
| 4258 | named reference in a condition test (see the section about conditions | |
| 4259 | below), either to check whether a subpattern has matched, or to check | |
| 4260 | for recursion, all subpatterns with the same name are tested. If the | |
| 4261 | condition is true for any one of them, the overall condition is true. | |
| 4262 | This is the same behaviour as testing by number. For further details of | |
| 4263 | the interfaces for handling named subpatterns, see the pcreapi documen- | |
| 4264 | tation. | |
| 4265 | ||
| 4266 | Warning: You cannot use different names to distinguish between two sub- | Warning: You cannot use different names to distinguish between two sub- |
| 4267 | patterns with the same number (see the previous section) because PCRE | patterns with the same number because PCRE uses only the numbers when |
| 4268 | uses only the numbers when matching. | matching. For this reason, an error is given at compile time if differ- |
| 4269 | ent names are given to subpatterns with the same number. However, you | |
| 4270 | can give the same name to subpatterns with the same number, even when | |
| 4271 | PCRE_DUPNAMES is not set. | |
| 4272 | ||
| 4273 | ||
| 4274 | REPETITION | REPETITION |
| # | Line 4037 REPETITION | Line 4285 REPETITION |
| 4285 | a character class | a character class |
| 4286 | a back reference (see next section) | a back reference (see next section) |
| 4287 | a parenthesized subpattern (unless it is an assertion) | a parenthesized subpattern (unless it is an assertion) |
| 4288 | a recursive or "subroutine" call to a subpattern | |
| 4289 | ||
| 4290 | The general repetition quantifier specifies a minimum and maximum num- | The general repetition quantifier specifies a minimum and maximum num- |
| 4291 | ber of permitted matches, by giving the two numbers in curly brackets | ber of permitted matches, by giving the two numbers in curly brackets |
| # | Line 4151 REPETITION | Line 4400 REPETITION |
| 4400 | mization, or alternatively using ^ to indicate anchoring explicitly. | mization, or alternatively using ^ to indicate anchoring explicitly. |
| 4401 | ||
| 4402 | However, there is one situation where the optimization cannot be used. | However, there is one situation where the optimization cannot be used. |
| 4403 | When .* is inside capturing parentheses that are the subject of a | When .* is inside capturing parentheses that are the subject of a back |
| 4404 | backreference elsewhere in the pattern, a match at the start may fail | reference elsewhere in the pattern, a match at the start may fail where |
| 4405 | where a later one succeeds. Consider, for example: | a later one succeeds. Consider, for example: |
| 4406 | ||
| 4407 | (.*)abc\1 | (.*)abc\1 |
| 4408 | ||
| # | Line 4360 BACK REFERENCES | Line 4609 BACK REFERENCES |
| 4609 | ||
| 4610 | There may be more than one back reference to the same subpattern. If a | There may be more than one back reference to the same subpattern. If a |
| 4611 | subpattern has not actually been used in a particular match, any back | subpattern has not actually been used in a particular match, any back |
| 4612 | references to it always fail. For example, the pattern | references to it always fail by default. For example, the pattern |
| 4613 | ||
| 4614 | (a|(bc))\2 | (a|(bc))\2 |
| 4615 | ||
| 4616 | always fails if it starts to match "a" rather than "bc". Because there | always fails if it starts to match "a" rather than "bc". However, if |
| 4617 | may be many capturing parentheses in a pattern, all digits following | the PCRE_JAVASCRIPT_COMPAT option is set at compile time, a back refer- |
| 4618 | the backslash are taken as part of a potential back reference number. | ence to an unset value matches an empty string. |
| 4619 | If the pattern continues with a digit character, some delimiter must be | |
| 4620 | used to terminate the back reference. If the PCRE_EXTENDED option is | Because there may be many capturing parentheses in a pattern, all dig- |
| 4621 | set, this can be whitespace. Otherwise an empty comment (see "Com- | its following a backslash are taken as part of a potential back refer- |
| 4622 | ments" below) can be used. | ence number. If the pattern continues with a digit character, some |
| 4623 | delimiter must be used to terminate the back reference. If the | |
| 4624 | A back reference that occurs inside the parentheses to which it refers | PCRE_EXTENDED option is set, this can be whitespace. Otherwise, the \g{ |
| 4625 | fails when the subpattern is first used, so, for example, (a\1) never | syntax or an empty comment (see "Comments" below) can be used. |
| 4626 | matches. However, such references can be useful inside repeated sub- | |
| 4627 | Recursive back references | |
| 4628 | ||
| 4629 | A back reference that occurs inside the parentheses to which it refers | |
| 4630 | fails when the subpattern is first used, so, for example, (a\1) never | |
| 4631 | matches. However, such references can be useful inside repeated sub- | |
| 4632 | patterns. For example, the pattern | patterns. For example, the pattern |
| 4633 | ||
| 4634 | (a|b\1)+ | (a|b\1)+ |
| 4635 | ||
| 4636 | matches any number of "a"s and also "aba", "ababbaa" etc. At each iter- | matches any number of "a"s and also "aba", "ababbaa" etc. At each iter- |
| 4637 | ation of the subpattern, the back reference matches the character | ation of the subpattern, the back reference matches the character |
| 4638 | string corresponding to the previous iteration. In order for this to | string corresponding to the previous iteration. In order for this to |
| 4639 | work, the pattern must be such that the first iteration does not need | work, the pattern must be such that the first iteration does not need |
| 4640 | to match the back reference. This can be done using alternation, as in | to match the back reference. This can be done using alternation, as in |
| 4641 | the example above, or by a quantifier with a minimum of zero. | the example above, or by a quantifier with a minimum of zero. |
| 4642 | ||
| 4643 | Back references of this type cause the group that they reference to be | |
| 4644 | treated as an atomic group. Once the whole group has been matched, a | |
| 4645 | subsequent matching failure cannot cause backtracking into the middle | |
| 4646 | of the group. | |
| 4647 | ||
| 4648 | ||
| 4649 | ASSERTIONS | ASSERTIONS |
| 4650 | ||
| # | Line 4433 ASSERTIONS | Line 4692 ASSERTIONS |
| 4692 | If you want to force a matching failure at some point in a pattern, the | If you want to force a matching failure at some point in a pattern, the |
| 4693 | most convenient way to do it is with (?!) because an empty string | most convenient way to do it is with (?!) because an empty string |
| 4694 | always matches, so an assertion that requires there not to be an empty | always matches, so an assertion that requires there not to be an empty |
| 4695 | string must always fail. | string must always fail. The Perl 5.10 backtracking control verb |
| 4696 | (*FAIL) or (*F) is essentially a synonym for (?!). | |
| 4697 | ||
| 4698 | Lookbehind assertions | Lookbehind assertions |
| 4699 | ||
| 4700 | Lookbehind assertions start with (?<= for positive assertions and (?<! | Lookbehind assertions start with (?<= for positive assertions and (?<! |
| 4701 | for negative assertions. For example, | for negative assertions. For example, |
| 4702 | ||
| 4703 | (?<!foo)bar | (?<!foo)bar |
| 4704 | ||
| 4705 | does find an occurrence of "bar" that is not preceded by "foo". The | does find an occurrence of "bar" that is not preceded by "foo". The |
| 4706 | contents of a lookbehind assertion are restricted such that all the | contents of a lookbehind assertion are restricted such that all the |
| 4707 | strings it matches must have a fixed length. However, if there are sev- | strings it matches must have a fixed length. However, if there are sev- |
| 4708 | eral top-level alternatives, they do not all have to have the same | eral top-level alternatives, they do not all have to have the same |
| 4709 | fixed length. Thus | fixed length. Thus |
| 4710 | ||
| 4711 | (?<=bullock|donkey) | (?<=bullock|donkey) |
| # | Line 4454 ASSERTIONS | Line 4714 ASSERTIONS |
| 4714 | ||
| 4715 | (?<!dogs?|cats?) | (?<!dogs?|cats?) |
| 4716 | ||
| 4717 | causes an error at compile time. Branches that match different length | causes an error at compile time. Branches that match different length |
| 4718 | strings are permitted only at the top level of a lookbehind assertion. | strings are permitted only at the top level of a lookbehind assertion. |
| 4719 | This is an extension compared with Perl (at least for 5.8), which | This is an extension compared with Perl (5.8 and 5.10), which requires |
| 4720 | requires all branches to match the same length of string. An assertion | all branches to match the same length of string. An assertion such as |
| such as | ||
| 4721 | ||
| 4722 | (?<=ab(c|de)) | (?<=ab(c|de)) |
| 4723 | ||
| 4724 | is not permitted, because its single top-level branch can match two | is not permitted, because its single top-level branch can match two |
| 4725 | different lengths, but it is acceptable if rewritten to use two top- | different lengths, but it is acceptable to PCRE if rewritten to use two |
| 4726 | level branches: | top-level branches: |
| 4727 | ||
| 4728 | (?<=abc|abde) | (?<=abc|abde) |
| 4729 | ||
| 4730 | In some cases, the Perl 5.10 escape sequence \K (see above) can be used | In some cases, the Perl 5.10 escape sequence \K (see above) can be used |
| 4731 | instead of a lookbehind assertion; this is not restricted to a fixed- | instead of a lookbehind assertion to get round the fixed-length |
| 4732 | length. | restriction. |
| 4733 | ||
| 4734 | The implementation of lookbehind assertions is, for each alternative, | The implementation of lookbehind assertions is, for each alternative, |
| 4735 | to temporarily move the current position back by the fixed length and | to temporarily move the current position back by the fixed length and |
| # | Line 4482 ASSERTIONS | Line 4741 ASSERTIONS |
| 4741 | ble to calculate the length of the lookbehind. The \X and \R escapes, | ble to calculate the length of the lookbehind. The \X and \R escapes, |
| 4742 | which can match different numbers of bytes, are also not permitted. | which can match different numbers of bytes, are also not permitted. |
| 4743 | ||
| 4744 | "Subroutine" calls (see below) such as (?2) or (?&X) are permitted in | |
| 4745 | lookbehinds, as long as the subpattern matches a fixed-length string. | |
| 4746 | Recursion, however, is not supported. | |
| 4747 | ||
| 4748 | Possessive quantifiers can be used in conjunction with lookbehind | Possessive quantifiers can be used in conjunction with lookbehind |
| 4749 | assertions to specify efficient matching at the end of the subject | assertions to specify efficient matching of fixed-length strings at the |
| 4750 | string. Consider a simple pattern such as | end of subject strings. Consider a simple pattern such as |
| 4751 | ||
| 4752 | abcd$ | abcd$ |
| 4753 | ||
| # | Line 4547 CONDITIONAL SUBPATTERNS | Line 4810 CONDITIONAL SUBPATTERNS |
| 4810 | ||
| 4811 | It is possible to cause the matching process to obey a subpattern con- | It is possible to cause the matching process to obey a subpattern con- |
| 4812 | ditionally or to choose between two alternative subpatterns, depending | ditionally or to choose between two alternative subpatterns, depending |
| 4813 | on the result of an assertion, or whether a previous capturing subpat- | on the result of an assertion, or whether a specific capturing subpat- |
| 4814 | tern matched or not. The two possible forms of conditional subpattern | tern has already been matched. The two possible forms of conditional |
| 4815 | are | subpattern are: |
| 4816 | ||
| 4817 | (?(condition)yes-pattern) | (?(condition)yes-pattern) |
| 4818 | (?(condition)yes-pattern|no-pattern) | (?(condition)yes-pattern|no-pattern) |
| # | Line 4564 CONDITIONAL SUBPATTERNS | Line 4827 CONDITIONAL SUBPATTERNS |
| 4827 | Checking for a used subpattern by number | Checking for a used subpattern by number |
| 4828 | ||
| 4829 | If the text between the parentheses consists of a sequence of digits, | If the text between the parentheses consists of a sequence of digits, |
| 4830 | the condition is true if the capturing subpattern of that number has | the condition is true if a capturing subpattern of that number has pre- |
| 4831 | previously matched. An alternative notation is to precede the digits | viously matched. If there is more than one capturing subpattern with |
| 4832 | with a plus or minus sign. In this case, the subpattern number is rela- | the same number (see the earlier section about duplicate subpattern |
| 4833 | tive rather than absolute. The most recently opened parentheses can be | numbers), the condition is true if any of them have been set. An alter- |
| 4834 | referenced by (?(-1), the next most recent by (?(-2), and so on. In | native notation is to precede the digits with a plus or minus sign. In |
| 4835 | looping constructs it can also make sense to refer to subsequent groups | this case, the subpattern number is relative rather than absolute. The |
| 4836 | with constructs such as (?(+2). | most recently opened parentheses can be referenced by (?(-1), the next |
| 4837 | most recent by (?(-2), and so on. In looping constructs it can also | |
| 4838 | make sense to refer to subsequent groups with constructs such as | |
| 4839 | (?(+2). | |
| 4840 | ||
| 4841 | Consider the following pattern, which contains non-significant white | Consider the following pattern, which contains non-significant white |
| 4842 | space to make it more readable (assume the PCRE_EXTENDED option) and to | space to make it more readable (assume the PCRE_EXTENDED option) and to |
| 4843 | divide it into three parts for ease of discussion: | divide it into three parts for ease of discussion: |
| 4844 | ||
| 4845 | ( \( )? [^()]+ (?(1) \) ) | ( \( )? [^()]+ (?(1) \) ) |
| 4846 | ||
| 4847 | The first part matches an optional opening parenthesis, and if that | The first part matches an optional opening parenthesis, and if that |
| 4848 | character is present, sets it as the first captured substring. The sec- | character is present, sets it as the first captured substring. The sec- |
| 4849 | ond part matches one or more characters that are not parentheses. The | ond part matches one or more characters that are not parentheses. The |
| 4850 | third part is a conditional subpattern that tests whether the first set | third part is a conditional subpattern that tests whether the first set |
| 4851 | of parentheses matched or not. If they did, that is, if subject started | of parentheses matched or not. If they did, that is, if subject started |
| 4852 | with an opening parenthesis, the condition is true, and so the yes-pat- | with an opening parenthesis, the condition is true, and so the yes-pat- |
| 4853 | tern is executed and a closing parenthesis is required. Otherwise, | tern is executed and a closing parenthesis is required. Otherwise, |
| 4854 | since no-pattern is not present, the subpattern matches nothing. In | since no-pattern is not present, the subpattern matches nothing. In |
| 4855 | other words, this pattern matches a sequence of non-parentheses, | other words, this pattern matches a sequence of non-parentheses, |
| 4856 | optionally enclosed in parentheses. | optionally enclosed in parentheses. |
| 4857 | ||
| 4858 | If you were embedding this pattern in a larger one, you could use a | If you were embedding this pattern in a larger one, you could use a |
| 4859 | relative reference: | relative reference: |
| 4860 | ||
| 4861 | ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... | ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... |
| 4862 | ||
| 4863 | This makes the fragment independent of the parentheses in the larger | This makes the fragment independent of the parentheses in the larger |
| 4864 | pattern. | pattern. |
| 4865 | ||
| 4866 | Checking for a used subpattern by name | Checking for a used subpattern by name |
| 4867 | ||
| 4868 | Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a | Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a |
| 4869 | used subpattern by name. For compatibility with earlier versions of | used subpattern by name. For compatibility with earlier versions of |
| 4870 | PCRE, which had this facility before Perl, the syntax (?(name)...) is | PCRE, which had this facility before Perl, the syntax (?(name)...) is |
| 4871 | also recognized. However, there is a possible ambiguity with this syn- | also recognized. However, there is a possible ambiguity with this syn- |
| 4872 | tax, because subpattern names may consist entirely of digits. PCRE | tax, because subpattern names may consist entirely of digits. PCRE |
| 4873 | looks first for a named subpattern; if it cannot find one and the name | looks first for a named subpattern; if it cannot find one and the name |
| 4874 | consists entirely of digits, PCRE looks for a subpattern of that num- | consists entirely of digits, PCRE looks for a subpattern of that num- |
| 4875 | ber, which must be greater than zero. Using subpattern names that con- | ber, which must be greater than zero. Using subpattern names that con- |
| 4876 | sist entirely of digits is not recommended. | sist entirely of digits is not recommended. |
| 4877 | ||
| 4878 | Rewriting the above example to use a named subpattern gives this: | Rewriting the above example to use a named subpattern gives this: |
| 4879 | ||
| 4880 | (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) | (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) |
| 4881 | ||
| 4882 | If the name used in a condition of this kind is a duplicate, the test | |
| 4883 | is applied to all subpatterns of the same name, and is true if any one | |
| 4884 | of them has matched. | |
| 4885 | ||
| 4886 | Checking for pattern recursion | Checking for pattern recursion |
| 4887 | ||
| 4888 | If the condition is the string (R), and there is no subpattern with the | If the condition is the string (R), and there is no subpattern with the |
| 4889 | name R, the condition is true if a recursive call to the whole pattern | name R, the condition is true if a recursive call to the whole pattern |
| 4890 | or any subpattern has been made. If digits or a name preceded by amper- | or any subpattern has been made. If digits or a name preceded by amper- |
| 4891 | sand follow the letter R, for example: | sand follow the letter R, for example: |
| 4892 | ||
| 4893 | (?(R3)...) or (?(R&name)...) | (?(R3)...) or (?(R&name)...) |
| 4894 | ||
| 4895 | the condition is true if the most recent recursion is into the subpat- | the condition is true if the most recent recursion is into a subpattern |
| 4896 | tern whose number or name is given. This condition does not check the | whose number or name is given. This condition does not check the entire |
| 4897 | entire recursion stack. | recursion stack. If the name used in a condition of this kind is a |
| 4898 | duplicate, the test is applied to all subpatterns of the same name, and | |
| 4899 | is true if any one of them is the most recent recursion. | |
| 4900 | ||
| 4901 | At "top level", all these recursion test conditions are false. Recur- | At "top level", all these recursion test conditions are false. The |
| 4902 | sive patterns are described below. | syntax for recursive patterns is described below. |
| 4903 | ||
| 4904 | Defining subpatterns for use by reference only | Defining subpatterns for use by reference only |
| 4905 | ||
| 4906 | If the condition is the string (DEFINE), and there is no subpattern | If the condition is the string (DEFINE), and there is no subpattern |
| 4907 | with the name DEFINE, the condition is always false. In this case, | with the name DEFINE, the condition is always false. In this case, |
| 4908 | there may be only one alternative in the subpattern. It is always | there may be only one alternative in the subpattern. It is always |
| 4909 | skipped if control reaches this point in the pattern; the idea of | skipped if control reaches this point in the pattern; the idea of |
| 4910 | DEFINE is that it can be used to define "subroutines" that can be ref- | DEFINE is that it can be used to define "subroutines" that can be ref- |
| 4911 | erenced from elsewhere. (The use of "subroutines" is described below.) | erenced from elsewhere. (The use of "subroutines" is described below.) |
| 4912 | For example, a pattern to match an IPv4 address could be written like | For example, a pattern to match an IPv4 address could be written like |
| 4913 | this (ignore whitespace and line breaks): | this (ignore whitespace and line breaks): |
| 4914 | ||
| 4915 | (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) | (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) |
| 4916 | \b (?&byte) (\.(?&byte)){3} \b | \b (?&byte) (\.(?&byte)){3} \b |
| 4917 | ||
| 4918 | The first part of the pattern is a DEFINE group inside which a another | The first part of the pattern is a DEFINE group inside which a another |
| 4919 | group named "byte" is defined. This matches an individual component of | group named "byte" is defined. This matches an individual component of |
| 4920 | an IPv4 address (a number less than 256). When matching takes place, | an IPv4 address (a number less than 256). When matching takes place, |
| 4921 | this part of the pattern is skipped because DEFINE acts like a false | this part of the pattern is skipped because DEFINE acts like a false |
| 4922 | condition. | condition. The rest of the pattern uses references to the named group |
| 4923 | to match the four dot-separated components of an IPv4 address, insist- | |
| 4924 | The rest of the pattern uses references to the named group to match the | ing on a word boundary at each end. |
| four dot-separated components of an IPv4 address, insisting on a word | ||
| boundary at each end. | ||
| 4925 | ||
| 4926 | Assertion conditions | Assertion conditions |
| 4927 | ||
| 4928 | If the condition is not in any of the above formats, it must be an | If the condition is not in any of the above formats, it must be an |
| 4929 | assertion. This may be a positive or negative lookahead or lookbehind | assertion. This may be a positive or negative lookahead or lookbehind |
| 4930 | assertion. Consider this pattern, again containing non-significant | assertion. Consider this pattern, again containing non-significant |
| 4931 | white space, and with the two alternatives on the second line: | white space, and with the two alternatives on the second line: |
| 4932 | ||
| 4933 | (?(?=[^a-z]*[a-z]) | (?(?=[^a-z]*[a-z]) |
| 4934 | \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) | \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) |
| 4935 | ||
| 4936 | The condition is a positive lookahead assertion that matches an | The condition is a positive lookahead assertion that matches an |
| 4937 | optional sequence of non-letters followed by a letter. In other words, | optional sequence of non-letters followed by a letter. In other words, |
| 4938 | it tests for the presence of at least one letter in the subject. If a | it tests for the presence of at least one letter in the subject. If a |
| 4939 | letter is found, the subject is matched against the first alternative; | letter is found, the subject is matched against the first alternative; |
| 4940 | otherwise it is matched against the second. This pattern matches | otherwise it is matched against the second. This pattern matches |
| 4941 | strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are | strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are |
| 4942 | letters and dd are digits. | letters and dd are digits. |
| 4943 | ||
| 4944 | ||
| 4945 | COMMENTS | COMMENTS |
| 4946 | ||
| 4947 | The sequence (?# marks the start of a comment that continues up to the | The sequence (?# marks the start of a comment that continues up to the |
| 4948 | next closing parenthesis. Nested parentheses are not permitted. The | next closing parenthesis. Nested parentheses are not permitted. The |
| 4949 | characters that make up a comment play no part in the pattern matching | characters that make up a comment play no part in the pattern matching |
| 4950 | at all. | at all. |
| 4951 | ||
| 4952 | If the PCRE_EXTENDED option is set, an unescaped # character outside a | If the PCRE_EXTENDED option is set, an unescaped # character outside a |
| 4953 | character class introduces a comment that continues to immediately | character class introduces a comment that continues to immediately |
| 4954 | after the next newline in the pattern. | after the next newline in the pattern. |
| 4955 | ||
| 4956 | ||
| 4957 | RECURSIVE PATTERNS | RECURSIVE PATTERNS |
| 4958 | ||
| 4959 | Consider the problem of matching a string in parentheses, allowing for | Consider the problem of matching a string in parentheses, allowing for |
| 4960 | unlimited nested parentheses. Without the use of recursion, the best | unlimited nested parentheses. Without the use of recursion, the best |
| 4961 | that can be done is to use a pattern that matches up to some fixed | that can be done is to use a pattern that matches up to some fixed |
| 4962 | depth of nesting. It is not possible to handle an arbitrary nesting | depth of nesting. It is not possible to handle an arbitrary nesting |
| 4963 | depth. | depth. |
| 4964 | ||
| 4965 | For some time, Perl has provided a facility that allows regular expres- | For some time, Perl has provided a facility that allows regular expres- |
| 4966 | sions to recurse (amongst other things). It does this by interpolating | sions to recurse (amongst other things). It does this by interpolating |
| 4967 | Perl code in the expression at run time, and the code can refer to the | Perl code in the expression at run time, and the code can refer to the |
| 4968 | expression itself. A Perl pattern using code interpolation to solve the | expression itself. A Perl pattern using code interpolation to solve the |
| 4969 | parentheses problem can be created like this: | parentheses problem can be created like this: |
| 4970 | ||
| # | Line 4705 RECURSIVE PATTERNS | Line 4974 RECURSIVE PATTERNS |
| 4974 | refers recursively to the pattern in which it appears. | refers recursively to the pattern in which it appears. |
| 4975 | ||
| 4976 | Obviously, PCRE cannot support the interpolation of Perl code. Instead, | Obviously, PCRE cannot support the interpolation of Perl code. Instead, |
| 4977 | it supports special syntax for recursion of the entire pattern, and | it supports special syntax for recursion of the entire pattern, and |
| 4978 | also for individual subpattern recursion. After its introduction in | also for individual subpattern recursion. After its introduction in |
| 4979 | PCRE and Python, this kind of recursion was introduced into Perl at | PCRE and Python, this kind of recursion was subsequently introduced |
| 4980 | release 5.10. | into Perl at release 5.10. |
| 4981 | ||
| 4982 | A special item that consists of (? followed by a number greater than | A special item that consists of (? followed by a number greater than |
| 4983 | zero and a closing parenthesis is a recursive call of the subpattern of | zero and a closing parenthesis is a recursive call of the subpattern of |
| 4984 | the given number, provided that it occurs inside that subpattern. (If | the given number, provided that it occurs inside that subpattern. (If |
| 4985 | not, it is a "subroutine" call, which is described in the next sec- | not, it is a "subroutine" call, which is described in the next sec- |
| 4986 | tion.) The special item (?R) or (?0) is a recursive call of the entire | tion.) The special item (?R) or (?0) is a recursive call of the entire |
| 4987 | regular expression. | regular expression. |
| 4988 | ||
| In PCRE (like Python, but unlike Perl), a recursive subpattern call is | ||
| always treated as an atomic group. That is, once it has matched some of | ||
| the subject string, it is never re-entered, even if it contains untried | ||
| alternatives and there is a subsequent matching failure. | ||
| 4989 | This PCRE pattern solves the nested parentheses problem (assume the | This PCRE pattern solves the nested parentheses problem (assume the |
| 4990 | PCRE_EXTENDED option is set so that white space is ignored): | PCRE_EXTENDED option is set so that white space is ignored): |
| 4991 | ||
| 4992 | \( ( (?>[^()]+) | (?R) )* \) | \( ( [^()]++ | (?R) )* \) |
| 4993 | ||
| 4994 | First it matches an opening parenthesis. Then it matches any number of | First it matches an opening parenthesis. Then it matches any number of |
| 4995 | substrings which can either be a sequence of non-parentheses, or a | substrings which can either be a sequence of non-parentheses, or a |
| 4996 | recursive match of the pattern itself (that is, a correctly parenthe- | recursive match of the pattern itself (that is, a correctly parenthe- |
| 4997 | sized substring). Finally there is a closing parenthesis. | sized substring). Finally there is a closing parenthesis. Note the use |
| 4998 | of a possessive quantifier to avoid backtracking into sequences of non- | |
| 4999 | parentheses. | |
| 5000 | ||
| 5001 | If this were part of a larger pattern, you would not want to recurse | If this were part of a larger pattern, you would not want to recurse |
| 5002 | the entire pattern, so instead you could use this: | the entire pattern, so instead you could use this: |
| 5003 | ||
| 5004 | ( \( ( (?>[^()]+) | (?1) )* \) ) | ( \( ( [^()]++ | (?1) )* \) ) |
| 5005 | ||
| 5006 | We have put the pattern into parentheses, and caused the recursion to | We have put the pattern into parentheses, and caused the recursion to |
| 5007 | refer to them instead of the whole pattern. | refer to them instead of the whole pattern. |
| 5008 | ||
| 5009 | In a larger pattern, keeping track of parenthesis numbers can be | In a larger pattern, keeping track of parenthesis numbers can be |
| 5010 | tricky. This is made easier by the use of relative references. (A Perl | tricky. This is made easier by the use of relative references (a Perl |
| 5011 | 5.10 feature.) Instead of (?1) in the pattern above you can write | 5.10 feature). Instead of (?1) in the pattern above you can write |
| 5012 | (?-2) to refer to the second most recently opened parentheses preceding | (?-2) to refer to the second most recently opened parentheses preceding |
| 5013 | the recursion. In other words, a negative number counts capturing | the recursion. In other words, a negative number counts capturing |
| 5014 | parentheses leftwards from the point at which it is encountered. | parentheses leftwards from the point at which it is encountered. |
| # | Line 4757 RECURSIVE PATTERNS | Line 5023 RECURSIVE PATTERNS |
| 5023 | syntax for this is (?&name); PCRE's earlier syntax (?P>name) is also | syntax for this is (?&name); PCRE's earlier syntax (?P>name) is also |
| 5024 | supported. We could rewrite the above example as follows: | supported. We could rewrite the above example as follows: |
| 5025 | ||
| 5026 | (?<pn> \( ( (?>[^()]+) | (?&pn) )* \) ) | (?<pn> \( ( [^()]++ | (?&pn) )* \) ) |
| 5027 | ||
| 5028 | If there is more than one subpattern with the same name, the earliest | If there is more than one subpattern with the same name, the earliest |
| 5029 | one is used. | one is used. |
| 5030 | ||
| 5031 | This particular example pattern that we have been looking at contains | This particular example pattern that we have been looking at contains |
| 5032 | nested unlimited repeats, and so the use of atomic grouping for match- | nested unlimited repeats, and so the use of a possessive quantifier for |
| 5033 | ing strings of non-parentheses is important when applying the pattern | matching strings of non-parentheses is important when applying the pat- |
| 5034 | to strings that do not match. For example, when this pattern is applied | tern to strings that do not match. For example, when this pattern is |
| 5035 | to | applied to |
| 5036 | ||
| 5037 | (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() | (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() |
| 5038 | ||
| 5039 | it yields "no match" quickly. However, if atomic grouping is not used, | it yields "no match" quickly. However, if a possessive quantifier is |
| 5040 | the match runs for a very long time indeed because there are so many | not used, the match runs for a very long time indeed because there are |
| 5041 | different ways the + and * repeats can carve up the subject, and all | so many different ways the + and * repeats can carve up the subject, |
| 5042 | have to be tested before failure can be reported. | and all have to be tested before failure can be reported. |
| 5043 | ||
| 5044 | At the end of a match, the values set for any capturing subpatterns are | At the end of a match, the values of capturing parentheses are those |
| 5045 | those from the outermost level of the recursion at which the subpattern | from the outermost level. If you want to obtain intermediate values, a |
| 5046 | value is set. If you want to obtain intermediate values, a callout | callout function can be used (see below and the pcrecallout documenta- |
| 5047 | function can be used (see below and the pcrecallout documentation). If | tion). If the pattern above is matched against |
| the pattern above is matched against | ||
| 5048 | ||
| 5049 | (ab(cd)ef) | (ab(cd)ef) |
| 5050 | ||
| 5051 | the value for the capturing parentheses is "ef", which is the last | the value for the inner capturing parentheses (numbered 2) is "ef", |
| 5052 | value taken on at the top level. If additional parentheses are added, | which is the last value taken on at the top level. If a capturing sub- |
| 5053 | giving | pattern is not matched at the top level, its final value is unset, even |
| 5054 | if it is (temporarily) set at a deeper level. | |
| 5055 | \( ( ( (?>[^()]+) | (?R) )* ) \) | |
| 5056 | ^ ^ | If there are more than 15 capturing parentheses in a pattern, PCRE has |
| 5057 | ^ ^ | to obtain extra memory to store data during a recursion, which it does |
| 5058 | by using pcre_malloc, freeing it via pcre_free afterwards. If no memory | |
| 5059 | the string they capture is "ab(cd)ef", the contents of the top level | can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. |
| parentheses. If there are more than 15 capturing parentheses in a pat- | ||
| tern, PCRE has to obtain extra memory to store data during a recursion, | ||
| which it does by using pcre_malloc, freeing it via pcre_free after- | ||
| wards. If no memory can be obtained, the match fails with the | ||
| PCRE_ERROR_NOMEMORY error. | ||
| 5060 | ||
| 5061 | Do not confuse the (?R) item with the condition (R), which tests for | Do not confuse the (?R) item with the condition (R), which tests for |
| 5062 | recursion. Consider this pattern, which matches text in angle brack- | recursion. Consider this pattern, which matches text in angle brack- |
| # | Line 4810 RECURSIVE PATTERNS | Line 5070 RECURSIVE PATTERNS |
| 5070 | two different alternatives for the recursive and non-recursive cases. | two different alternatives for the recursive and non-recursive cases. |
| 5071 | The (?R) item is the actual recursive call. | The (?R) item is the actual recursive call. |
| 5072 | ||
| 5073 | Recursion difference from Perl | |
| 5074 | ||
| 5075 | In PCRE (like Python, but unlike Perl), a recursive subpattern call is | |
| 5076 | always treated as an atomic group. That is, once it has matched some of | |
| 5077 | the subject string, it is never re-entered, even if it contains untried | |
| 5078 | alternatives and there is a subsequent matching failure. This can be | |
| 5079 | illustrated by the following pattern, which purports to match a palin- | |
| 5080 | dromic string that contains an odd number of characters (for example, | |
| 5081 | "a", "aba", "abcba", "abcdcba"): | |
| 5082 | ||
| 5083 | ^(.|(.)(?1)\2)$ | |
| 5084 | ||
| 5085 | The idea is that it either matches a single character, or two identical | |
| 5086 | characters surrounding a sub-palindrome. In Perl, this pattern works; | |
| 5087 | in PCRE it does not if the pattern is longer than three characters. | |
| 5088 | Consider the subject string "abcba": | |
| 5089 | ||
| 5090 | At the top level, the first character is matched, but as it is not at | |
| 5091 | the end of the string, the first alternative fails; the second alterna- | |
| 5092 | tive is taken and the recursion kicks in. The recursive call to subpat- | |
| 5093 | tern 1 successfully matches the next character ("b"). (Note that the | |
| 5094 | beginning and end of line tests are not part of the recursion). | |
| 5095 | ||
| 5096 | Back at the top level, the next character ("c") is compared with what | |
| 5097 | subpattern 2 matched, which was "a". This fails. Because the recursion | |
| 5098 | is treated as an atomic group, there are now no backtracking points, | |
| 5099 | and so the entire match fails. (Perl is able, at this point, to re- | |
| 5100 | enter the recursion and try the second alternative.) However, if the | |
| 5101 | pattern is written with the alternatives in the other order, things are | |
| 5102 | different: | |
| 5103 | ||
| 5104 | ^((.)(?1)\2|.)$ | |
| 5105 | ||
| 5106 | This time, the recursing alternative is tried first, and continues to | |
| 5107 | recurse until it runs out of characters, at which point the recursion | |
| 5108 | fails. But this time we do have another alternative to try at the | |
| 5109 | higher level. That is the big difference: in the previous case the | |
| 5110 | remaining alternative is at a deeper recursion level, which PCRE cannot | |
| 5111 | use. | |
| 5112 | ||
| 5113 | To change the pattern so that matches all palindromic strings, not just | |
| 5114 | those with an odd number of characters, it is tempting to change the | |
| 5115 | pattern to this: | |
| 5116 | ||
| 5117 | ^((.)(?1)\2|.?)$ | |
| 5118 | ||
| 5119 | Again, this works in Perl, but not in PCRE, and for the same reason. | |
| 5120 | When a deeper recursion has matched a single character, it cannot be | |
| 5121 | entered again in order to match an empty string. The solution is to | |
| 5122 | separate the two cases, and write out the odd and even cases as alter- | |
| 5123 | natives at the higher level: | |
| 5124 | ||
| 5125 | ^(?:((.)(?1)\2|)|((.)(?3)\4|.)) | |
| 5126 | ||
| 5127 | If you want to match typical palindromic phrases, the pattern has to | |
| 5128 | ignore all non-word characters, which can be done like this: | |
| 5129 | ||
| 5130 | ^\W*+(?:((.)\W*+(?1)\W*+\2|)|((.)\W*+(?3)\W*+\4|\W*+.\W*+))\W*+$ | |
| 5131 | ||
| 5132 | If run with the PCRE_CASELESS option, this pattern matches phrases such | |
| 5133 | as "A man, a plan, a canal: Panama!" and it works well in both PCRE and | |
| 5134 | Perl. Note the use of the possessive quantifier *+ to avoid backtrack- | |
| 5135 | ing into sequences of non-word characters. Without this, PCRE takes a | |
| 5136 | great deal longer (ten times or more) to match typical phrases, and | |
| 5137 | Perl takes so long that you think it has gone into a loop. | |
| 5138 | ||
| 5139 | WARNING: The palindrome-matching patterns above work only if the sub- | |
| 5140 | ject string does not start with a palindrome that is shorter than the | |
| 5141 | entire string. For example, although "abcba" is correctly matched, if | |
| 5142 | the subject is "ababa", PCRE finds the palindrome "aba" at the start, | |
| 5143 | then fails at top level because the end of the string does not follow. | |
| 5144 | Once again, it cannot jump back into the recursion to try other alter- | |
| 5145 | natives, so the entire match fails. | |
| 5146 | ||
| 5147 | ||
| 5148 | SUBPATTERNS AS SUBROUTINES | SUBPATTERNS AS SUBROUTINES |
| 5149 | ||
| # | Line 4836 SUBPATTERNS AS SUBROUTINES | Line 5170 SUBPATTERNS AS SUBROUTINES |
| 5170 | two strings. Another example is given in the discussion of DEFINE | two strings. Another example is given in the discussion of DEFINE |
| 5171 | above. | above. |
| 5172 | ||
| 5173 | Like recursive subpatterns, a "subroutine" call is always treated as an | Like recursive subpatterns, a subroutine call is always treated as an |
| 5174 | atomic group. That is, once it has matched some of the subject string, | atomic group. That is, once it has matched some of the subject string, |
| 5175 | it is never re-entered, even if it contains untried alternatives and | it is never re-entered, even if it contains untried alternatives and |
| 5176 | there is a subsequent matching failure. | there is a subsequent matching failure. Any capturing parentheses that |
| 5177 | are set during the subroutine call revert to their previous values | |
| 5178 | afterwards. | |
| 5179 | ||
| 5180 | When a subpattern is used as a subroutine, processing options such as | When a subpattern is used as a subroutine, processing options such as |
| 5181 | case-independence are fixed when the subpattern is defined. They cannot | case-independence are fixed when the subpattern is defined. They cannot |
| # | Line 4922 BACKTRACKING CONTROL | Line 5258 BACKTRACKING CONTROL |
| 5258 | (*FAIL), which behaves like a failing negative assertion, they cause an | (*FAIL), which behaves like a failing negative assertion, they cause an |
| 5259 | error if encountered by pcre_dfa_exec(). | error if encountered by pcre_dfa_exec(). |
| 5260 | ||
| 5261 | If any of these verbs are used in an assertion or subroutine subpattern | |
| 5262 | (including recursive subpatterns), their effect is confined to that | |
| 5263 | subpattern; it does not extend to the surrounding pattern. Note that | |
| 5264 | such subpatterns are processed as anchored at the point where they are | |
| 5265 | tested. | |
| 5266 | ||
| 5267 | The new verbs make use of what was previously invalid syntax: an open- | The new verbs make use of what was previously invalid syntax: an open- |
| 5268 | ing parenthesis followed by an asterisk. In Perl, they are generally of | ing parenthesis followed by an asterisk. They are generally of the form |
| 5269 | the form (*VERB:ARG) but PCRE does not support the use of arguments, so | (*VERB) or (*VERB:NAME). Some may take either form, with differing be- |
| 5270 | its general form is just (*VERB). Any number of these verbs may occur | haviour, depending on whether or not an argument is present. An name is |
| 5271 | in a pattern. There are two kinds: | a sequence of letters, digits, and underscores. If the name is empty, |
| 5272 | that is, if the closing parenthesis immediately follows the colon, the | |
| 5273 | effect is as if the colon were not there. Any number of these verbs may | |
| 5274 | occur in a pattern. | |
| 5275 | ||
| 5276 | PCRE contains some optimizations that are used to speed up matching by | |
| 5277 | running some checks at the start of each match attempt. For example, it | |
| 5278 | may know the minimum length of matching subject, or that a particular | |
| 5279 | character must be present. When one of these optimizations suppresses | |
| 5280 | the running of a match, any included backtracking verbs will not, of | |
| 5281 | course, be processed. You can suppress the start-of-match optimizations | |
| 5282 | by setting the PCRE_NO_START_OPTIMIZE option when calling pcre_exec(). | |
| 5283 | ||
| 5284 | Verbs that act immediately | Verbs that act immediately |
| 5285 | ||
| 5286 | The following verbs act as soon as they are encountered: | The following verbs act as soon as they are encountered. They may not |
| 5287 | be followed by a name. | |
| 5288 | ||
| 5289 | (*ACCEPT) | (*ACCEPT) |
| 5290 | ||
| 5291 | This verb causes the match to end successfully, skipping the remainder | This verb causes the match to end successfully, skipping the remainder |
| 5292 | of the pattern. When inside a recursion, only the innermost pattern is | of the pattern. When inside a recursion, only the innermost pattern is |
| 5293 | ended immediately. PCRE differs from Perl in what happens if the | ended immediately. If (*ACCEPT) is inside capturing parentheses, the |
| 5294 | (*ACCEPT) is inside capturing parentheses. In Perl, the data so far is | data so far is captured. (This feature was added to PCRE at release |
| 5295 | captured: in PCRE no data is captured. For example: | 8.00.) For example: |
| 5296 | ||
| 5297 | A(A|B(*ACCEPT)|C)D | A((?:A|B(*ACCEPT)|C)D) |
| 5298 | ||
| 5299 | This matches "AB", "AAD", or "ACD", but when it matches "AB", no data | This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is cap- |
| 5300 | is captured. | tured by the outer parentheses. |
| 5301 | ||
| 5302 | (*FAIL) or (*F) | (*FAIL) or (*F) |
| 5303 | ||
| # | Line 4959 BACKTRACKING CONTROL | Line 5313 BACKTRACKING CONTROL |
| 5313 | A match with the string "aaaa" always fails, but the callout is taken | A match with the string "aaaa" always fails, but the callout is taken |
| 5314 | before each backtrack happens (in this example, 10 times). | before each backtrack happens (in this example, 10 times). |
| 5315 | ||
| 5316 | Recording which path was taken | |
| 5317 | ||
| 5318 | There is one verb whose main purpose is to track how a match was | |
| 5319 | arrived at, though it also has a secondary use in conjunction with | |
| 5320 | advancing the match starting point (see (*SKIP) below). | |
| 5321 | ||
| 5322 | (*MARK:NAME) or (*:NAME) | |
| 5323 | ||
| 5324 | A name is always required with this verb. There may be as many | |
| 5325 | instances of (*MARK) as you like in a pattern, and their names do not | |
| 5326 | have to be unique. | |
| 5327 | ||
| 5328 | When a match succeeds, the name of the last-encountered (*MARK) is | |
| 5329 | passed back to the caller via the pcre_extra data structure, as | |
| 5330 | described in the section on pcre_extra in the pcreapi documentation. No | |
| 5331 | data is returned for a partial match. Here is an example of pcretest | |
| 5332 | output, where the /K modifier requests the retrieval and outputting of | |
| 5333 | (*MARK) data: | |
| 5334 | ||
| 5335 | /X(*MARK:A)Y|X(*MARK:B)Z/K | |
| 5336 | XY | |
| 5337 | 0: XY | |
| 5338 | MK: A | |
| 5339 | XZ | |
| 5340 | 0: XZ | |
| 5341 | MK: B | |
| 5342 | ||
| 5343 | The (*MARK) name is tagged with "MK:" in this output, and in this exam- | |
| 5344 | ple it indicates which of the two alternatives matched. This is a more | |
| 5345 | efficient way of obtaining this information than putting each alterna- | |
| 5346 | tive in its own capturing parentheses. | |
| 5347 | ||
| 5348 | A name may also be returned after a failed match if the final path | |
| 5349 | through the pattern involves (*MARK). However, unless (*MARK) used in | |
| 5350 | conjunction with (*COMMIT), this is unlikely to happen for an unan- | |
| 5351 | chored pattern because, as the starting point for matching is advanced, | |
| 5352 | the final check is often with an empty string, causing a failure before | |
| 5353 | (*MARK) is reached. For example: | |
| 5354 | ||
| 5355 | /X(*MARK:A)Y|X(*MARK:B)Z/K | |
| 5356 | XP | |
| 5357 | No match | |
| 5358 | ||
| 5359 | There are three potential starting points for this match (starting with | |
| 5360 | X, starting with P, and with an empty string). If the pattern is | |
| 5361 | anchored, the result is different: | |
| 5362 | ||
| 5363 | /^X(*MARK:A)Y|^X(*MARK:B)Z/K | |
| 5364 | XP | |
| 5365 | No match, mark = B | |
| 5366 | ||
| 5367 | PCRE's start-of-match optimizations can also interfere with this. For | |
| 5368 | example, if, as a result of a call to pcre_study(), it knows the mini- | |
| 5369 | mum subject length for a match, a shorter subject will not be scanned | |
| 5370 | at all. | |
| 5371 | ||
| 5372 | Note that similar anomalies (though different in detail) exist in Perl, | |
| 5373 | no doubt for the same reasons. The use of (*MARK) data after a failed | |
| 5374 | match of an unanchored pattern is not recommended, unless (*COMMIT) is | |
| 5375 | involved. | |
| 5376 | ||
| 5377 | Verbs that act after backtracking | Verbs that act after backtracking |
| 5378 | ||
| 5379 | The following verbs do nothing when they are encountered. Matching con- | The following verbs do nothing when they are encountered. Matching con- |
| 5380 | tinues with what follows, but if there is no subsequent match, a fail- | tinues with what follows, but if there is no subsequent match, causing |
| 5381 | ure is forced. The verbs differ in exactly what kind of failure | a backtrack to the verb, a failure is forced. That is, backtracking |
| 5382 | occurs. | cannot pass to the left of the verb. However, when one of these verbs |
| 5383 | appears inside an atomic group, its effect is confined to that group, | |
| 5384 | because once the group has been matched, there is never any backtrack- | |
| 5385 | ing into it. In this situation, backtracking can "jump back" to the | |
| 5386 | left of the entire atomic group. (Remember also, as stated above, that | |
| 5387 | this localization also applies in subroutine calls and assertions.) | |
| 5388 | ||
| 5389 | These verbs differ in exactly what kind of failure occurs when back- | |
| 5390 | tracking reaches them. | |
| 5391 | ||
| 5392 | (*COMMIT) | (*COMMIT) |
| 5393 | ||
| 5394 | This verb causes the whole match to fail outright if the rest of the | This verb, which may not be followed by a name, causes the whole match |
| 5395 | pattern does not match. Even if the pattern is unanchored, no further | to fail outright if the rest of the pattern does not match. Even if the |
| 5396 | attempts to find a match by advancing the start point take place. Once | pattern is unanchored, no further attempts to find a match by advancing |
| 5397 | (*COMMIT) has been passed, pcre_exec() is committed to finding a match | the starting point take place. Once (*COMMIT) has been passed, |
| 5398 | at the current starting point, or not at all. For example: | pcre_exec() is committed to finding a match at the current starting |
| 5399 | point, or not at all. For example: | |
| 5400 | ||
| 5401 | a+(*COMMIT)b | a+(*COMMIT)b |
| 5402 | ||
| 5403 | This matches "xxaab" but not "aacaab". It can be thought of as a kind | This matches "xxaab" but not "aacaab". It can be thought of as a kind |
| 5404 | of dynamic anchor, or "I've started, so I must finish." | of dynamic anchor, or "I've started, so I must finish." The name of the |
| 5405 | most recently passed (*MARK) in the path is passed back when (*COMMIT) | |
| 5406 | (*PRUNE) | forces a match failure. |
| 5407 | ||
| 5408 | Note that (*COMMIT) at the start of a pattern is not the same as an | |
| 5409 | anchor, unless PCRE's start-of-match optimizations are turned off, as | |
| 5410 | shown in this pcretest example: | |
| 5411 | ||
| 5412 | /(*COMMIT)abc/ | |
| 5413 | xyzabc | |
| 5414 | 0: abc | |
| 5415 | xyzabc\Y | |
| 5416 | No match | |
| 5417 | ||
| 5418 | This verb causes the match to fail at the current position if the rest | PCRE knows that any match must start with "a", so the optimization |
| 5419 | of the pattern does not match. If the pattern is unanchored, the normal | skips along the subject to "a" before running the first match attempt, |
| 5420 | "bumpalong" advance to the next starting character then happens. Back- | which succeeds. When the optimization is disabled by the \Y escape in |
| 5421 | tracking can occur as usual to the left of (*PRUNE), or when matching | the second subject, the match starts at "x" and so the (*COMMIT) causes |
| 5422 | to the right of (*PRUNE), but if there is no match to the right, back- | it to fail without trying any other starting points. |
| 5423 | tracking cannot cross (*PRUNE). In simple cases, the use of (*PRUNE) | |
| 5424 | is just an alternative to an atomic group or possessive quantifier, but | (*PRUNE) or (*PRUNE:NAME) |
| 5425 | there are some uses of (*PRUNE) that cannot be expressed in any other | |
| 5426 | way. | This verb causes the match to fail at the current starting position in |
| 5427 | the subject if the rest of the pattern does not match. If the pattern | |
| 5428 | is unanchored, the normal "bumpalong" advance to the next starting | |
| 5429 | character then happens. Backtracking can occur as usual to the left of | |
| 5430 | (*PRUNE), before it is reached, or when matching to the right of | |
| 5431 | (*PRUNE), but if there is no match to the right, backtracking cannot | |