/[pcre]/code/trunk/doc/pcreapi.3

Diff of /code/trunk/doc/pcreapi.3

Parent Directory | Revision Log | View Patch Patch

-revision 225 by ph10,
Mon Aug 20 14:38:34 2007 UTC
+revision 659 by ph10,
Tue Aug 16 09:48:29 2011 UTC
 Line 132 
 an application that uses PCRE. The heade
  and PCRE_MINOR to contain the major and minor release numbers for the library.
  Applications can use these to include support for different releases of PCRE.
  .P
+ In a Windows environment, if you want to statically link an application program
+ against a non-dll \fBpcre.a\fP file, you must define PCRE_STATIC before
+ including \fBpcre.h\fP or \fBpcrecpp.h\fP, because otherwise the
+ \fBpcre_malloc()\fP and \fBpcre_free()\fP exported functions will be declared
+ \fB__declspec(dllimport)\fP, with unwanted results.
+ .P
  The functions \fBpcre_compile()\fP, \fBpcre_compile2()\fP, \fBpcre_study()\fP,
  and \fBpcre_exec()\fP are used for compiling and matching regular expressions
  in a Perl-compatible manner. A sample program that demonstrates the simplest
- way of using them is provided in the file called \fIpcredemo.c\fP in the source
+ way of using them is provided in the file called \fIpcredemo.c\fP in the PCRE
- distribution. The
+ source distribution. A listing of this program is given in the
+ .\" HREF
+ \fBpcredemo\fP
+ .\"
+ documentation, and the
  .\" HREF
  \fBpcresample\fP
  .\"
- documentation describes how to run it.
+ documentation describes how to compile and run it.
  .P
  A second matching function, \fBpcre_dfa_exec()\fP, which is not
  Perl-compatible, is also provided. This uses a different algorithm for the
  matching. The alternative algorithm finds all possible matches (at a given
- point in the subject), and scans the subject just once. However, this algorithm
+ point in the subject), and scans the subject just once (unless there are
- does not return captured substrings. A description of the two matching
+ lookbehind assertions). However, this algorithm does not return captured
- algorithms and their advantages and disadvantages is given in the
+ substrings. A description of the two matching algorithms and their advantages
+ and disadvantages is given in the
  .\" HREF
  \fBpcrematching\fP
  .\"
-Line 218 
 points during a matching operation. Deta
+Line 229 
 points during a matching operation. Deta
  documentation.
  .
  .
+ .\" HTML <a name="newlines"></a>
  .SH NEWLINES
  .rs
  .sp
-Line 235 
 The default default is LF, which is the
+Line 247 
 The default default is LF, which is the
  default can be overridden, either when a pattern is compiled, or when it is
  matched.
  .P
+ At compile time, the newline convention can be specified by the \fIoptions\fP
+ argument of \fBpcre_compile()\fP, or it can be specified by special text at the
+ start of the pattern itself; this overrides any other settings. See the
+ .\" HREF
+ \fBpcrepattern\fP
+ .\"
+ page for details of the special character sequences.
+ .P
  In the PCRE documentation the word "newline" is used to mean "the character or
  pair of characters that indicate a line break". The choice of newline
  convention affects the handling of the dot, circumflex, and dollar
  metacharacters, the handling of #-comments in /x mode, and, when CRLF is a
  recognized line ending sequence, the match position advancement for a
- non-anchored pattern. The choice of newline convention does not affect the
+ non-anchored pattern. There is more detail about this in the
- interpretation of the \en or \er escape sequences.
+ .\" HTML <a href="#execoptions">
+ .\" </a>
+ section on \fBpcre_exec()\fP options
+ .\"
+ below.
+ .P
+ The choice of newline convention does not affect the interpretation of
+ the \en or \er escape sequences, nor does it affect what \eR matches, which is
+ controlled in a similar way, but by separate options.
  .
  .
  .SH MULTITHREADING
-Line 300 
 properties is available; otherwise it is
+Line 328 
 properties is available; otherwise it is
  .sp
  The output is an integer whose value specifies the default character sequence
  that is recognized as meaning "newline". The four values that are supported
- are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF, and -1 for ANY. The
+ are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF, and -1 for ANY.
- default should normally be the standard sequence for your operating system.
+ Though they are derived from ASCII, the same values are returned in EBCDIC
+ environments. The default should normally correspond to the standard sequence
+ for your operating system.
+ .sp
+   PCRE_CONFIG_BSR
+ .sp
+ The output is an integer whose value indicates what character sequences the \eR
+ escape sequence matches by default. A value of 0 means that \eR matches any
+ Unicode line ending sequence; a value of 1 means that \eR matches only CR, LF,
+ or CRLF. The default can be overridden when a pattern is compiled or matched.
  .sp
    PCRE_CONFIG_LINK_SIZE
  .sp
-Line 323 
 documentation.
+Line 360 
 documentation.
  .sp
    PCRE_CONFIG_MATCH_LIMIT
  .sp
- The output is an integer that gives the default limit for the number of
+ The output is a long integer that gives the default limit for the number of
  internal matching function calls in a \fBpcre_exec()\fP execution. Further
  details are given with \fBpcre_exec()\fP below.
  .sp
    PCRE_CONFIG_MATCH_LIMIT_RECURSION
  .sp
- The output is an integer that gives the default limit for the depth of
+ The output is a long integer that gives the default limit for the depth of
  recursion when calling the internal matching function in a \fBpcre_exec()\fP
  execution. Further details are given with \fBpcre_exec()\fP below.
  .sp
-Line 364 
 avoiding the use of the stack.
+Line 401 
 avoiding the use of the stack.
  Either of the functions \fBpcre_compile()\fP or \fBpcre_compile2()\fP can be
  called to compile a pattern into an internal form. The only difference between
  the two interfaces is that \fBpcre_compile2()\fP has an additional argument,
- \fIerrorcodeptr\fP, via which a numerical error code can be returned.
+ \fIerrorcodeptr\fP, via which a numerical error code can be returned. To avoid
+ too much repetition, we refer just to \fBpcre_compile()\fP below, but the
+ information applies equally to \fBpcre_compile2()\fP.
  .P
  The pattern is a C string terminated by a binary zero, and is passed in the
  \fIpattern\fP argument. A pointer to a single block of memory that is obtained
-Line 380 
 argument, which is an address (see below
+Line 419 
 argument, which is an address (see below
  .P
  The \fIoptions\fP argument contains various bit settings that affect the
  compilation. It should be zero if no options are required. The available
- options are described below. Some of them, in particular, those that are
+ options are described below. Some of them (in particular, those that are
- compatible with Perl, can also be set and unset from within the pattern (see
+ compatible with Perl, but some others as well) can also be set and unset from
- the detailed description in the
+ within the pattern (see the detailed description in the
  .\" HREF
  \fBpcrepattern\fP
  .\"
- documentation). For these options, the contents of the \fIoptions\fP argument
+ documentation). For those options that can be different in different parts of
- specifies their initial settings at the start of compilation and execution. The
+ the pattern, the contents of the \fIoptions\fP argument specifies their
- PCRE_ANCHORED and PCRE_NEWLINE_\fIxxx\fP options can be set at the time of
+ settings at the start of compilation and execution. The PCRE_ANCHORED,
- matching as well as at compile time.
+ PCRE_BSR_\fIxxx\fP, PCRE_NEWLINE_\fIxxx\fP, PCRE_NO_UTF8_CHECK, and
+ PCRE_NO_START_OPT options can be set at the time of matching as well as at
+ compile time.
  .P
  If \fIerrptr\fP is NULL, \fBpcre_compile()\fP returns NULL immediately.
  Otherwise, if compilation of a pattern fails, \fBpcre_compile()\fP returns
  NULL, and sets the variable pointed to by \fIerrptr\fP to point to a textual
  error message. This is a static string that is part of the library. You must
- not try to free it. The offset from the start of the pattern to the character
+ not try to free it. Normally, the offset from the start of the pattern to the
- where the error was discovered is placed in the variable pointed to by
+ byte that was being processed when the error was discovered is placed in the
- \fIerroffset\fP, which must not be NULL. If it is, an immediate error is given.
+ variable pointed to by \fIerroffset\fP, which must not be NULL (if it is, an
+ immediate error is given). However, for an invalid UTF-8 string, the offset is
+ that of the first byte of the failing character. Also, some errors are not
+ detected until checks are carried out when the whole pattern has been scanned;
+ in these cases the offset passed back is the length of the pattern.
+ .P
+ Note that the offset is in bytes, not characters, even in UTF-8 mode. It may
+ sometimes point into the middle of a UTF-8 character.
  .P
  If \fBpcre_compile2()\fP is used instead of \fBpcre_compile()\fP, and the
  \fIerrorcodeptr\fP argument is not NULL, a non-zero error code number is
-Line 444 
 facility, see the
+Line 492 
 facility, see the
  .\"
  documentation.
  .sp
+   PCRE_BSR_ANYCRLF
+   PCRE_BSR_UNICODE
+ .sp
+ These options (which are mutually exclusive) control what the \eR escape
+ sequence matches. The choice is either to match only CR, LF, or CRLF, or to
+ match any Unicode newline sequence. The default is specified when PCRE is
+ built. It can be overridden from within the pattern, or by setting an option
+ when a compiled pattern is matched.
+ .sp
    PCRE_CASELESS
  .sp
  If this bit is set, letters in the pattern match both upper and lower case
-Line 467 
 pattern.
+Line 524 
 pattern.
  .sp
    PCRE_DOTALL
  .sp
- If this bit is set, a dot metacharater in the pattern matches all characters,
+ If this bit is set, a dot metacharacter in the pattern matches a character of
- including those that indicate newline. Without it, a dot does not match when
+ any value, including one that indicates a newline. However, it only ever
- the current position is at a newline. This option is equivalent to Perl's /s
+ matches one character, even if newlines are coded as CRLF. Without this option,
- option, and it can be changed within a pattern by a (?s) option setting. A
+ a dot does not match when the current position is at a newline. This option is
- negative class such as [^a] always matches newline characters, independent of
+ equivalent to Perl's /s option, and it can be changed within a pattern by a
- the setting of this option.
+ (?s) option setting. A negative class such as [^a] always matches newline
+ characters, independent of the setting of this option.
  .sp
    PCRE_DUPNAMES
  .sp
-Line 494 
 unescaped # outside a character class an
+Line 552 
 unescaped # outside a character class an
  ignored. This is equivalent to Perl's /x option, and it can be changed within a
  pattern by a (?x) option setting.
  .P
+ Which characters are interpreted as newlines is controlled by the options
+ passed to \fBpcre_compile()\fP or by a special sequence at the start of the
+ pattern, as described in the section entitled
+ .\" HTML <a href="pcrepattern.html#newlines">
+ .\" </a>
+ "Newline conventions"
+ .\"
+ in the \fBpcrepattern\fP documentation. Note that the end of this type of
+ comment is a literal newline sequence in the pattern; escape sequences that
+ happen to represent a newline do not count.
+ .P
  This option makes it possible to include comments inside complicated patterns.
  Note, however, that this applies only to data characters. Whitespace characters
  may never appear within special character sequences in a pattern, for example
- within the sequence (?( which introduces a conditional subpattern.
+ within the sequence (?( that introduces a conditional subpattern.
  .sp
    PCRE_EXTRA
  .sp
-Line 507 
 set, any backslash in a pattern that is
+Line 576 
 set, any backslash in a pattern that is
  special meaning causes an error, thus reserving these combinations for future
  expansion. By default, as in Perl, a backslash followed by a letter with no
  special meaning is treated as a literal. (Perl can, however, be persuaded to
- give a warning for this.) There are at present no other features controlled by
+ give an error for this, by running it with the -w option.) There are at present
- this option. It can also be set by a (?X) option setting within a pattern.
+ no other features controlled by this option. It can also be set by a (?X)
+ option setting within a pattern.
  .sp
    PCRE_FIRSTLINE
  .sp
-Line 516 
 If this option is set, an unanchored pat
+Line 586 
 If this option is set, an unanchored pat
  the first newline in the subject string, though the matched text may continue
  over the newline.
  .sp
+   PCRE_JAVASCRIPT_COMPAT
+ .sp
+ If this option is set, PCRE's behaviour is changed in some ways so that it is
+ compatible with JavaScript rather than Perl. The changes are as follows:
+ .P
+ (1) A lone closing square bracket in a pattern causes a compile-time error,
+ because this is illegal in JavaScript (by default it is treated as a data
+ character). Thus, the pattern AB]CD becomes illegal when this option is set.
+ .P
+ (2) At run time, a back reference to an unset subpattern group matches an empty
+ string (by default this causes the current matching alternative to fail). A
+ pattern such as (\e1)(a) succeeds when this option is set (assuming it can find
+ an "a" in the subject), whereas it fails by default, for Perl compatibility.
+ .sp
    PCRE_MULTILINE
  .sp
  By default, PCRE treats the subject string as consisting of a single line of
-Line 557 
 option, the combination may or may not b
+Line 641 
 option, the combination may or may not b
  PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to PCRE_NEWLINE_CRLF, but
  other combinations may yield unused numbers and cause an error.
  .P
- The only time that a line break is specially recognized when compiling a
+ The only time that a line break in a pattern is specially recognized when
- pattern is if PCRE_EXTENDED is set, and an unescaped # outside a character
+ compiling is when PCRE_EXTENDED is set. CR and LF are whitespace characters,
- class is encountered. This indicates a comment that lasts until after the next
+ and so are ignored in this mode. Also, an unescaped # outside a character class
- line break sequence. In other circumstances, line break sequences are treated
+ indicates a comment that lasts until after the next line break sequence. In
- as literal data, except that in PCRE_EXTENDED mode, both CR and LF are treated
+ other circumstances, line break sequences in patterns are treated as literal
- as whitespace characters and are therefore ignored.
+ data.
  .P
  The newline option that is set at compile time becomes the default that is used
  for \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP, but it can be overridden.
-Line 575 
 were followed by ?: but named parenthese
+Line 659 
 were followed by ?: but named parenthese
  they acquire numbers in the usual way). There is no equivalent of this option
  in Perl.
  .sp
+   NO_START_OPTIMIZE
+ .sp
+ This is an option that acts at matching time; that is, it is really an option
+ for \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. If it is set at compile time,
+ it is remembered with the compiled pattern and assumed at matching time. For
+ details see the discussion of PCRE_NO_START_OPTIMIZE
+ .\" HTML <a href="#execoptions">
+ .\" </a>
+ below.
+ .\"
+ .sp
+   PCRE_UCP
+ .sp
+ This option changes the way PCRE processes \eB, \eb, \eD, \ed, \eS, \es, \eW,
+ \ew, and some of the POSIX character classes. By default, only ASCII characters
+ are recognized, but if PCRE_UCP is set, Unicode properties are used instead to
+ classify characters. More details are given in the section on
+ .\" HTML <a href="pcre.html#genericchartypes">
+ .\" </a>
+ generic character types
+ .\"
+ in the
+ .\" HREF
+ \fBpcrepattern\fP
+ .\"
+ page. If you set PCRE_UCP, matching one of the items it affects takes much
+ longer. The option is available only if PCRE has been compiled with Unicode
+ property support.
+ .sp
    PCRE_UNGREEDY
  .sp
  This option inverts the "greediness" of the quantifiers so that they are not
-Line 639 
 out of use. To avoid confusion, they hav
+Line 752 
 out of use. To avoid confusion, they hav
  nothing to repeat
  [this code is not in use]
  internal error: unexpected repeat
- unrecognized character after (?
+ unrecognized character after (? or (?-
  POSIX named classes are supported only within a class
  missing )
  reference to non-existent subpattern
-Line 647 
 out of use. To avoid confusion, they hav
+Line 760 
 out of use. To avoid confusion, they hav
  unknown option bit(s) set
  missing ) after comment
  [this code is not in use]
- regular expression too large
+ regular expression is too large
  failed to get memory
  unmatched parentheses
  internal error: code overflow
-Line 664 
 out of use. To avoid confusion, they hav
+Line 777 
 out of use. To avoid confusion, they hav
  character value in \ex{...} sequence is too large
  invalid condition (?(0)
  \eC not allowed in lookbehind assertion
- PCRE does not support \eL, \el, \eN, \eU, or \eu
+ PCRE does not support \eL, \el, \eN{name}, \eU, or \eu
  number after (?C is > 255
  closing ) for (?C expected
  recursive call could loop indefinitely
-Line 676 
 out of use. To avoid confusion, they hav
+Line 789 
 out of use. To avoid confusion, they hav
  malformed \eP or \ep sequence
  unknown property name after \eP or \ep
  subpattern name is too long (maximum 32 characters)
- too many named subpatterns (maximum 10,000)
+ too many named subpatterns (maximum 10000)
  [this code is not in use]
  octal value is greater than \e377 (not in UTF-8 mode)
  internal error: overran compiling workspace
- internal error: previously-checked referenced subpattern not found
+ internal error: previously-checked referenced subpattern
+         not found
  DEFINE group contains more than one branch
  repeating a DEFINE group is not allowed
- inconsistent NEWLINE options"
+ inconsistent NEWLINE options
- \eg is not followed by a braced name or an optionally braced
+ \eg is not followed by a braced, angle-bracketed, or quoted
-         non-zero number
+         name/number or by a plain number
- (?+ or (?- or (?(+ or (?(- must be followed by a non-zero number
+ a numbered reference must not be zero
+ an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT)
+ (*VERB) not recognized
+ number is too big
+ subpattern name expected
+ digit expected after (?+
+ ] is an invalid data character in JavaScript compatibility mode
+ different names for subpatterns of the same number are
+         not allowed
+ (*MARK) must have an argument
+ this version of PCRE is not compiled with PCRE_UCP support
+ \ec must be followed by an ASCII character
+ \ek is not followed by a braced, angle-bracketed, or quoted name
+ .sp
+ The numbers 32 and 10000 in errors 48 and 49 are defaults; different values may
+ be used if the limits were changed when PCRE was built.
  .
  .
+ .\" HTML <a name="studyingapattern"></a>
  .SH "STUDYING A PATTERN"
  .rs
  .sp
-Line 705 
 help speed up matching, \fBpcre_study()\
+Line 835 
 help speed up matching, \fBpcre_study()\
  results of the study.
  .P
  The returned value from \fBpcre_study()\fP can be passed directly to
- \fBpcre_exec()\fP. However, a \fBpcre_extra\fP block also contains other
+ \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. However, a \fBpcre_extra\fP block
- fields that can be set by the caller before the block is passed; these are
+ also contains other fields that can be set by the caller before the block is
- described
+ passed; these are described
  .\" HTML <a href="#extradata">
  .\" </a>
  below
  .\"
  in the section on matching a pattern.
  .P
- If studying the pattern does not produce any additional information
+ If studying the pattern does not produce any useful information,
  \fBpcre_study()\fP returns NULL. In that circumstance, if the calling program
- wants to pass any of the other fields to \fBpcre_exec()\fP, it must set up its
+ wants to pass any of the other fields to \fBpcre_exec()\fP or
- own \fBpcre_extra\fP block.
+ \fBpcre_dfa_exec()\fP, it must set up its own \fBpcre_extra\fP block.
  .P
  The second argument of \fBpcre_study()\fP contains option bits. At present, no
  options are defined, and this argument should always be zero.
-Line 737 
 This is a typical call to \fBpcre_study\
+Line 867 
 This is a typical call to \fBpcre_study\
 ,              /* no options exist */
      &error);        /* set to NULL or points to a message */
  .sp
- At present, studying a pattern is useful only for non-anchored patterns that do
+ Studying a pattern does two things: first, a lower bound for the length of
- not have a single fixed starting character. A bitmap of possible starting
+ subject string that is needed to match the pattern is computed. This does not
- bytes is created.
+ mean that there are any strings of that length that match, but it does
+ guarantee that no shorter strings match. The value is used by
+ \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP to avoid wasting time by trying to
+ match strings that are shorter than the lower bound. You can find out the value
+ in a calling program via the \fBpcre_fullinfo()\fP function.
+ .P
+ Studying a pattern is also useful for non-anchored patterns that do not have a
+ single fixed starting character. A bitmap of possible starting bytes is
+ created. This speeds up finding a position in the subject at which to start
+ matching.
+ .P
+ The two optimizations just described can be disabled by setting the
+ PCRE_NO_START_OPTIMIZE option when calling \fBpcre_exec()\fP or
+ \fBpcre_dfa_exec()\fP. You might want to do this if your pattern contains
+ callouts or (*MARK), and you want to make use of these facilities in cases
+ where matching fails. See the discussion of PCRE_NO_START_OPTIMIZE
+ .\" HTML <a href="#execoptions">
+ .\" </a>
+ below.
+ .\"
  .
  .
  .\" HTML <a name="localesupport"></a>
-Line 749 
 bytes is created.
+Line 898 
 bytes is created.
  PCRE handles caseless matching, and determines whether characters are letters,
  digits, or whatever, by reference to a set of tables, indexed by character
  value. When running in UTF-8 mode, this applies only to characters with codes
- less than 128. Higher-valued codes never match escapes such as \ew or \ed, but
+ less than 128. By default, higher-valued codes never match escapes such as \ew
- can be tested with \ep if PCRE is built with Unicode character property
+ or \ed, but they can be tested with \ep if PCRE is built with Unicode character
- support. The use of locales with Unicode is discouraged. If you are handling
+ property support. Alternatively, the PCRE_UCP option can be set at compile
- characters with codes greater than 128, you should either use UTF-8 and
+ time; this causes \ew and friends to use Unicode property support instead of
- Unicode, or use locales, but not try to mix the two.
+ built-in tables. The use of locales with Unicode is discouraged. If you are
+ handling characters with codes greater than 128, you should either use UTF-8
+ and Unicode, or use locales, but not try to mix the two.
  .P
  PCRE contains an internal set of tables that are used when the final argument
  of \fBpcre_compile()\fP is NULL. These are sufficient for many applications.
-Line 798 
 one in which it was compiled. Passing ta
+Line 949 
 one in which it was compiled. Passing ta
  below in the section on matching a pattern.
  .
  .
+ .\" HTML <a name="infoaboutpattern"></a>
  .SH "INFORMATION ABOUT A PATTERN"
  .rs
  .sp
-Line 882 
 table indicating a fixed set of bytes fo
+Line 1034 
 table indicating a fixed set of bytes fo
  string, a pointer to the table is returned. Otherwise NULL is returned. The
  fourth argument should point to an \fBunsigned char *\fP variable.
  .sp
+   PCRE_INFO_HASCRORLF
+ .sp
+ Return 1 if the pattern contains any explicit matches for CR or LF characters,
+ otherwise 0. The fourth argument should point to an \fBint\fP variable. An
+ explicit match is either a literal CR or LF character, or \er or \en.
+ .sp
    PCRE_INFO_JCHANGED
  .sp
- Return 1 if the (?J) option setting is used in the pattern, otherwise 0. The
+ Return 1 if the (?J) or (?-J) option setting is used in the pattern, otherwise
- fourth argument should point to an \fBint\fP variable. The (?J) internal option
+. The fourth argument should point to an \fBint\fP variable. (?J) and
- setting changes the local PCRE_DUPNAMES option.
+ (?-J) set and unset the local PCRE_DUPNAMES option, respectively.
  .sp
    PCRE_INFO_LASTLITERAL
  .sp
-Line 898 
 follows something of variable length. Fo
+Line 1056 
 follows something of variable length. Fo
  /^a\ed+z\ed+/ the returned value is "z", but for /^a\edz\ed/ the returned value
  is -1.
  .sp
+   PCRE_INFO_MINLENGTH
+ .sp
+ If the pattern was studied and a minimum length for matching subject strings
+ was computed, its value is returned. Otherwise the returned value is -1. The
+ value is a number of characters, not bytes (this may be relevant in UTF-8
+ mode). The fourth argument should point to an \fBint\fP variable. A
+ non-negative value is a lower bound to the length of any matching string. There
+ may not be any strings of that length that do actually match, but every string
+ that does match is at least that long.
+ .sp
    PCRE_INFO_NAMECOUNT
    PCRE_INFO_NAMEENTRYSIZE
    PCRE_INFO_NAMETABLE
-Line 918 
 entry; both of these return an \fBint\fP
+Line 1086 
 entry; both of these return an \fBint\fP
  length of the longest name. PCRE_INFO_NAMETABLE returns a pointer to the first
  entry of the table (a pointer to \fBchar\fP). The first two bytes of each entry
  are the number of the capturing parenthesis, most significant byte first. The
- rest of the entry is the corresponding name, zero terminated. The names are in
+ rest of the entry is the corresponding name, zero terminated.
- alphabetical order. When PCRE_DUPNAMES is set, duplicate names are in order of
+ .P
- their parentheses numbers. For example, consider the following pattern (assume
+ The names are in alphabetical order. Duplicate names may appear if (?| is used
- PCRE_EXTENDED is set, so white space - including newlines - is ignored):
+ to create multiple groups with the same number, as described in the
+ .\" HTML <a href="pcrepattern.html#dupsubpatternnumber">
+ .\" </a>
+ section on duplicate subpattern numbers
+ .\"
+ in the
+ .\" HREF
+ \fBpcrepattern\fP
+ .\"
+ page. Duplicate names for subpatterns with different numbers are permitted only
+ if PCRE_DUPNAMES is set. In all cases of duplicate names, they appear in the
+ table in the order in which they were found in the pattern. In the absence of
+ (?| this is the order of increasing number; when (?| is used this is not
+ necessarily the case because later subpatterns may have lower numbers.
+ .P
+ As a simple example of the name/number table, consider the following pattern
+ (assume PCRE_EXTENDED is set, so white space - including newlines - is
+ ignored):
  .sp
  .\" JOIN
    (?<date> (?<year>(\ed\ed)?\ed\ed) -
-Line 942 
 different for each compiled pattern.
+Line 1127 
 different for each compiled pattern.
  .sp
    PCRE_INFO_OKPARTIAL
  .sp
- Return 1 if the pattern can be used for partial matching, otherwise 0. The
+ Return 1 if the pattern can be used for partial matching with
- fourth argument should point to an \fBint\fP variable. The
+ \fBpcre_exec()\fP, otherwise 0. The fourth argument should point to an
+ \fBint\fP variable. From release 8.00, this always returns 1, because the
+ restrictions that previously applied to partial matching have been lifted. The
  .\" HREF
  \fBpcrepartial\fP
  .\"
- documentation lists the restrictions that apply to patterns when partial
+ documentation gives details of partial matching.
- matching is used.
  .sp
    PCRE_INFO_OPTIONS
  .sp
-Line 982 
 variable.
+Line 1168 
 variable.
  .sp
    PCRE_INFO_STUDYSIZE
  .sp
- Return the size of the data block pointed to by the \fIstudy_data\fP field in
+ Return the size of the data block pointed to by the \fIstudy_data\fP field in a
- a \fBpcre_extra\fP block. That is, it is the value that was passed to
+ \fBpcre_extra\fP block. If \fBpcre_extra\fP is NULL, or there is no study data,
- \fBpcre_malloc()\fP when PCRE was getting memory into which to place the data
+ zero is returned. The fourth argument should point to a \fBsize_t\fP variable.
- created by \fBpcre_study()\fP. The fourth argument should point to a
+ The \fIstudy_data\fP field is set by \fBpcre_study()\fP to record information
- \fBsize_t\fP variable.
+ that will speed up matching (see the section entitled
+ .\" HTML <a href="#studyingapattern">
+ .\" </a>
+ "Studying a pattern"
+ .\"
+ above). The format of the \fIstudy_data\fP block is private, but its length
+ is made available via this option so that it can be saved and restored (see the
+ .\" HREF
+ \fBpcreprecompile\fP
+ .\"
+ documentation for details).
  .
  .
  .SH "OBSOLETE INFO FUNCTION"
-Line 1047 
 is different. (This seems a highly unlik
+Line 1243 
 is different. (This seems a highly unlik
  .P
  The function \fBpcre_exec()\fP is called to match a subject string against a
  compiled pattern, which is passed in the \fIcode\fP argument. If the
- pattern has been studied, the result of the study should be passed in the
+ pattern was studied, the result of the study should be passed in the
  \fIextra\fP argument. This function is the main matching facility of the
  library, and it operates in a Perl-like manner. For specialist use there is
  also an alternative matching function, which is described
-Line 1081 
 Here is an example of a simple call to \
+Line 1277 
 Here is an example of a simple call to \
      ovector,        /* vector of integers for substring information */
 );            /* number of elements (NOT size in bytes) */
  .
+ .
  .\" HTML <a name="extradata"></a>
  .SS "Extra data for \fBpcre_exec()\fR"
  .rs
-Line 1097 
 fields (not necessarily in this order):
+Line 1294 
 fields (not necessarily in this order):
    unsigned long int \fImatch_limit_recursion\fP;
    void *\fIcallout_data\fP;
    const unsigned char *\fItables\fP;
+   unsigned char **\fImark\fP;
  .sp
  The \fIflags\fP field is a bitmap that specifies which of the other fields
  are set. The flag bits are:
-Line 1106 
 are set. The flag bits are:
+Line 1304 
 are set. The flag bits are:
    PCRE_EXTRA_MATCH_LIMIT_RECURSION
    PCRE_EXTRA_CALLOUT_DATA
    PCRE_EXTRA_TABLES
+   PCRE_EXTRA_MARK
  .sp
  Other flag bits should be set to zero. The \fIstudy_data\fP field is set in the
  \fBpcre_extra\fP block that is returned by \fBpcre_study()\fP, together with
-Line 1115 
 the block by setting the other fields an
+Line 1314 
 the block by setting the other fields an
  The \fImatch_limit\fP field provides a means of preventing PCRE from using up a
  vast amount of resources when running patterns that are not going to match,
  but which have a very large number of possibilities in their search trees. The
- classic example is the use of nested unlimited repeats.
+ classic example is a pattern that uses nested unlimited repeats.
  .P
  Internally, PCRE uses a function called \fBmatch()\fP which it calls repeatedly
  (sometimes recursively). The limit set by \fImatch_limit\fP is imposed on the
-Line 1148 
 with a \fBpcre_extra\fP block in which \
+Line 1347 
 with a \fBpcre_extra\fP block in which \
  PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the \fIflags\fP field. If the limit
  is exceeded, \fBpcre_exec()\fP returns PCRE_ERROR_RECURSIONLIMIT.
  .P
- The \fIpcre_callout\fP field is used in conjunction with the "callout" feature,
+ The \fIcallout_data\fP field is used in conjunction with the "callout" feature,
- which is described in the
+ and is described in the
  .\" HREF
  \fBpcrecallout\fP
  .\"
-Line 1168 
 called. See the
+Line 1367 
 called. See the
  \fBpcreprecompile\fP
  .\"
  documentation for a discussion of saving compiled patterns for later use.
+ .P
+ If PCRE_EXTRA_MARK is set in the \fIflags\fP field, the \fImark\fP field must
+ be set to point to a \fBchar *\fP variable. If the pattern contains any
+ backtracking control verbs such as (*MARK:NAME), and the execution ends up with
+ a name to pass back, a pointer to the name string (zero terminated) is placed
+ in the variable pointed to by the \fImark\fP field. The names are within the
+ compiled pattern; if you wish to retain such a name you must copy it before
+ freeing the memory of a compiled pattern. If there is no name to pass back, the
+ variable pointed to by the \fImark\fP field set to NULL. For details of the
+ backtracking control verbs, see the section entitled
+ .\" HTML <a href="pcrepattern#backtrackcontrol">
+ .\" </a>
+ "Backtracking control"
+ .\"
+ in the
+ .\" HREF
+ \fBpcrepattern\fP
+ .\"
+ documentation.
+ .
  .
+ .\" HTML <a name="execoptions"></a>
  .SS "Option bits for \fBpcre_exec()\fP"
  .rs
  .sp
  The unused bits of the \fIoptions\fP argument for \fBpcre_exec()\fP must be
  zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_\fIxxx\fP,
- PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK and PCRE_PARTIAL.
+ PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
+ PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and
+ PCRE_PARTIAL_HARD.
  .sp
    PCRE_ANCHORED
  .sp
-Line 1183 
 matching position. If a pattern was comp
+Line 1405 
 matching position. If a pattern was comp
  to be anchored by virtue of its contents, it cannot be made unachored at
  matching time.
  .sp
+   PCRE_BSR_ANYCRLF
+   PCRE_BSR_UNICODE
+ .sp
+ These options (which are mutually exclusive) control what the \eR escape
+ sequence matches. The choice is either to match only CR, LF, or CRLF, or to
+ match any Unicode newline sequence. These options override the choice that was
+ made or defaulted when the pattern was compiled.
+ .sp
    PCRE_NEWLINE_CR
    PCRE_NEWLINE_LF
    PCRE_NEWLINE_CRLF
-Line 1194 
 the pattern was compiled. For details, s
+Line 1424 
 the pattern was compiled. For details, s
  \fBpcre_compile()\fP above. During matching, the newline choice affects the
  behaviour of the dot, circumflex, and dollar metacharacters. It may also alter
  the way the match position is advanced after a match failure for an unanchored
- pattern. When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is
+ pattern.
- set, and a match attempt fails when the current position is at a CRLF sequence,
+ .P
- the match position is advanced by two characters instead of one, in other
+ When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is set, and a
- words, to after the CRLF.
+ match attempt for an unanchored pattern fails when the current position is at a
- .P
+ CRLF sequence, and the pattern contains no explicit matches for CR or LF
- Anomalous effects can occur when CRLF is a valid newline sequence and explicit
+ characters, the match position is advanced by two characters instead of one, in
- \er or \en escapes appear in the pattern. For example, the string "\er\enA"
+ other words, to after the CRLF.
- matches the unanchored pattern \enA but not [X\en]A. This happens because, in
+ .P
- the first case, PCRE knows that the match must start with \en, and so it skips
+ The above rule is a compromise that makes the most common cases work as
- there before trying to match. In the second case, it has no knowledge about the
+ expected. For example, if the pattern is .+A (and the PCRE_DOTALL option is not
- starting character, so it starts matching at the beginning of the string, and
+ set), it does not match the string "\er\enA" because, after failing at the
- on failing, skips over the CRLF as described above. However, if the pattern is
+ start, it skips both the CR and the LF before retrying. However, the pattern
- studied, the match succeeds, because then PCRE once again knows where to start.
+ [\er\en]A does match that string, because it contains an explicit CR or LF
+ reference, and so advances only by one character after the first failure.
+ .P
+ An explicit match for CR of LF is either a literal appearance of one of those
+ characters, or one of the \er or \en escape sequences. Implicit matches such as
+ [^X] do not count, nor does \es (which includes CR and LF in the characters
+ that it matches).
+ .P
+ Notwithstanding the above, anomalous effects may still occur when CRLF is a
+ valid newline sequence and explicit \er or \en escapes appear in the pattern.
  .sp
    PCRE_NOTBOL
  .sp
-Line 1232 
 match the empty string, the entire match
+Line 1471 
 match the empty string, the entire match
  .sp
    a?b?
  .sp
- is applied to a string not beginning with "a" or "b", it matches the empty
+ is applied to a string not beginning with "a" or "b", it matches an empty
  string at the start of the subject. With PCRE_NOTEMPTY set, this match is not
  valid, so PCRE searches further into the string for occurrences of "a" or "b".
- .P
+ .sp
- Perl has no direct equivalent of PCRE_NOTEMPTY, but it does make a special case
+   PCRE_NOTEMPTY_ATSTART
- of a pattern match of the empty string within its \fBsplit()\fP function, and
+ .sp
- when using the /g modifier. It is possible to emulate Perl's behaviour after
+ This is like PCRE_NOTEMPTY, except that an empty string match that is not at
- matching a null string by first trying the match again at the same offset with
+ the start of the subject is permitted. If the pattern is anchored, such a match
- PCRE_NOTEMPTY and PCRE_ANCHORED, and then if that fails by advancing the
+ can occur only if the pattern contains \eK.
- starting offset (see below) and trying an ordinary match again. There is some
+ .P
- code that demonstrates how to do this in the \fIpcredemo.c\fP sample program.
+ Perl has no direct equivalent of PCRE_NOTEMPTY or PCRE_NOTEMPTY_ATSTART, but it
+ does make a special case of a pattern match of the empty string within its
+ \fBsplit()\fP function, and when using the /g modifier. It is possible to
+ emulate Perl's behaviour after matching a null string by first trying the match
+ again at the same offset with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then
+ if that fails, by advancing the starting offset (see below) and trying an
+ ordinary match again. There is some code that demonstrates how to do this in
+ the
+ .\" HREF
+ \fBpcredemo\fP
+ .\"
+ sample program. In the most general case, you have to check to see if the
+ newline convention recognizes CRLF as a newline, and if so, and the current
+ character is CR followed by LF, advance the starting offset by two characters
+ instead of one.
+ .sp
+   PCRE_NO_START_OPTIMIZE
+ .sp
+ There are a number of optimizations that \fBpcre_exec()\fP uses at the start of
+ a match, in order to speed up the process. For example, if it is known that an
+ unanchored match must start with a specific character, it searches the subject
+ for that character, and fails immediately if it cannot find it, without
+ actually running the main matching function. This means that a special item
+ such as (*COMMIT) at the start of a pattern is not considered until after a
+ suitable starting point for the match has been found. When callouts or (*MARK)
+ items are in use, these "start-up" optimizations can cause them to be skipped
+ if the pattern is never actually used. The start-up optimizations are in effect
+ a pre-scan of the subject that takes place before the pattern is run.
+ .P
+ The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, possibly
+ causing performance to suffer, but ensuring that in cases where the result is
+ "no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK)
+ are considered at every possible starting position in the subject string. If
+ PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching
+ time.
+ .P
+ Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching operation.
+ Consider the pattern
+ .sp
+   (*COMMIT)ABC
+ .sp
+ When this is compiled, PCRE records the fact that a match must start with the
+ character "A". Suppose the subject string is "DEFABC". The start-up
+ optimization scans along the subject, finds "A" and runs the first match
+ attempt from there. The (*COMMIT) item means that the pattern must match the
+ current starting position, which in this case, it does. However, if the same
+ match is run with PCRE_NO_START_OPTIMIZE set, the initial scan along the
+ subject string does not happen. The first match attempt is run starting from
+ "D" and when this fails, (*COMMIT) prevents any further matches being tried, so
+ the overall result is "no match". If the pattern is studied, more start-up
+ optimizations may be used. For example, a minimum length for the subject may be
+ recorded. Consider the pattern
+ .sp
+   (*MARK:A)(X|Y)
+ .sp
+ The minimum length for a match is one character. If the subject is "ABC", there
+ will be attempts to match "ABC", "BC", "C", and then finally an empty string.
+ If the pattern is studied, the final attempt does not take place, because PCRE
+ knows that the subject is too short, and so the (*MARK) is never encountered.
+ In this case, studying the pattern does not affect the overall match result,
+ which is still "no match", but it does affect the auxiliary information that is
+ returned.
  .sp
    PCRE_NO_UTF8_CHECK
  .sp
-Line 1260 
 in the main
+Line 1560 
 in the main
  \fBpcre\fP
  .\"
  page. If an invalid UTF-8 sequence of bytes is found, \fBpcre_exec()\fP returns
- the error PCRE_ERROR_BADUTF8. If \fIstartoffset\fP contains an invalid value,
+ the error PCRE_ERROR_BADUTF8 or, if PCRE_PARTIAL_HARD is set and the problem is
- PCRE_ERROR_BADUTF8_OFFSET is returned.
+ a truncated UTF-8 character at the end of the subject, PCRE_ERROR_SHORTUTF8. In
+ both cases, information about the precise nature of the error may also be
+ returned (see the descriptions of these errors in the section entitled \fIError
+ return values from\fP \fBpcre_exec()\fP
+ .\" HTML <a href="#errorlist">
+ .\" </a>
+ below).
+ .\"
+ If \fIstartoffset\fP contains a value that does not point to the start of a
+ UTF-8 character (or to the end of the subject), PCRE_ERROR_BADUTF8_OFFSET is
+ returned.
  .P
  If you already know that your subject is valid, and you want to skip these
  checks for performance reasons, you can set the PCRE_NO_UTF8_CHECK option when
  calling \fBpcre_exec()\fP. You might want to do this for the second and
  subsequent calls to \fBpcre_exec()\fP if you are making repeated calls to find
  all the matches in a single subject string. However, you should be sure that
- the value of \fIstartoffset\fP points to the start of a UTF-8 character. When
+ the value of \fIstartoffset\fP points to the start of a UTF-8 character (or the
- PCRE_NO_UTF8_CHECK is set, the effect of passing an invalid UTF-8 string as a
+ end of the subject). When PCRE_NO_UTF8_CHECK is set, the effect of passing an
- subject, or a value of \fIstartoffset\fP that does not point to the start of a
+ invalid UTF-8 string as a subject or an invalid value of \fIstartoffset\fP is
- UTF-8 character, is undefined. Your program may crash.
+ undefined. Your program may crash.
  .sp
-   PCRE_PARTIAL
+   PCRE_PARTIAL_HARD
- .sp
+   PCRE_PARTIAL_SOFT
- This option turns on the partial matching feature. If the subject string fails
+ .sp
- to match the pattern, but at some point during the matching process the end of
+ These options turn on the partial matching feature. For backwards
- the subject was reached (that is, the subject partially matches the pattern and
+ compatibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial match
- the failure to match occurred only because there were not enough subject
+ occurs if the end of the subject string is reached successfully, but there are
- characters), \fBpcre_exec()\fP returns PCRE_ERROR_PARTIAL instead of
+ not enough subject characters to complete the match. If this happens when
- PCRE_ERROR_NOMATCH. When PCRE_PARTIAL is used, there are restrictions on what
+ PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set, matching continues by
- may appear in the pattern. These are discussed in the
+ testing any remaining alternatives. Only if no complete match can be found is
+ PCRE_ERROR_PARTIAL returned instead of PCRE_ERROR_NOMATCH. In other words,
+ PCRE_PARTIAL_SOFT says that the caller is prepared to handle a partial match,
+ but only if no complete match can be found.
+ .P
+ If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this case, if a
+ partial match is found, \fBpcre_exec()\fP immediately returns
+ PCRE_ERROR_PARTIAL, without considering any other alternatives. In other words,
+ when PCRE_PARTIAL_HARD is set, a partial match is considered to be more
+ important that an alternative complete match.
+ .P
+ In both cases, the portion of the string that was inspected when the partial
+ match was found is set as the first matching string. There is a more detailed
+ discussion of partial and multi-segment matching, with examples, in the
  .\" HREF
  \fBpcrepartial\fP
  .\"
  documentation.
  .
+ .
  .SS "The string to be matched by \fBpcre_exec()\fP"
  .rs
  .sp
  The subject string is passed to \fBpcre_exec()\fP as a pointer in
- \fIsubject\fP, a length in \fIlength\fP, and a starting byte offset in
+ \fIsubject\fP, a length (in bytes) in \fIlength\fP, and a starting byte offset
- \fIstartoffset\fP. In UTF-8 mode, the byte offset must point to the start of a
+ in \fIstartoffset\fP. If this is negative or greater than the length of the
- UTF-8 character. Unlike the pattern string, the subject may contain binary zero
+ subject, \fBpcre_exec()\fP returns PCRE_ERROR_BADOFFSET. When the starting
- bytes. When the starting offset is zero, the search for a match starts at the
+ offset is zero, the search for a match starts at the beginning of the subject,
- beginning of the subject, and this is by far the most common case.
+ and this is by far the most common case. In UTF-8 mode, the byte offset must
+ point to the start of a UTF-8 character (or the end of the subject). Unlike the
+ pattern string, the subject may contain binary zero bytes.
  .P
  A non-zero starting offset is useful when searching for another match in the
  same subject by calling \fBpcre_exec()\fP again after a previous success.
-Line 1315 
 start of the subject, which is deemed to
+Line 1641 
 start of the subject, which is deemed to
  set to 4, it finds the second occurrence of "iss" because it is able to look
  behind the starting point to discover that it is preceded by a letter.
  .P
+ Finding all the matches in a subject is tricky when the pattern can match an
+ empty string. It is possible to emulate Perl's /g behaviour by first trying the
+ match again at the same offset, with the PCRE_NOTEMPTY_ATSTART and
+ PCRE_ANCHORED options, and then if that fails, advancing the starting offset
+ and trying an ordinary match again. There is some code that demonstrates how to
+ do this in the
+ .\" HREF
+ \fBpcredemo\fP
+ .\"
+ sample program. In the most general case, you have to check to see if the
+ newline convention recognizes CRLF as a newline, and if so, and the current
+ character is CR followed by LF, advance the starting offset by two characters
+ instead of one.
+ .P
  If a non-zero starting offset is passed when the pattern is anchored, one
  attempt to match at the given offset is made. This can only succeed if the
  pattern does not require the match to be at the start of the subject.
  .
+ .
  .SS "How \fBpcre_exec()\fP returns captured substrings"
  .rs
  .sp
-Line 1329 
 pattern. Following the usage in Jeffrey
+Line 1670 
 pattern. Following the usage in Jeffrey
  a fragment of a pattern that picks out a substring. PCRE supports several other
  kinds of parenthesized subpattern that do not cause substrings to be captured.
  .P
- Captured substrings are returned to the caller via a vector of integer offsets
+ Captured substrings are returned to the caller via a vector of integers whose
- whose address is passed in \fIovector\fP. The number of elements in the vector
+ address is passed in \fIovector\fP. The number of elements in the vector is
- is passed in \fIovecsize\fP, which must be a non-negative number. \fBNote\fP:
+ passed in \fIovecsize\fP, which must be a non-negative number. \fBNote\fP: this
- this argument is NOT the size of \fIovector\fP in bytes.
+ argument is NOT the size of \fIovector\fP in bytes.
  .P
  The first two-thirds of the vector is used to pass back captured substrings,
  each substring using a pair of integers. The remaining third of the vector is
  used as workspace by \fBpcre_exec()\fP while matching capturing subpatterns,
- and is not available for passing back information. The length passed in
+ and is not available for passing back information. The number passed in
  \fIovecsize\fP should always be a multiple of three. If it is not, it is
  rounded down.
  .P
  When a match is successful, information about captured substrings is returned
  in pairs of integers, starting at the beginning of \fIovector\fP, and
- continuing up to two-thirds of its length at the most. The first element of a
+ continuing up to two-thirds of its length at the most. The first element of
- pair is set to the offset of the first character in a substring, and the second
+ each pair is set to the byte offset of the first character in a substring, and
- is set to the offset of the first character after the end of a substring. The
+ the second is set to the byte offset of the first character after the end of a
- first pair, \fIovector[0]\fP and \fIovector[1]\fP, identify the portion of the
+ substring. \fBNote\fP: these values are always byte offsets, even in UTF-8
- subject string matched by the entire pattern. The next pair is used for the
+ mode. They are not character counts.
- first capturing subpattern, and so on. The value returned by \fBpcre_exec()\fP
+ .P
- is one more than the highest numbered pair that has been set. For example, if
+ The first pair of integers, \fIovector[0]\fP and \fIovector[1]\fP, identify the
- two substrings have been captured, the returned value is 3. If there are no
+ portion of the subject string matched by the entire pattern. The next pair is
- capturing subpatterns, the return value from a successful match is 1,
+ used for the first capturing subpattern, and so on. The value returned by
- indicating that just the first pair of offsets has been set.
+ \fBpcre_exec()\fP is one more than the highest numbered pair that has been set.
+ For example, if two substrings have been captured, the returned value is 3. If
+ there are no capturing subpatterns, the return value from a successful match is
+, indicating that just the first pair of offsets has been set.
  .P
  If a capturing subpattern is matched repeatedly, it is the last portion of the
  string that it matched that is returned.
  .P
  If the vector is too small to hold all the captured substring offsets, it is
  used as far as possible (up to two-thirds of its length), and the function
- returns a value of zero. In particular, if the substring offsets are not of
+ returns a value of zero. If the substring offsets are not of interest,
- interest, \fBpcre_exec()\fP may be called with \fIovector\fP passed as NULL and
+ \fBpcre_exec()\fP may be called with \fIovector\fP passed as NULL and
  \fIovecsize\fP as zero. However, if the pattern contains back references and
  the \fIovector\fP is not big enough to remember the related substrings, PCRE
  has to get additional memory for use during matching. Thus it is usually
  advisable to supply an \fIovector\fP.
  .P
- The \fBpcre_info()\fP function can be used to find out how many capturing
+ The \fBpcre_fullinfo()\fP function can be used to find out how many capturing
  subpatterns there are in a compiled pattern. The smallest size for
  \fIovector\fP that will allow for \fIn\fP captured substrings, in addition to
  the offsets of the substring matched by the whole pattern, is (\fIn\fP+1)*3.
-Line 1382 
 Offset values that correspond to unused
+Line 1726 
 Offset values that correspond to unused
  expression are also set to -1. For example, if the string "abc" is matched
  against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not matched. The
  return from the function is 2, because the highest used capturing subpattern
- number is 1. However, you can refer to the offsets for the second and third
+ number is 1, and the offsets for for the second and third capturing subpatterns
- capturing subpatterns if you wish (assuming the vector is large enough, of
+ (assuming the vector is large enough, of course) are set to -1.
- course).
+ .P
+ \fBNote\fP: Elements of \fIovector\fP that do not correspond to capturing
+ parentheses in the pattern are never changed. That is, if a pattern contains
+ \fIn\fP capturing parentheses, no more than \fIovector[0]\fP to
+ \fIovector[2n+1]\fP are set by \fBpcre_exec()\fP. The other elements retain
+ whatever values they previously had.
  .P
  Some convenience functions are provided for extracting the captured substrings
  as separate strings. These are described below.
  .
+ .
  .\" HTML <a name="errorlist"></a>
  .SS "Error return values from \fBpcre_exec()\fP"
  .rs
-Line 1430 
 If a pattern contains back references, b
+Line 1780 
 If a pattern contains back references, b
  gets a block of memory at the start of matching to use for this purpose. If the
  call via \fBpcre_malloc()\fP fails, this error is given. The memory is
  automatically freed at the end of matching.
+ .P
+ This error is also given if \fBpcre_stack_malloc()\fP fails in
+ \fBpcre_exec()\fP. This can happen only when PCRE has been compiled with
+ \fB--disable-stack-for-recursion\fP.
  .sp
    PCRE_ERROR_NOSUBSTRING    (-7)
  .sp
-Line 1454 
 documentation for details.
+Line 1808 
 documentation for details.
  .sp
    PCRE_ERROR_BADUTF8        (-10)
  .sp
- A string that contains an invalid UTF-8 byte sequence was passed as a subject.
+ A string that contains an invalid UTF-8 byte sequence was passed as a subject,
+ and the PCRE_NO_UTF8_CHECK option was not set. If the size of the output vector
+ (\fIovecsize\fP) is at least 2, the byte offset to the start of the the invalid
+ UTF-8 character is placed in the first element, and a reason code is placed in
+ the second element. The reason codes are listed in the
+ .\" HTML <a href="#badutf8reasons">
+ .\" </a>
+ following section.
+ .\"
+ For backward compatibility, if PCRE_PARTIAL_HARD is set and the problem is a
+ truncated UTF-8 character at the end of the subject (reason codes 1 to 5),
+ PCRE_ERROR_SHORTUTF8 is returned instead of PCRE_ERROR_BADUTF8.
  .sp
    PCRE_ERROR_BADUTF8_OFFSET (-11)
  .sp
- The UTF-8 byte sequence that was passed as a subject was valid, but the value
+ The UTF-8 byte sequence that was passed as a subject was checked and found to
- of \fIstartoffset\fP did not point to the beginning of a UTF-8 character.
+ be valid (the PCRE_NO_UTF8_CHECK option was not set), but the value of
+ \fIstartoffset\fP did not point to the beginning of a UTF-8 character or the
+ end of the subject.
  .sp
    PCRE_ERROR_PARTIAL        (-12)
  .sp
-Line 1471 
 documentation for details of partial mat
+Line 1838 
 documentation for details of partial mat
  .sp
    PCRE_ERROR_BADPARTIAL     (-13)
  .sp
- The PCRE_PARTIAL option was used with a compiled pattern containing items that
+ This code is no longer in use. It was formerly returned when the PCRE_PARTIAL
- are not supported for partial matching. See the
+ option was used with a compiled pattern containing items that were not
- .\" HREF
+ supported for partial matching. From release 8.00 onwards, there are no
- \fBpcrepartial\fP
+ restrictions on partial matching.
- .\"
- documentation for details of partial matching.
  .sp
    PCRE_ERROR_INTERNAL       (-14)
  .sp
-Line 1496 
 description above.
+Line 1861 
 description above.
    PCRE_ERROR_BADNEWLINE     (-23)
  .sp
  An invalid combination of PCRE_NEWLINE_\fIxxx\fP options was given.
+ .sp
+   PCRE_ERROR_BADOFFSET      (-24)
+ .sp
+ The value of \fIstartoffset\fP was negative or greater than the length of the
+ subject, that is, the value in \fIlength\fP.
+ .sp
+   PCRE_ERROR_SHORTUTF8      (-25)
+ .sp
+ This error is returned instead of PCRE_ERROR_BADUTF8 when the subject string
+ ends with a truncated UTF-8 character and the PCRE_PARTIAL_HARD option is set.
+ Information about the failure is returned as for PCRE_ERROR_BADUTF8. It is in
+ fact sufficient to detect this case, but this special error code for
+ PCRE_PARTIAL_HARD precedes the implementation of returned information; it is
+ retained for backwards compatibility.
+ .sp
+   PCRE_ERROR_RECURSELOOP    (-26)
+ .sp
+ This error is returned when \fBpcre_exec()\fP detects a recursion loop within
+ the pattern. Specifically, it means that either the whole pattern or a
+ subpattern has been called recursively for the second time at the same position
+ in the subject string. Some simple patterns that might do this are detected and
+ faulted at compile time, but more complicated cases, in particular mutual
+ recursions between two different subpatterns, cannot be detected until run
+ time.
  .P
  Error numbers -16 to -20 and -22 are not used by \fBpcre_exec()\fP.
  .
  .
+ .\" HTML <a name="badutf8reasons"></a>
+ .SS "Reason codes for invalid UTF-8 strings"
+ .rs
+ .sp
+ When \fBpcre_exec()\fP returns either PCRE_ERROR_BADUTF8 or
+ PCRE_ERROR_SHORTUTF8, and the size of the output vector (\fIovecsize\fP) is at
+ least 2, the offset of the start of the invalid UTF-8 character is placed in
+ the first output vector element (\fIovector[0]\fP) and a reason code is placed
+ in the second element (\fIovector[1]\fP). The reason codes are given names in
+ the \fBpcre.h\fP header file:
+ .sp
+   PCRE_UTF8_ERR1
+   PCRE_UTF8_ERR2
+   PCRE_UTF8_ERR3
+   PCRE_UTF8_ERR4
+   PCRE_UTF8_ERR5
+ .sp
+ The string ends with a truncated UTF-8 character; the code specifies how many
+ bytes are missing (1 to 5). Although RFC 3629 restricts UTF-8 characters to be
+ no longer than 4 bytes, the encoding scheme (originally defined by RFC 2279)
+ allows for up to 6 bytes, and this is checked first; hence the possibility of
+or 5 missing bytes.
+ .sp
+   PCRE_UTF8_ERR6
+   PCRE_UTF8_ERR7
+   PCRE_UTF8_ERR8
+   PCRE_UTF8_ERR9
+   PCRE_UTF8_ERR10
+ .sp
+ The two most significant bits of the 2nd, 3rd, 4th, 5th, or 6th byte of the
+ character do not have the binary value 0b10 (that is, either the most
+ significant bit is 0, or the next bit is 1).
+ .sp
+   PCRE_UTF8_ERR11
+   PCRE_UTF8_ERR12
+ .sp
+ A character that is valid by the RFC 2279 rules is either 5 or 6 bytes long;
+ these code points are excluded by RFC 3629.
+ .sp
+   PCRE_UTF8_ERR13
+ .sp
+ A 4-byte character has a value greater than 0x10fff; these code points are
+ excluded by RFC 3629.
+ .sp
+   PCRE_UTF8_ERR14
+ .sp
+ A 3-byte character has a value in the range 0xd800 to 0xdfff; this range of
+ code points are reserved by RFC 3629 for use with UTF-16, and so are excluded
+ from UTF-8.
+ .sp
+   PCRE_UTF8_ERR15
+   PCRE_UTF8_ERR16
+   PCRE_UTF8_ERR17
+   PCRE_UTF8_ERR18
+   PCRE_UTF8_ERR19
+ .sp
+ A 2-, 3-, 4-, 5-, or 6-byte character is "overlong", that is, it codes for a
+ value that can be represented by fewer bytes, which is invalid. For example,
+ the two bytes 0xc0, 0xae give the value 0x2e, whose correct coding uses just
+ one byte.
+ .sp
+   PCRE_UTF8_ERR20
+ .sp
+ The two most significant bits of the first byte of a character have the binary
+ value 0b10 (that is, the most significant bit is 1 and the second is 0). Such a
+ byte can only validly occur as the second or subsequent byte of a multi-byte
+ character.
+ .sp
+   PCRE_UTF8_ERR21
+ .sp
+ The first byte of a character has the value 0xfe or 0xff. These values can
+ never occur in a valid UTF-8 string.
+ .
+ .
  .SH "EXTRACTING CAPTURED SUBSTRINGS BY NUMBER"
  .rs
  .sp
-Line 1646 
 These functions call \fBpcre_get_stringn
+Line 2109 
 These functions call \fBpcre_get_stringn
  then call \fBpcre_copy_substring()\fP or \fBpcre_get_substring()\fP, as
  appropriate. \fBNOTE:\fP If PCRE_DUPNAMES is set and there are duplicate names,
  the behaviour may not be what you want (see the next section).
- .
+ .P
+ \fBWarning:\fP If the pattern uses the (?| feature to set up multiple
+ subpatterns with the same number, as described in the
+ .\" HTML <a href="pcrepattern.html#dupsubpatternnumber">
+ .\" </a>
+ section on duplicate subpattern numbers
+ .\"
+ in the
+ .\" HREF
+ \fBpcrepattern\fP
+ .\"
+ page, you cannot use names to distinguish the different subpatterns, because
+ names are not included in the compiled code. The matching process uses only
+ numbers. For this reason, the use of different names for subpatterns of the
+ same number causes an error at compile time.
  .
  .SH "DUPLICATE SUBPATTERN NAMES"
  .rs
-Line 1656 
 the behaviour may not be what you want (
+Line 2133 
 the behaviour may not be what you want (
  .B const char *\fIname\fP, char **\fIfirst\fP, char **\fIlast\fP);
  .PP
  When a pattern is compiled with the PCRE_DUPNAMES option, names for subpatterns
- are not required to be unique. Normally, patterns with duplicate names are such
+ are not required to be unique. (Duplicate names are always allowed for
- that in any one match, only one of the named subpatterns participates. An
+ subpatterns with the same number, created by using the (?| feature. Indeed, if
- example is shown in the
+ such subpatterns are named, they are required to use the same names.)
+ .P
+ Normally, patterns with duplicate names are such that in any one match, only
+ one of the named subpatterns participates. An example is shown in the
  .\" HREF
  \fBpcrepattern\fP
  .\"
-Line 1678 
 fourth are pointers to variables which a
+Line 2158 
 fourth are pointers to variables which a
  has run, they point to the first and last entries in the name-to-number table
  for the given name. The function itself returns the length of each entry, or
  PCRE_ERROR_NOSUBSTRING (-7) if there are none. The format of the table is
- described above in the section entitled \fIInformation about a pattern\fP.
+ described above in the section entitled \fIInformation about a pattern\fP
+ .\" HTML <a href="#infoaboutpattern">
+ .\" </a>
+ above.
+ .\"
  Given all the relevant entries for the name, you can extract each of their
  numbers, and hence the captured data, if any.
  .
-Line 1722 
 a compiled pattern, using a matching alg
+Line 2206 
 a compiled pattern, using a matching alg
  just once, and does not backtrack. This has different characteristics to the
  normal algorithm, and is not compatible with Perl. Some of the features of PCRE
  patterns are not supported. Nevertheless, there are times when this kind of
- matching can be useful. For a discussion of the two matching algorithms, see
+ matching can be useful. For a discussion of the two matching algorithms, and a
- the
+ list of features that \fBpcre_dfa_exec()\fP does not support, see the
  .\" HREF
  \fBpcrematching\fP
  .\"
-Line 1762 
 Here is an example of a simple call to \
+Line 2246 
 Here is an example of a simple call to \
  .sp
  The unused bits of the \fIoptions\fP argument for \fBpcre_dfa_exec()\fP must be
  zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_\fIxxx\fP,
- PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL,
+ PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
- PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last three of these are
+ PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF, PCRE_BSR_UNICODE, PCRE_NO_START_OPTIMIZE,
- the same as for \fBpcre_exec()\fP, so their description is not repeated here.
+ PCRE_PARTIAL_HARD, PCRE_PARTIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART.
- .sp
+ All but the last four of these are exactly the same as for \fBpcre_exec()\fP,
-   PCRE_PARTIAL
+ so their description is not repeated here.
  .sp
- This has the same general effect as it does for \fBpcre_exec()\fP, but the
+   PCRE_PARTIAL_HARD
- details are slightly different. When PCRE_PARTIAL is set for
+   PCRE_PARTIAL_SOFT
- \fBpcre_dfa_exec()\fP, the return code PCRE_ERROR_NOMATCH is converted into
+ .sp
- PCRE_ERROR_PARTIAL if the end of the subject is reached, there have been no
+ These have the same general effect as they do for \fBpcre_exec()\fP, but the
- complete matches, but there is still at least one matching possibility. The
+ details are slightly different. When PCRE_PARTIAL_HARD is set for
- portion of the string that provided the partial match is set as the first
+ \fBpcre_dfa_exec()\fP, it returns PCRE_ERROR_PARTIAL if the end of the subject
- matching string.
+ is reached and there is still at least one matching possibility that requires
+ additional characters. This happens even if some complete matches have also
+ been found. When PCRE_PARTIAL_SOFT is set, the return code PCRE_ERROR_NOMATCH
+ is converted into PCRE_ERROR_PARTIAL if the end of the subject is reached,
+ there have been no complete matches, but there is still at least one matching
+ possibility. The portion of the string that was inspected when the longest
+ partial match was found is set as the first matching string in both cases.
+ There is a more detailed discussion of partial and multi-segment matching, with
+ examples, in the
+ .\" HREF
+ \fBpcrepartial\fP
+ .\"
+ documentation.
  .sp
    PCRE_DFA_SHORTEST
  .sp
-Line 1785 
 matching point in the subject string.
+Line 2281 
 matching point in the subject string.
  .sp
    PCRE_DFA_RESTART
  .sp
- When \fBpcre_dfa_exec()\fP is called with the PCRE_PARTIAL option, and returns
+ When \fBpcre_dfa_exec()\fP returns a partial match, it is possible to call it
- a partial match, it is possible to call it again, with additional subject
+ again, with additional subject characters, and have it continue with the same
- characters, and have it continue with the same match. The PCRE_DFA_RESTART
+ match. The PCRE_DFA_RESTART option requests this action; when it is set, the
- option requests this action; when it is set, the \fIworkspace\fP and
+ \fIworkspace\fP and \fIwscount\fP options must reference the same vector as
- \fIwscount\fP options must reference the same vector as before because data
+ before because data about the match so far is left in them after a partial
- about the match so far is left in them after a partial match. There is more
+ match. There is more discussion of this facility in the
- discussion of this facility in the
  .\" HREF
  \fBpcrepartial\fP
  .\"
  documentation.
  .
+ .
  .SS "Successful returns from \fBpcre_dfa_exec()\fP"
  .rs
  .sp
-Line 1830 
 matching string is given first. If there
+Line 2326 
 matching string is given first. If there
  \fIovector\fP, the yield of the function is zero, and the vector is filled with
  the longest matches.
  .
+ .
  .SS "Error returns from \fBpcre_dfa_exec()\fP"
  .rs
  .sp
-Line 1895 
 Cambridge CB2 3QH, England.
+Line 2392 
 Cambridge CB2 3QH, England.
  .rs
  .sp
  .nf
- Last updated: 20 August 2007
+ Last updated: 13 August 2011
- Copyright (c) 1997-2007 University of Cambridge.
+ Copyright (c) 1997-2011 University of Cambridge.
  .fi

 Legend:



Removed from v.225
 


changed lines


 
Added in v.659
 Legend:



Removed from v.225
 


changed lines


 
Added in v.659
-Removed from v.225
+Added in v.659

webmaster@exim.org	ViewVC Help
Powered by ViewVC 1.1.12