/[pcre]/code/trunk/doc/pcreapi.3

Diff of /code/trunk/doc/pcreapi.3

Parent Directory | Revision Log | View Patch Patch

-revision 526 by ph10,
Wed May 26 10:54:18 2010 UTC
+revision 579 by ph10,
Wed Nov 24 17:39:25 2010 UTC
 Line 428 
 within the pattern (see the detailed des
  documentation). For those options that can be different in different parts of
  the pattern, the contents of the \fIoptions\fP argument specifies their
  settings at the start of compilation and execution. The PCRE_ANCHORED,
- PCRE_BSR_\fIxxx\fP, and PCRE_NEWLINE_\fIxxx\fP options can be set at the time
+ PCRE_BSR_\fIxxx\fP, PCRE_NEWLINE_\fIxxx\fP, PCRE_NO_UTF8_CHECK, and
- of matching as well as at compile time.
+ PCRE_NO_START_OPT options can be set at the time of matching as well as at
+ compile time.
  .P
  If \fIerrptr\fP is NULL, \fBpcre_compile()\fP returns NULL immediately.
  Otherwise, if compilation of a pattern fails, \fBpcre_compile()\fP returns
  NULL, and sets the variable pointed to by \fIerrptr\fP to point to a textual
  error message. This is a static string that is part of the library. You must
- not try to free it. The byte offset from the start of the pattern to the
+ not try to free it. The offset from the start of the pattern to the byte that
- character that was being processed when the error was discovered is placed in
+ was being processed when the error was discovered is placed in the variable
- the variable pointed to by \fIerroffset\fP, which must not be NULL. If it is,
+ pointed to by \fIerroffset\fP, which must not be NULL. If it is, an immediate
- an immediate error is given. Some errors are not detected until checks are
+ error is given. Some errors are not detected until checks are carried out when
- carried out when the whole pattern has been scanned; in this case the offset is
+ the whole pattern has been scanned; in this case the offset is set to the end
- set to the end of the pattern.
+ of the pattern.
+ .P
+ Note that the offset is in bytes, not characters, even in UTF-8 mode. It may
+ point into the middle of a UTF-8 character (for example, when
+ PCRE_ERROR_BADUTF8 is returned for an invalid UTF-8 string).
  .P
  If \fBpcre_compile2()\fP is used instead of \fBpcre_compile()\fP, and the
  \fIerrorcodeptr\fP argument is not NULL, a non-zero error code number is
-Line 519 
 pattern.
+Line 524 
 pattern.
  .sp
    PCRE_DOTALL
  .sp
- If this bit is set, a dot metacharater in the pattern matches all characters,
+ If this bit is set, a dot metacharacter in the pattern matches a character of
- including those that indicate newline. Without it, a dot does not match when
+ any value, including one that indicates a newline. However, it only ever
- the current position is at a newline. This option is equivalent to Perl's /s
+ matches one character, even if newlines are coded as CRLF. Without this option,
- option, and it can be changed within a pattern by a (?s) option setting. A
+ a dot does not match when the current position is at a newline. This option is
- negative class such as [^a] always matches newline characters, independent of
+ equivalent to Perl's /s option, and it can be changed within a pattern by a
- the setting of this option.
+ (?s) option setting. A negative class such as [^a] always matches newline
+ characters, independent of the setting of this option.
  .sp
    PCRE_DUPNAMES
  .sp
-Line 546 
 unescaped # outside a character class an
+Line 552 
 unescaped # outside a character class an
  ignored. This is equivalent to Perl's /x option, and it can be changed within a
  pattern by a (?x) option setting.
  .P
+ Which characters are interpreted as newlines
+ is controlled by the options passed to \fBpcre_compile()\fP or by a special
+ sequence at the start of the pattern, as described in the section entitled
+ .\" HTML <a href="pcrepattern.html#newlines">
+ .\" </a>
+ "Newline conventions"
+ .\"
+ in the \fBpcrepattern\fP documentation. Note that the end of this type of
+ comment is a literal newline sequence in the pattern; escape sequences that
+ happen to represent a newline do not count.
+ .P
  This option makes it possible to include comments inside complicated patterns.
  Note, however, that this applies only to data characters. Whitespace characters
  may never appear within special character sequences in a pattern, for example
- within the sequence (?( which introduces a conditional subpattern.
+ within the sequence (?( that introduces a conditional subpattern.
  .sp
    PCRE_EXTRA
  .sp
-Line 624 
 option, the combination may or may not b
+Line 641 
 option, the combination may or may not b
  PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to PCRE_NEWLINE_CRLF, but
  other combinations may yield unused numbers and cause an error.
  .P
- The only time that a line break is specially recognized when compiling a
+ The only time that a line break in a pattern is specially recognized when
- pattern is if PCRE_EXTENDED is set, and an unescaped # outside a character
+ compiling is when PCRE_EXTENDED is set. CR and LF are whitespace characters,
- class is encountered. This indicates a comment that lasts until after the next
+ and so are ignored in this mode. Also, an unescaped # outside a character class
- line break sequence. In other circumstances, line break sequences are treated
+ indicates a comment that lasts until after the next line break sequence. In
- as literal data, except that in PCRE_EXTENDED mode, both CR and LF are treated
+ other circumstances, line break sequences in patterns are treated as literal
- as whitespace characters and are therefore ignored.
+ data.
  .P
  The newline option that is set at compile time becomes the default that is used
  for \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP, but it can be overridden.
-Line 642 
 were followed by ?: but named parenthese
+Line 659 
 were followed by ?: but named parenthese
  they acquire numbers in the usual way). There is no equivalent of this option
  in Perl.
  .sp
+   NO_START_OPTIMIZE
+ .sp
+ This is an option that acts at matching time; that is, it is really an option
+ for \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. If it is set at compile time,
+ it is remembered with the compiled pattern and assumed at matching time. For
+ details see the discussion of PCRE_NO_START_OPTIMIZE
+ .\" HTML <a href="#execoptions">
+ .\" </a>
+ below.
+ .\"
+ .sp
    PCRE_UCP
  .sp
- This option changes the way PCRE processes \eb, \ed, \es, \ew, and some of the
+ This option changes the way PCRE processes \eB, \eb, \eD, \ed, \eS, \es, \eW,
- POSIX character classes. By default, only ASCII characters are recognized, but
+ \ew, and some of the POSIX character classes. By default, only ASCII characters
- if PCRE_UCP is set, Unicode properties are used instead to classify characters.
+ are recognized, but if PCRE_UCP is set, Unicode properties are used instead to
- More details are given in the section on
+ classify characters. More details are given in the section on
  .\" HTML <a href="pcre.html#genericchartypes">
  .\" </a>
  generic character types
-Line 765 
 out of use. To avoid confusion, they hav
+Line 793 
 out of use. To avoid confusion, they hav
  [this code is not in use]
  octal value is greater than \e377 (not in UTF-8 mode)
  internal error: overran compiling workspace
- internal error: previously-checked referenced subpattern not found
+ internal error: previously-checked referenced subpattern
+         not found
  DEFINE group contains more than one branch
  repeating a DEFINE group is not allowed
  inconsistent NEWLINE options
-Line 778 
 out of use. To avoid confusion, they hav
+Line 807 
 out of use. To avoid confusion, they hav
  subpattern name expected
  digit expected after (?+
  ] is an invalid data character in JavaScript compatibility mode
- different names for subpatterns of the same number are not allowed
+ different names for subpatterns of the same number are
+         not allowed
  (*MARK) must have an argument
  this version of PCRE is not compiled with PCRE_UCP support
  .sp
-Line 846 
 Studying a pattern is also useful for no
+Line 876 
 Studying a pattern is also useful for no
  single fixed starting character. A bitmap of possible starting bytes is
  created. This speeds up finding a position in the subject at which to start
  matching.
+ .P
+ The two optimizations just described can be disabled by setting the
+ PCRE_NO_START_OPTIMIZE option when calling \fBpcre_exec()\fP or
+ \fBpcre_dfa_exec()\fP. You might want to do this if your pattern contains
+ callouts or (*MARK), and you want to make use of these facilities in cases
+ where matching fails. See the discussion of PCRE_NO_START_OPTIMIZE
+ .\" HTML <a href="#execoptions">
+ .\" </a>
+ below.
+ .\"
  .
  .
  .\" HTML <a name="localesupport"></a>
-Line 1438 
 the
+Line 1478 
 the
  .\" HREF
  \fBpcredemo\fP
  .\"
- sample program.
+ sample program. In the most general case, you have to check to see if the
+ newline convention recognizes CRLF as a newline, and if so, and the current
+ character is CR followed by LF, advance the starting offset by two characters
+ instead of one.
  .sp
    PCRE_NO_START_OPTIMIZE
  .sp
  There are a number of optimizations that \fBpcre_exec()\fP uses at the start of
- a match, in order to speed up the process. For example, if it is known that a
+ a match, in order to speed up the process. For example, if it is known that an
- match must start with a specific character, it searches the subject for that
+ unanchored match must start with a specific character, it searches the subject
- character, and fails immediately if it cannot find it, without actually running
+ for that character, and fails immediately if it cannot find it, without
- the main matching function. When callouts are in use, these optimizations can
+ actually running the main matching function. This means that a special item
- cause them to be skipped. This option disables the "start-up" optimizations,
+ such as (*COMMIT) at the start of a pattern is not considered until after a
- causing performance to suffer, but ensuring that the callouts do occur.
+ suitable starting point for the match has been found. When callouts or (*MARK)
+ items are in use, these "start-up" optimizations can cause them to be skipped
+ if the pattern is never actually used. The start-up optimizations are in effect
+ a pre-scan of the subject that takes place before the pattern is run.
+ .P
+ The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, possibly
+ causing performance to suffer, but ensuring that in cases where the result is
+ "no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK)
+ are considered at every possible starting position in the subject string. If
+ PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching
+ time.
+ .P
+ Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching operation.
+ Consider the pattern
+ .sp
+   (*COMMIT)ABC
+ .sp
+ When this is compiled, PCRE records the fact that a match must start with the
+ character "A". Suppose the subject string is "DEFABC". The start-up
+ optimization scans along the subject, finds "A" and runs the first match
+ attempt from there. The (*COMMIT) item means that the pattern must match the
+ current starting position, which in this case, it does. However, if the same
+ match is run with PCRE_NO_START_OPTIMIZE set, the initial scan along the
+ subject string does not happen. The first match attempt is run starting from
+ "D" and when this fails, (*COMMIT) prevents any further matches being tried, so
+ the overall result is "no match". If the pattern is studied, more start-up
+ optimizations may be used. For example, a minimum length for the subject may be
+ recorded. Consider the pattern
+ .sp
+   (*MARK:A)(X|Y)
+ .sp
+ The minimum length for a match is one character. If the subject is "ABC", there
+ will be attempts to match "ABC", "BC", "C", and then finally an empty string.
+ If the pattern is studied, the final attempt does not take place, because PCRE
+ knows that the subject is too short, and so the (*MARK) is never encountered.
+ In this case, studying the pattern does not affect the overall match result,
+ which is still "no match", but it does affect the auxiliary information that is
+ returned.
  .sp
    PCRE_NO_UTF8_CHECK
  .sp
-Line 1466 
 in the main
+Line 1546 
 in the main
  \fBpcre\fP
  .\"
  page. If an invalid UTF-8 sequence of bytes is found, \fBpcre_exec()\fP returns
- the error PCRE_ERROR_BADUTF8. If \fIstartoffset\fP contains an invalid value,
+ the error PCRE_ERROR_BADUTF8 or, if PCRE_PARTIAL_HARD is set and the problem is
- PCRE_ERROR_BADUTF8_OFFSET is returned.
+ a truncated UTF-8 character at the end of the subject, PCRE_ERROR_SHORTUTF8. If
+ \fIstartoffset\fP contains a value that does not point to the start of a UTF-8
+ character (or to the end of the subject), PCRE_ERROR_BADUTF8_OFFSET is
+ returned.
  .P
  If you already know that your subject is valid, and you want to skip these
  checks for performance reasons, you can set the PCRE_NO_UTF8_CHECK option when
  calling \fBpcre_exec()\fP. You might want to do this for the second and
  subsequent calls to \fBpcre_exec()\fP if you are making repeated calls to find
  all the matches in a single subject string. However, you should be sure that
- the value of \fIstartoffset\fP points to the start of a UTF-8 character. When
+ the value of \fIstartoffset\fP points to the start of a UTF-8 character (or the
- PCRE_NO_UTF8_CHECK is set, the effect of passing an invalid UTF-8 string as a
+ end of the subject). When PCRE_NO_UTF8_CHECK is set, the effect of passing an
- subject, or a value of \fIstartoffset\fP that does not point to the start of a
+ invalid UTF-8 string as a subject or an invalid value of \fIstartoffset\fP is
- UTF-8 character, is undefined. Your program may crash.
+ undefined. Your program may crash.
  .sp
    PCRE_PARTIAL_HARD
    PCRE_PARTIAL_SOFT
-Line 1486 
 These options turn on the partial matchi
+Line 1569 
 These options turn on the partial matchi
  compatibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial match
  occurs if the end of the subject string is reached successfully, but there are
  not enough subject characters to complete the match. If this happens when
- PCRE_PARTIAL_HARD is set, \fBpcre_exec()\fP immediately returns
+ PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set, matching continues by
- PCRE_ERROR_PARTIAL. Otherwise, if PCRE_PARTIAL_SOFT is set, matching continues
+ testing any remaining alternatives. Only if no complete match can be found is
- by testing any other alternatives. Only if they all fail is PCRE_ERROR_PARTIAL
+ PCRE_ERROR_PARTIAL returned instead of PCRE_ERROR_NOMATCH. In other words,
- returned (instead of PCRE_ERROR_NOMATCH). The portion of the string that
+ PCRE_PARTIAL_SOFT says that the caller is prepared to handle a partial match,
- was inspected when the partial match was found is set as the first matching
+ but only if no complete match can be found.
- string. There is a more detailed discussion in the
+ .P
+ If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this case, if a
+ partial match is found, \fBpcre_exec()\fP immediately returns
+ PCRE_ERROR_PARTIAL, without considering any other alternatives. In other words,
+ when PCRE_PARTIAL_HARD is set, a partial match is considered to be more
+ important that an alternative complete match.
+ .P
+ In both cases, the portion of the string that was inspected when the partial
+ match was found is set as the first matching string. There is a more detailed
+ discussion of partial and multi-segment matching, with examples, in the
  .\" HREF
  \fBpcrepartial\fP
  .\"
  documentation.
  .
+ .
  .SS "The string to be matched by \fBpcre_exec()\fP"
  .rs
  .sp
  The subject string is passed to \fBpcre_exec()\fP as a pointer in
  \fIsubject\fP, a length (in bytes) in \fIlength\fP, and a starting byte offset
- in \fIstartoffset\fP. In UTF-8 mode, the byte offset must point to the start of
+ in \fIstartoffset\fP. If this is negative or greater than the length of the
- a UTF-8 character. Unlike the pattern string, the subject may contain binary
+ subject, \fBpcre_exec()\fP returns PCRE_ERROR_BADOFFSET. When the starting
- zero bytes. When the starting offset is zero, the search for a match starts at
+ offset is zero, the search for a match starts at the beginning of the subject,
- the beginning of the subject, and this is by far the most common case.
+ and this is by far the most common case. In UTF-8 mode, the byte offset must
+ point to the start of a UTF-8 character (or the end of the subject). Unlike the
+ pattern string, the subject may contain binary zero bytes.
  .P
  A non-zero starting offset is useful when searching for another match in the
  same subject by calling \fBpcre_exec()\fP again after a previous success.
-Line 1525 
 start of the subject, which is deemed to
+Line 1620 
 start of the subject, which is deemed to
  set to 4, it finds the second occurrence of "iss" because it is able to look
  behind the starting point to discover that it is preceded by a letter.
  .P
+ Finding all the matches in a subject is tricky when the pattern can match an
+ empty string. It is possible to emulate Perl's /g behaviour by first trying the
+ match again at the same offset, with the PCRE_NOTEMPTY_ATSTART and
+ PCRE_ANCHORED options, and then if that fails, advancing the starting offset
+ and trying an ordinary match again. There is some code that demonstrates how to
+ do this in the
+ .\" HREF
+ \fBpcredemo\fP
+ .\"
+ sample program. In the most general case, you have to check to see if the
+ newline convention recognizes CRLF as a newline, and if so, and the current
+ character is CR followed by LF, advance the starting offset by two characters
+ instead of one.
+ .P
  If a non-zero starting offset is passed when the pattern is anchored, one
  attempt to match at the given offset is made. This can only succeed if the
  pattern does not require the match to be at the start of the subject.
  .
+ .
  .SS "How \fBpcre_exec()\fP returns captured substrings"
  .rs
  .sp
-Line 1595 
 Offset values that correspond to unused
+Line 1705 
 Offset values that correspond to unused
  expression are also set to -1. For example, if the string "abc" is matched
  against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not matched. The
  return from the function is 2, because the highest used capturing subpattern
- number is 1. However, you can refer to the offsets for the second and third
+ number is 1, and the offsets for for the second and third capturing subpatterns
- capturing subpatterns if you wish (assuming the vector is large enough, of
+ (assuming the vector is large enough, of course) are set to -1.
- course).
+ .P
+ \fBNote\fP: Elements of \fIovector\fP that do not correspond to capturing
+ parentheses in the pattern are never changed. That is, if a pattern contains
+ \fIn\fP capturing parentheses, no more than \fIovector[0]\fP to
+ \fIovector[2n+1]\fP are set by \fBpcre_exec()\fP. The other elements retain
+ whatever values they previously had.
  .P
  Some convenience functions are provided for extracting the captured substrings
  as separate strings. These are described below.
-Line 1643 
 If a pattern contains back references, b
+Line 1758 
 If a pattern contains back references, b
  gets a block of memory at the start of matching to use for this purpose. If the
  call via \fBpcre_malloc()\fP fails, this error is given. The memory is
  automatically freed at the end of matching.
+ .P
+ This error is also given if \fBpcre_stack_malloc()\fP fails in
+ \fBpcre_exec()\fP. This can happen only when PCRE has been compiled with
+ \fB--disable-stack-for-recursion\fP.
  .sp
    PCRE_ERROR_NOSUBSTRING    (-7)
  .sp
-Line 1668 
 documentation for details.
+Line 1787 
 documentation for details.
    PCRE_ERROR_BADUTF8        (-10)
  .sp
  A string that contains an invalid UTF-8 byte sequence was passed as a subject.
+ However, if PCRE_PARTIAL_HARD is set and the problem is a truncated UTF-8
+ character at the end of the subject, PCRE_ERROR_SHORTUTF8 is used instead.
  .sp
    PCRE_ERROR_BADUTF8_OFFSET (-11)
  .sp
  The UTF-8 byte sequence that was passed as a subject was valid, but the value
- of \fIstartoffset\fP did not point to the beginning of a UTF-8 character.
+ of \fIstartoffset\fP did not point to the beginning of a UTF-8 character or the
+ end of the subject.
  .sp
    PCRE_ERROR_PARTIAL        (-12)
  .sp
-Line 1707 
 description above.
+Line 1829 
 description above.
    PCRE_ERROR_BADNEWLINE     (-23)
  .sp
  An invalid combination of PCRE_NEWLINE_\fIxxx\fP options was given.
+ .sp
+   PCRE_ERROR_BADOFFSET      (-24)
+ .sp
+ The value of \fIstartoffset\fP was negative or greater than the length of the
+ subject, that is, the value in \fIlength\fP.
+ .sp
+   PCRE_ERROR_SHORTUTF8      (-25)
+ .sp
+ The subject string ended with an incomplete (truncated) UTF-8 character, and
+ the PCRE_PARTIAL_HARD option was set. Without this option, PCRE_ERROR_BADUTF8
+ is returned in this situation.
  .P
  Error numbers -16 to -20 and -22 are not used by \fBpcre_exec()\fP.
  .
-Line 1991 
 Here is an example of a simple call to \
+Line 2124 
 Here is an example of a simple call to \
  The unused bits of the \fIoptions\fP argument for \fBpcre_dfa_exec()\fP must be
  zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_\fIxxx\fP,
  PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
- PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, PCRE_PARTIAL_SOFT, PCRE_DFA_SHORTEST,
+ PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF, PCRE_BSR_UNICODE, PCRE_NO_START_OPTIMIZE,
- and PCRE_DFA_RESTART. All but the last four of these are exactly the same as
+ PCRE_PARTIAL_HARD, PCRE_PARTIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART.
- for \fBpcre_exec()\fP, so their description is not repeated here.
+ All but the last four of these are exactly the same as for \fBpcre_exec()\fP,
+ so their description is not repeated here.
  .sp
    PCRE_PARTIAL_HARD
    PCRE_PARTIAL_SOFT
-Line 2008 
 is converted into PCRE_ERROR_PARTIAL if
+Line 2142 
 is converted into PCRE_ERROR_PARTIAL if
  there have been no complete matches, but there is still at least one matching
  possibility. The portion of the string that was inspected when the longest
  partial match was found is set as the first matching string in both cases.
+ There is a more detailed discussion of partial and multi-segment matching, with
+ examples, in the
+ .\" HREF
+ \fBpcrepartial\fP
+ .\"
+ documentation.
  .sp
    PCRE_DFA_SHORTEST
  .sp
-Line 2127 
 Cambridge CB2 3QH, England.
+Line 2267 
 Cambridge CB2 3QH, England.
  .rs
  .sp
  .nf
- Last updated: 26 May 2010
+ Last updated: 21 November 2010
  Copyright (c) 1997-2010 University of Cambridge.
  .fi

 Legend:



Removed from v.526
 


changed lines


 
Added in v.579
 Legend:



Removed from v.526
 


changed lines


 
Added in v.579
-Removed from v.526
+Added in v.579

webmaster@exim.org	ViewVC Help
Powered by ViewVC 1.1.12