/[pcre]/code/tags/pcre-1.09/pcre.3
ViewVC logotype

Contents of /code/tags/pcre-1.09/pcre.3

Parent Directory Parent Directory | Revision Log Revision Log


Revision 22 - (show annotations) (download)
Sat Feb 24 21:38:39 2007 UTC (7 years, 9 months ago) by nigel
File size: 46646 byte(s)
Tag code/trunk as code/tags/pcre-1.09.

1 .TH PCRE 3
2 .SH NAME
3 pcre - Perl-compatible regular expressions.
4 .SH SYNOPSIS
5 .B #include <pcre.h>
6 .PP
7 .SM
8 .br
9 .B pcre *pcre_compile(const char *\fIpattern\fR, int \fIoptions\fR,
10 .ti +5n
11 .B const char **\fIerrptr\fR, int *\fIerroffset\fR);
12 .PP
13 .br
14 .B pcre_extra *pcre_study(const pcre *\fIcode\fR, int \fIoptions\fR,
15 .ti +5n
16 .B const char **\fIerrptr\fR);
17 .PP
18 .br
19 .B int pcre_exec(const pcre *\fIcode\fR, "const pcre_extra *\fIextra\fR,"
20 .ti +5n
21 .B "const char *\fIsubject\fR," int \fIlength\fR, int \fIoptions\fR,
22 .ti +5n
23 .B int *\fIovector\fR, int \fIovecsize\fR);
24 .PP
25 .br
26 .B int pcre_info(const pcre *\fIcode\fR, int *\fIoptptr\fR, int
27 .B *\fIfirstcharptr\fR);
28 .PP
29 .br
30 .B char *pcre_version(void);
31 .PP
32 .br
33 .B void *(*pcre_malloc)(size_t);
34 .PP
35 .br
36 .B void (*pcre_free)(void *);
37 .PP
38 .br
39 .B unsigned char *pcre_cbits[128];
40 .PP
41 .br
42 .B unsigned char *pcre_ctypes[256];
43 .PP
44 .br
45 .B unsigned char *pcre_fcc[256];
46 .PP
47 .br
48 .B unsigned char *pcre_lcc[256];
49
50
51
52 .SH DESCRIPTION
53 The PCRE library is a set of functions that implement regular expression
54 pattern matching using the same syntax and semantics as Perl 5, with just a few
55 differences (see below). The current implementation corresponds to Perl 5.004.
56
57 PCRE has its own native API, which is described in this man page. There is also
58 a set of wrapper functions that correspond to the POSIX API. See
59 \fBpcreposix (3)\fR.
60
61 The three functions \fBpcre_compile()\fR, \fBpcre_study()\fR, and
62 \fBpcre_exec()\fR are used for compiling and matching regular expressions. The
63 function \fBpcre_info()\fR is used to find out information about a compiled
64 pattern, while the function \fBpcre_version()\fR returns a pointer to a string
65 containing the version of PCRE and its date of release.
66
67 The global variables \fBpcre_malloc\fR and \fBpcre_free\fR initially contain
68 the entry points of the standard \fBmalloc()\fR and \fBfree()\fR functions
69 respectively. PCRE calls the memory management functions via these variables,
70 so a calling program can replace them if it wishes to intercept the calls. This
71 should be done before calling any PCRE functions.
72
73 The other global variables are character tables. They are initialized when PCRE
74 is compiled, from source that is generated by reference to the C character type
75 functions, but which the maintainer of PCRE is free to modify. In principle
76 they could also be modified at runtime. See PCRE's README file for more
77 details.
78
79
80 .SH MULTI-THREADING
81 The PCRE functions can be used in multi-threading applications, with the
82 proviso that the character tables and the memory management functions pointed
83 to by \fBpcre_malloc\fR and \fBpcre_free\fR will be shared by all threads.
84
85 The compiled form of a regular expression is not altered during matching, so
86 the same compiled pattern can safely be used by several threads at once.
87
88
89 .SH COMPILING A PATTERN
90 The function \fBpcre_compile()\fR is called to compile a pattern into an
91 internal form. The pattern is a C string terminated by a binary zero, and
92 is passed in the argument \fIpattern\fR. A pointer to the compiled code block
93 is returned. The \fBpcre\fR type is defined for this for convenience, but in
94 fact \fBpcre\fR is just a typedef for \fBvoid\fR, since the contents of the
95 block are not defined.
96 .PP
97 The size of a compiled pattern is roughly proportional to the length of the
98 pattern string, except that each character class (other than those containing
99 just a single character, negated or not) requires 33 bytes, and repeat
100 quantifiers with a minimum greater than one or a bounded maximum cause the
101 relevant portions of the compiled pattern to be replicated.
102 .PP
103 The \fIoptions\fR argument contains independent bits that affect the
104 compilation. It should be zero if no options are required. Those options that
105 are compabible with Perl can also be set at compile time from within the
106 pattern (see the detailed description of regular expressions below) and all
107 options except PCRE_EXTENDED, PCRE_EXTRA and PCRE_UNGREEDY can be set at the
108 time of matching.
109 .PP
110 If \fIerrptr\fR is NULL, \fBpcre_compile()\fR returns NULL immediately.
111 Otherwise, if compilation of a pattern fails, \fBpcre_compile()\fR returns
112 NULL, and sets the variable pointed to by \fIerrptr\fR to point to a textual
113 error message.
114
115 The offset from the start of the pattern to the character where the error was
116 discovered is placed in the variable pointed to by \fIerroffset\fR, which must
117 not be NULL. If it is, an immediate error is given.
118 .PP
119 The following option bits are defined in the header file:
120
121 PCRE_ANCHORED
122
123 If this bit is set, the pattern is forced to be "anchored", that is, it is
124 constrained to match only at the start of the string which is being searched
125 (the "subject string"). This effect can also be achieved by appropriate
126 constructs in the pattern itself, which is the only way to do it in Perl.
127
128 PCRE_CASELESS
129
130 If this bit is set, letters in the pattern match both upper and lower case
131 letters in any subject string. It is equivalent to Perl's /i option.
132
133 PCRE_DOLLAR_ENDONLY
134
135 If this bit is set, a dollar metacharacter in the pattern matches only at the
136 end of the subject string. By default, it also matches immediately before the
137 final character if it is a newline (but not before any other newlines). The
138 PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. There is no
139 equivalent to this option in Perl.
140
141 PCRE_DOTALL
142
143 If this bit is set, a dot metacharater in the pattern matches all characters,
144 including newlines. By default, newlines are excluded. This option is
145 equivalent to Perl's /s option. A negative class such as [^a] always matches a
146 newline character, independent of the setting of this option.
147
148 PCRE_EXTENDED
149
150 If this bit is set, whitespace characters in the pattern are totally ignored
151 except when escaped or inside a character class, and characters between an
152 unescaped # outside a character class and the next newline character,
153 inclusive, are also ignored. This is equivalent to Perl's /x option, and makes
154 it possible to include comments inside complicated patterns.
155
156 PCRE_MULTILINE
157
158 By default, PCRE treats the subject string as consisting of a single "line" of
159 characters (even if it actually contains several newlines). The "start of line"
160 metacharacter (^) matches only at the start of the string, while the "end of
161 line" metacharacter ($) matches only at the end of the string, or before a
162 terminating newline. This is the same as Perl.
163
164 When PCRE_MULTILINE it is set, the "start of line" and "end of line" constructs
165 match immediately following or immediately before any newline in the subject
166 string, respectively, as well as at the very start and end. This is equivalent
167 to Perl's /m option. If there are no "\\n" characters in a subject string, or
168 no occurrences of ^ or $ in a pattern, setting PCRE_MULTILINE has no
169 effect.
170
171 PCRE_EXTRA
172
173 This option turns on additional functionality of PCRE that is incompatible with
174 Perl. Any backslash in a pattern that is followed by a letter that has no
175 special meaning causes an error, thus reserving these combinations for future
176 expansion. By default, as in Perl, a backslash followed by a letter with no
177 special meaning is treated as a literal. There are two extra features currently
178 provided, and both are in some sense experimental additions that are useful for
179 influencing the progress of a match.
180
181 (1) The sequence \\X inserts a Prolog-like "cut" into the expression.
182
183 (2) Once a subpattern enclosed in (?>subpat) brackets has matched,
184 backtracking never goes back into the pattern.
185
186 See below for further details of both of these. PCRE_EXTRA can be set by a (?X)
187 option setting within the pattern, but this must precede anything in the
188 pattern which relies on its being set.
189
190 PCRE_UNGREEDY
191
192 This option inverts the "greediness" of the quantifiers so that they are not
193 greedy by default, but become greedy if followed by "?". It is not compatible
194 with Perl. It can also be set by a (?U) option setting within the pattern.
195
196
197
198 .SH STUDYING A PATTERN
199 When a pattern is going to be used several times, it is worth spending more
200 time analyzing it in order to speed up the time taken for matching. The
201 function \fBpcre_study()\fR takes a pointer to a compiled pattern as its first
202 argument, and returns a pointer to a \fBpcre_extra\fR block (another \fBvoid\fR
203 typedef) containing additional information about the pattern; this can be
204 passed to \fBpcre_exec()\fR. If no additional information is available, NULL
205 is returned.
206
207 The second argument contains option bits. The only one currently supported is
208 PCRE_CASELESS. It forces the studying to be done in a caseless manner, even if
209 the original pattern was compiled without PCRE_CASELESS. When the result of
210 \fBpcre_study()\fR is passed to \fBpcre_exec()\fR, it is used only if its
211 caseless state is the same as that of the matching process. A pattern that is
212 compiled without PCRE_CASELESS can be studied with and without PCRE_CASELESS,
213 and the appropriate data passed to \fBpcre_exec()\fR with and without the
214 PCRE_CASELESS flag.
215
216 The third argument for \fBpcre_study()\fR is a pointer to an error message. If
217 studying succeeds (even if no data is returned), the variable it points to is
218 set to NULL. Otherwise it points to a textual error message.
219
220 At present, studying a pattern is useful only for non-anchored patterns that do
221 not have a single fixed starting character. A bitmap of possible starting
222 characters is created.
223
224
225 .SH MATCHING A PATTERN
226 The function \fBpcre_exec()\fR is called to match a subject string against a
227 pre-compiled pattern, which is passed in the \fIcode\fR argument. If the
228 pattern has been studied, the result of the study should be passed in the
229 \fIextra\fR argument. Otherwise this must be NULL.
230
231 The subject string is passed as a pointer in \fIsubject\fR and a length in
232 \fIlength\fR. Unlike the pattern string, it may contain binary zero characters.
233
234 The options PCRE_ANCHORED, PCRE_CASELESS, PCRE_DOLLAR_ENDONLY, PCRE_DOTALL, and
235 PCRE_MULTILINE can be passed in the \fIoptions\fR argument, whose unused bits
236 must be zero. However, if a pattern is compiled with any of these options, they
237 cannot be unset when it is obeyed.
238
239 There are also two further options that can be set only at matching time:
240
241 PCRE_NOTBOL
242
243 The first character of the string is not the beginning of a line, so the
244 circumflex metacharacter should not match before it. Setting this without
245 PCRE_MULTILINE (at either compile or match time) causes circumflex never to
246 match.
247
248 PCRE_NOTEOL
249
250 The end of the string is not the end of a line, so the dollar metacharacter
251 should not match it. Setting this without PCRE_MULTILINE (at either compile or
252 match time) causes dollar never to match.
253
254 In general, a pattern matches a certain portion of the subject, and in
255 addition, further substrings from the subject may be picked out by parts of the
256 pattern. Following the usage in Jeffrey Friedl's book, this is called
257 "capturing" in what follows, and the phrase "capturing subpattern" is used for
258 a fragment of a pattern that picks out a substring. PCRE supports several other
259 kinds of parenthesized subpattern that do not cause substrings to be captured.
260
261 Captured substrings are returned to the caller via a vector of integer offsets
262 whose address is passed in \fIovector\fR. The number of elements in the vector
263 is passed in \fIovecsize\fR. This should always be an even number, because the
264 elements are used in pairs. If an odd number is passed, it is rounded down.
265
266 The first element of a pair is set to the offset of the first character in a
267 substring, and the second is set to the offset of the first character after the
268 end of a substring. The first pair, \fIovector[0]\fR and \fIovector[1]\fR,
269 identify the portion of the subject string matched by the entire pattern. The
270 next pair is used for the first capturing subpattern, and so on. The value
271 returned by \fBpcre_exec()\fR is the number of pairs that have been set. If
272 there are no capturing subpatterns, the return value from a successful match
273 is 1, indicating that just the first pair of offsets has been set.
274
275 It is possible for an capturing subpattern number \fIn+1\fR to match some
276 part of the subject when subpattern \fIn\fR has not been used at all. For
277 example, if the string "abc" is matched against the pattern "(a|(z))(bc)",
278 subpatterns 1 and 3 are matched, but 2 is not. When this happens, both offset
279 values corresponding to the unused subpattern are set to -1.
280
281 If a capturing subpattern is matched repeatedly, it is the last portion of the
282 string that it matched that gets returned.
283
284 If the vector is too small to hold all the captured substrings, it is used as
285 far as possible, and the function returns a value of zero. In particular, if
286 the substring offsets are not of interest, \fBpcre_exec()\fR may be called with
287 \fIovector\fR passed as NULL and \fIovecsize\fR as zero. However, if the
288 pattern contains back references and the \fIovector\fR isn't big enough to
289 remember the related substrings, PCRE has to get additional memory for use
290 during matching. Thus it is usually advisable to supply an \fIovector\fR.
291
292 Note that \fBpcre_info()\fR can be used to find out how many capturing
293 subpatterns there are in a compiled pattern.
294
295 If \fBpcre_exec()\fR fails, it returns a negative number. The following are
296 defined in the header file:
297
298 PCRE_ERROR_NOMATCH (-1)
299
300 The subject string did not match the pattern.
301
302 PCRE_ERROR_BADREF (-2)
303
304 There was a back-reference in the pattern to a capturing subpattern that had
305 not previously been set.
306
307 PCRE_ERROR_NULL (-3)
308
309 Either \fIcode\fR or \fIsubject\fR was passed as NULL, or \fIovector\fR was
310 NULL and \fIovecsize\fR was not zero.
311
312 PCRE_ERROR_BADOPTION (-4)
313
314 An unrecognized bit was set in the \fIoptions\fR argument.
315
316 PCRE_ERROR_BADMAGIC (-5)
317
318 PCRE stores a 4-byte "magic number" at the start of the compiled code, to catch
319 the case when it is passed a junk pointer. This is the error it gives when the
320 magic number isn't present.
321
322 PCRE_ERROR_UNKNOWN_NODE (-6)
323
324 While running the pattern match, an unknown item was encountered in the
325 compiled pattern. This error could be caused by a bug in PCRE or by overwriting
326 of the compiled pattern.
327
328 PCRE_ERROR_NOMEMORY (-7)
329
330 If a pattern contains back references, but the \fIovector\fR that is passed to
331 \fBpcre_exec()\fR is not big enough to remember the referenced substrings, PCRE
332 gets a block of memory at the start of matching to use for this purpose. If the
333 call via \fBpcre_malloc()\fR fails, this error is given. The memory is freed at
334 the end of matching.
335
336
337 .SH INFORMATION ABOUT A PATTERN
338 The \fBpcre_info()\fR function returns information about a compiled pattern.
339 Its yield is the number of capturing subpatterns, or one of the following
340 negative numbers:
341
342 PCRE_ERROR_NULL the argument \fIcode\fR was NULL
343 PCRE_ERROR_BADMAGIC the "magic number" was not found
344
345 If the \fIoptptr\fR argument is not NULL, a copy of the options with which the
346 pattern was compiled is placed in the integer it points to.
347
348 If the \fIfirstcharptr\fR argument is not NULL, is is used to pass back
349 information about the first character of any matched string. If there is a
350 fixed first character, e.g. from a pattern such as (cat|cow|coyote), then it is
351 returned in the integer pointed to by \fIfirstcharptr\fR. Otherwise, if the
352 pattern was compiled with the PCRE_MULTILINE option, and every branch started
353 with "^", then -1 is returned, indicating that the pattern will match at the
354 start of a subject string or after any "\\n" within the string. Otherwise -2 is
355 returned.
356
357
358 .SH LIMITATIONS
359 There are some size limitations in PCRE but it is hoped that they will never in
360 practice be relevant.
361 The maximum length of a compiled pattern is 65539 (sic) bytes.
362 All values in repeating quantifiers must be less than 65536.
363 The maximum number of capturing subpatterns is 99.
364 The maximum number of all parenthesized subpatterns, including capturing
365 subpatterns and assertions, is 200.
366
367 The maximum length of a subject string is the largest positive number that an
368 integer variable can hold. However, PCRE uses recursion to handle subpatterns
369 and indefinite repetition. This means that the available stack space may limit
370 the size of a subject string that can be processed by certain patterns.
371
372
373 .SH DIFFERENCES FROM PERL
374 The differences described here are with respect to Perl 5.004.
375
376 1. By default, a whitespace character is any character that the C library
377 function \fBisspace()\fR recognizes, though it is possible to compile PCRE with
378 alternative character type tables. Normally \fBisspace()\fR matches space,
379 formfeed, newline, carriage return, horizontal tab, and vertical tab. Perl 5
380 no longer includes vertical tab in its set of whitespace characters. The \\v
381 escape that was in the Perl documentation for a long time was never in fact
382 recognized. However, the character itself was treated as whitespace at least
383 up to 5.002. In 5.004 it does not match \\s.
384
385 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl permits
386 them, but they do not mean what you might think. For example, "(?!a){3}" does
387 not assert that the next three characters are not "a". It just asserts that the
388 next character is not "a" three times.
389
390 3. Capturing subpatterns that occur inside negative lookahead assertions are
391 counted, but their entries in the offsets vector are never set. Perl sets its
392 numerical variables from any such patterns that are matched before the
393 assertion fails to match something (thereby succeeding), but only if the
394 negative lookahead assertion contains just one branch.
395
396 4. Though binary zero characters are supported in the subject string, they are
397 not allowed in a pattern string because it is passed as a normal C string,
398 terminated by zero. The escape sequence "\\0" can be used in the pattern to
399 represent a binary zero.
400
401 5. The following Perl escape sequences are not supported: \\l, \\u, \\L, \\U,
402 \\E, \\Q. In fact these are implemented by Perl's general string-handling and
403 are not part of its pattern matching engine.
404
405 6. The Perl \\G assertion is not supported as it is not relevant to single
406 pattern matches.
407
408 7. If a backreference can never be matched, PCRE diagnoses an error. In a case
409 like
410
411 /(123)\\2/
412
413 the error occurs at compile time. Perl gives no compile time error; version
414 5.004 either always fails to match, or gives a segmentation fault at runtime.
415 In more complicated cases such as
416
417 /(1)(2)(3)(4)(5)(6)(7)(8)(9)(10\\10)/
418
419 PCRE returns PCRE_ERROR_BADREF at run time. Perl always fails to match.
420
421 8. PCRE provides some extensions to the Perl regular expression facilities:
422
423 (a) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ meta-
424 character matches only at the very end of the string.
425
426 (b) If PCRE_EXTRA is set, the \\X assertion (a Prolog-like "cut") is
427 recognized, and a backslash followed by a letter with no special meaning is
428 faulted. There is also a new kind of parenthesized subpattern starting with (?>
429 which has a block on backtracking into it once it has matched.
430
431 (c) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is
432 inverted, that is, by default they are not greedy, but if followed by a
433 question mark they are.
434
435
436 .SH REGULAR EXPRESSION DETAILS
437 The syntax and semantics of the regular expressions supported by PCRE are
438 described below. Regular expressions are also described in the Perl
439 documentation and in a number of other books, some of which have copious
440 examples. Jeffrey Friedl's "Mastering Regular Expressions", published by
441 O'Reilly (ISBN 1-56592-257-3), covers them in great detail. The description
442 here is intended as reference documentation.
443
444 A regular expression is a pattern that is matched against a subject string from
445 left to right. Most characters stand for themselves in a pattern, and match the
446 corresponding characters in the subject. As a trivial example, the pattern
447
448 The quick brown fox
449
450 matches a portion of a subject string that is identical to itself. The power of
451 regular expressions comes from the ability to include alternatives and
452 repetitions in the pattern. These are encoded in the pattern by the use of
453 \fImeta-characters\fR, which do not stand for themselves but instead are
454 interpreted in some special way.
455
456 There are two different sets of meta-characters: those that are recognized
457 anywhere in the pattern except within square brackets, and those that are
458 recognized in square brackets. Outside square brackets, the meta-characters are
459 as follows:
460
461 \\ general escape character with several uses
462 ^ assert start of subject (or line, in multiline mode)
463 $ assert end of subject (or line, in multiline mode)
464 . match any character except newline (by default)
465 [ start character class definition
466 | start of alternative branch
467 ( start subpattern
468 ) end subpattern
469 ? extends the meaning of (
470 also 0 or 1 quantifier
471 also quantifier minimizer
472 * 0 or more quantifier
473 + 1 or more quantifier
474 { start min/max quantifier
475
476 Part of a pattern that is in square brackets is called a "character class". In
477 a character class the only meta-characters are:
478
479 \\ general escape character
480 ^ negate the class, but only if the first character
481 - indicates character range
482 ] terminates the character class
483
484 The following sections describe the use of each of the meta-characters.
485
486
487 .SH BACKSLASH
488 The backslash character has several uses. Firstly, if it is followed by a
489 non-alphameric character, it takes away any special meaning that character may
490 have. This use of backslash as an escape character applies both inside and
491 outside character classes.
492
493 For example, if you want to match a "*" character, you write "\\*" in the
494 pattern. This applies whether or not the following character would otherwise be
495 interpreted as a meta-character, so it is always safe to precede a
496 non-alphameric with "\\" to specify that it stands for itself. In particular,
497 if you want to match a backslash, you write "\\\\".
498
499 If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the
500 pattern and characters between a "#" outside a character class and the next
501 newline character are ignored. An escaping backslash can be used to include a
502 whitespace or "#" character as part of the pattern.
503
504 A second use of backslash provides a way of encoding non-printing characters
505 in patterns in a visible manner. There is no restriction on the appearance of
506 non-printing characters, apart from the binary zero that terminates a pattern,
507 but when a pattern is being prepared by text editing, it is usually easier to
508 use one of the following escape sequences than the binary character it
509 represents:
510
511 \\a alarm, that is, the BEL character (hex 07)
512 \\cx "control-x", where x is any character
513 \\e escape (hex 1B)
514 \\f formfeed (hex 0C)
515 \\n newline (hex 0A)
516 \\r carriage return (hex 0D)
517 \\t tab (hex 09)
518 \\xhh character with hex code hh
519 \\ddd character with octal code ddd or backreference
520
521 The precise effect of "\\cx" is as follows: if "x" is a lower case letter, it
522 is converted to upper case. Then bit 6 of the character (hex 40) is inverted.
523 Thus "\\cz" becomes hex 1A, but "\\c{" becomes hex 3B, while "\\c;" becomes hex
524 7B.
525
526 After "\\x", up to two hexadecimal digits are read (letters can be in upper or
527 lower case).
528
529 After "\\0" up to two further octal digits are read. In both cases, if there
530 are fewer than two digits, just those that are present are used. Thus the
531 sequence "\\0\\x\\07" specifies two binary zeros followed by a BEL character.
532 Make sure you supply two digits if the character that follows could otherwise
533 be taken as another digit.
534
535 The handling of a backslash followed by a digit other than 0 is complicated.
536 Outside a character class, PCRE reads it and any following digits as a decimal
537 number. If the number is less than 10, or if there have been at least that many
538 previous capturing left parentheses in the expression, the entire sequence is
539 taken as a \fIback reference\fR. A description of how this works is given
540 later, following the discussion of parenthesized subpatterns.
541
542 Inside a character class, or if the decimal number is greater than 9 and there
543 have not been that many capturing subpatterns, PCRE re-reads up to three octal
544 digits following the backslash, and generates a single byte from the least
545 significant 8 bits of the value. Any subsequent digits stand for themselves.
546 For example:
547
548 \\040 is another way of writing a space
549 \\40 is the same, provided there are fewer than 40
550 previous capturing subpatterns
551 \\7 is always a back reference
552 \\11 might be a back reference, or another way of
553 writing a tab
554 \\011 is always a tab
555 \\0113 is a tab followed by the character "3"
556 \\113 is the character with octal code 113 (since there
557 can be no more than 99 back references)
558 \\377 is a byte consisting entirely of 1 bits
559 \\81 is either a back reference, or a binary zero
560 followed by the two characters "8" and "1"
561
562 Note that octal values of 100 or greater must not be introduced by a leading
563 zero, because no more than three octal digits are ever read.
564
565 All the sequences that define a single byte value can be used both inside and
566 outside character classes. In addition, inside a character class, the sequence
567 "\\b" is interpreted as the backspace character (hex 08). Outside a character
568 class it has a different meaning (see below).
569
570 The third use of backslash is for specifying generic character types:
571
572 \\d any decimal digit
573 \\D any character that is not a decimal digit
574 \\s any whitespace character
575 \\S any character that is not a whitespace character
576 \\w any "word" character
577 \\W any "non-word" character
578
579 Each pair of escape sequences partitions the complete set of characters into
580 two disjoint sets. Any given character matches one, and only one, of each pair.
581
582 A "word" character is any letter or digit or the underscore character, that is,
583 any character which can be part of a Perl "word". These character type
584 sequences can appear both inside and outside character classes. They each match
585 one character of the appropriate type. If the current matching point is at the
586 end of the subject string, all of them fail, since there is no character to
587 match.
588
589 The fourth use of backslash is for certain assertions. An assertion specifies a
590 condition that has to be met at a particular point in a match, without
591 consuming any characters from the subject string. The backslashed assertions
592 are
593
594 \\b word boundary
595 \\B not a word boundary
596 \\A start of subject (independent of multiline mode)
597 \\Z end of subject (independent of multiline mode)
598
599 Assertions may not appear in character classes (but note that "\\b" has a
600 different meaning, namely the backspace character, inside a character class).
601
602 A word boundary is a position in the subject string where the current character
603 and the previous character do not both match "\\w" or "\\W" (i.e. one matches
604 "\\w" and the other matches "\\W"), or the start or end of the string if the
605 first or last character matches "\\w", respectively. More complicated
606 assertions are also supported (see below).
607
608 The "\\A" and "\\Z" assertions differ from the traditional "^" and "$"
609 (described below) in that they only ever match at the very start and end of the
610 subject string, respectively, whatever options are set.
611
612 When the PCRE_EXTRA flag is set on a call to \fBpcre_compile()\fR, the
613 additional assertion \\X, which has no equivalent in Perl, is recognized.
614 This operates like the "cut" operation in Prolog: it prevents the matching
615 operation from backtracking past it. For example, if the expression
616
617 .*/foo
618
619 is matched against the string "/this/string/is/not" then after the greedy .*
620 has swallowed the whole string, PCRE keeps backtracking all the way to the
621 beginning before failing. If, on the other hand, the expression is
622
623 .*/\\Xfoo
624
625 then once it has discovered that "/not" is not "/foo", backtracking ceases, and
626 the match fails. See also the section on "once-only" subpatterns below.
627
628
629
630 .SH CIRCUMFLEX AND DOLLAR
631 Outside a character class, the circumflex character is an assertion which is
632 true only if the current matching point is at the start of the subject string,
633 in the default matching mode. Inside a character class, circumflex has an
634 entirely different meaning (see below).
635
636 Circumflex need not be the first character of the pattern if a number of
637 alternatives are involved, but it should be the first thing in each alternative
638 in which it appears if the pattern is ever to match that branch. If all
639 possible alternatives start with a circumflex, that is, if the pattern is
640 constrained to match only at the start of the subject, it is said to be an
641 "anchored" pattern. (There are also other constructs that can cause a pattern
642 to be anchored.)
643
644 A dollar character is an assertion which is true only if the current matching
645 point is at the end of the subject string, or immediately before a newline
646 character that is the last character in the string (by default). Dollar need
647 not be the last character of the pattern if a number of alternatives are
648 involved, but it should be the last item in any branch in which it appears.
649 Dollar has no special meaning in a character class.
650
651 The meaning of dollar can be changed so that it matches only at the very end of
652 the string, by setting the PCRE_DOLLAR_ENDONLY option at compile or matching
653 time.
654
655 The meanings of the circumflex and dollar characters are changed if the
656 PCRE_MULTILINE option is set at compile or matching time. When this is the
657 case, they match immediately after and immediately before an internal "\\n"
658 character, respectively, in addition to matching at the start and end of the
659 subject string. For example, the pattern /^abc$/ matches the subject string
660 "def\\nabc" in multiline mode, but not otherwise. Consequently, patterns that
661 are anchored in single line mode because all branches start with "^" are not
662 anchored in multiline mode. The PCRE_DOLLAR_ENDONLY option is ignored if
663 PCRE_MULTILINE is set.
664
665 Note that the sequences "\\A" and "\\Z" can be used to match the start and end
666 of the subject in both modes, and if all branches of a pattern start with "\\A"
667 is it always anchored.
668
669
670 .SH FULL STOP (PERIOD, DOT)
671 Outside a character class, a dot in the pattern matches any one character in
672 the subject, including a non-printing character, but not (by default) newline.
673 If the PCRE_DOTALL option is set, then dots match newlines as well. The
674 handling of dot is entirely independent of the handling of circumflex and
675 dollar, the only relationship being that they both involve newline characters.
676 Dot has no special meaning in a character class.
677
678
679 .SH SQUARE BRACKETS
680 An opening square bracket introduces a character class, terminated by a closing
681 square bracket. A closing square bracket on its own is not special. If a
682 closing square bracket is required as a member of the class, it should be the
683 first data character in the class (after an initial circumflex, if present) or
684 escaped with \\.
685
686 A character class matches a single character in the subject; the character must
687 be in the set of characters defined by the class, unless the first character in
688 the class is a circumflex, in which case the subject character must not be in
689 the set defined by the class. If a circumflex is actually required as a member
690 of the class, ensure it is not the first character, or escape it with \\.
691
692 For example, the character class [aeiou] matches any lower case vowel, while
693 [^aeiou] matches any character that is not a lower case vowel. Note that a
694 circumflex is just a convenient notation for specifying the characters which
695 are in the class by enumerating those that are not. It is not an assertion: it
696 still consumes a character from the subject string, and fails if the current
697 pointer is at the end of the string.
698
699 The newline character is never treated in any special way in character classes,
700 whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE options is. A class
701 such as [^a] will always match a newline.
702
703 The minus (hyphen) character can be used to specify a range of characters in a
704 character class. For example, [d-m] matches any letter between d and m,
705 inclusive. If a minus character is required in a class, it must be escaped with
706 \\ or appear in a position where it cannot be interpreted as indicating a
707 range, typically as the first or last character in the class. It is not
708 possible to have the character "]" as the end character of a range, since a
709 sequence such as [w-] is interpreted as a class of two characters. The octal or
710 hexadecimal representation of "]" can, however, be used to end a range.
711
712 Ranges operate in ASCII collating sequence. They can also be used for
713 characters specified numerically, for example [\\000-\\037]. If a range such as
714 [W-c] is used when PCRE_CASELESS is set, it matches the letters involved in
715 either case.
716
717 The character types \\d, \\D, \\s, \\S, \\w, and \\W may also appear in a
718 character class, and add the characters that they match to the class. For
719 example, the class [^\\W_] matches any letter or digit.
720
721 All non-alphameric characters other than \\, -, ^ (at the start) and the
722 terminating ] are non-special in character classes, but it does no harm if they
723 are escaped.
724
725
726 .SH VERTICAL BAR
727 Vertical bar characters are used to separate alternative patterns. The matching
728 process tries all the alternatives in turn. For example, the pattern
729
730 gilbert|sullivan
731
732 matches either "gilbert" or "sullivan". Any number of alternatives can be used,
733 and an empty alternative is permitted (matching the empty string).
734
735
736 .SH SUBPATTERNS
737 Subpatterns are delimited by parentheses (round brackets), which can be nested.
738 Marking part of a pattern as a subpattern does two things:
739
740 1. It localizes a set of alternatives. For example, the pattern
741
742 cat(aract|erpillar|)
743
744 matches one of the words "cat", "cataract", or "caterpillar". Without the
745 parentheses, it would match "cataract", "erpillar" or the empty string.
746
747 2. It sets up the subpattern as a capturing subpattern (as defined above).
748 When the whole pattern matches, that portion of the subject string that matched
749 the subpattern is passed back to the caller via the \fIovector\fR argument of
750 \fBpcre_exec()\fR. Opening parentheses are counted from left to right (starting
751 from 1) to obtain the numbers of the capturing subpatterns.
752
753 For example, if the string "the red king" is matched against the pattern
754
755 the ((red|white) (king|queen))
756
757 the captured substrings are "red king", "red", and "king", and are numbered 1,
758 2, and 3.
759
760 The fact that plain parentheses fulfil two functions is not always helpful.
761 There are often times when a grouping subpattern is required without a
762 capturing requirement. If an opening parenthesis is followed by "?:", the
763 subpattern does not do any capturing, and is not counted when computing the
764 number of any subsequent capturing subpatterns. For example, if the string "the
765 white queen" is matched against the pattern
766
767 the ((?:red|white) (king|queen))
768
769 the captured substrings are "white queen" and "queen", and are numbered 1 and
770 2. The maximum number of captured substrings is 99, and the maximum number of
771 all subpatterns, both capturing and non-capturing, is 200.
772
773
774 .SH BACK REFERENCES
775 Outside a character class, a backslash followed by a digit greater than 0 (and
776 possibly further digits) is a back reference to a capturing subpattern earlier
777 (i.e. to its left) in the pattern, provided there have been that many previous
778 capturing left parentheses. However, if the decimal number following the
779 backslash is less than 10, it is always taken as a back reference, and causes
780 an error if there have not been that many previous capturing left parentheses.
781 See the section entitled "Backslash" above for further details of the handling
782 of digits following a backslash.
783
784 A back reference matches whatever actually matched the capturing subpattern in
785 the current subject string, rather than anything matching the subpattern
786 itself. So the pattern
787
788 (sens|respons)e and \\1ibility
789
790 matches "sense and sensibility" and "response and responsibility", but not
791 "sense and responsibility".
792
793 There may be more than one back reference to the same subpattern. If a
794 subpattern has not actually been used in a particular match, then any back
795 references to it always fail. For example, the pattern
796
797 (a|(bc))\\2
798
799 always fails if it starts to match "a" rather than "bc". Because there may be
800 up to 99 back references, all digits following the backslash are taken
801 as part of a potential back reference number. If the pattern continues with a
802 digit character, then some delimiter must be used to terminate the back
803 reference. If the PCRE_EXTENDED option is set, this can be whitespace.
804 Otherwise an empty comment can be used.
805
806
807 .SH REPETITION
808 Repetition is specified by quantifiers, which can follow any of the following
809 items:
810
811 a single character, possibly escaped
812 the . metacharacter
813 a character class
814 a back reference
815 a parenthesized subpattern
816
817 The general repetition quantifier specifies a minimum and maximum number of
818 permitted matches, by giving the two numbers in curly brackets (braces),
819 separated by a comma. The numbers must be less than 65536, and the first must
820 be less than or equal to the second. For example:
821
822 z{2,4}
823
824 matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special
825 character. If the second number is omitted, but the comma is present, there is
826 no upper limit; if the second number and the comma are both omitted, the
827 quantifier specifies an exact number of required matches. Thus
828
829 [aeiou]{3,}
830
831 matches at least 3 successive vowels, but may match many more, while
832
833 \\d{8}
834
835 matches exactly 8 digits. An opening curly bracket that appears in a position
836 where a quantifier is not allowed, or one that does not match the syntax of a
837 quantifier, is taken as a literal character. For example, "{,6}" is not a
838 quantifier, but a literal string of four characters.
839
840 The quantifier {0} is permitted, causing the expression to behave as if the
841 previous item and the quantifier were not present.
842
843 For convenience (and historical compatibility) the three most common
844 quantifiers have single-character abbreviations:
845
846 * is equivalent to {0,}
847 + is equivalent to {1,}
848 ? is equivalent to {0,1}
849
850 By default, the quantifiers are "greedy", that is, they match as much as
851 possible (up to the maximum number of permitted times), without causing the
852 rest of the pattern to fail. The classic example of where this gives problems
853 is in trying to match comments in C programs. These appear between the
854 sequences /* and */ and within the sequence, individual * and / characters may
855 appear. An attempt to match C comments by applying the pattern
856
857 /\\*.*\\*/
858
859 to the string
860
861 /* first command */ not comment /* second comment */
862
863 fails, because it matches the entire string due to the greediness of the .*
864 item.
865
866 However, if a quantifier is followed by a question mark, then it ceases to be
867 greedy, and instead matches the minimum number of times possible, so the
868 pattern
869
870 /\\*.*?\\*/
871
872 does the right thing with the C comments. The meaning of the various
873 quantifiers is not otherwise changed, just the preferred number of matches.
874 Do not confuse this use of question mark with its use as a quantifier in its
875 own right. Because it has two uses, it can sometimes appear doubled, as in
876
877 \\d??\\d
878
879 which matches one digit by preference, but can match two if that is the only
880 way the rest of the pattern matches.
881
882 If the PCRE_UNGREEDY option is set (an option which is not available in Perl)
883 then the quantifiers are not greedy by default, but individual ones can be made
884 greedy by following they by a question mark. In other words, it inverts the
885 default behaviour.
886
887 When a parenthesized subpattern is quantified with a minimum repeat count that
888 is greater than 1 or with a limited maximum, more store is required for the
889 compiled pattern, in proportion to the size of the minimum or maximum.
890
891 If a pattern starts with .* then it is implicitly anchored, since whatever
892 follows will be tried against every character position in the subject string.
893 PCRE treats this as though it were preceded by \\A.
894
895 When a capturing subpattern is repeated, the value captured is the substring
896 that matched the final iteration. For example,
897
898 (\s*tweedle[dume]{3})+\\1
899
900 matches "tweedledum tweedledee tweedledee" but not "tweedledum tweedledee
901 tweedledum".
902
903
904 .SH ASSERTIONS
905 An assertion is a test on the characters following the current matching point
906 that does not actually consume any of those characters. The simple assertions
907 coded as \\b, \\B, \\A, \\Z, ^ and $ are described above. More complicated
908 assertions are coded as subpatterns starting with (?= for positive assertions,
909 and (?! for negative assertions. For example,
910
911 \\w+(?=;)
912
913 matches a word followed by a semicolon, but does not include the semicolon in
914 the match, and
915
916 foo(?!bar)
917
918 matches any occurrence of "foo" that is not followed by "bar". Note that the
919 apparently similar pattern
920
921 (?!foo)bar
922
923 does not find an occurrence of "bar" that is preceded by something other than
924 "foo"; it finds any occurrence of "bar" whatsoever, because the assertion
925 (?!foo) is always true when the next three characters are "bar".
926
927 Assertion subpatterns are not capturing subpatterns, and may not be repeated,
928 because it makes no sense to assert the same thing several times. If an
929 assertion contains capturing subpatterns within it, these are always counted
930 for the purposes of numbering the capturing subpatterns in the whole pattern.
931 Substring capturing is carried out for positive assertions, but it does not
932 make sense for negative assertions.
933
934 Assertions count towards the maximum of 200 parenthesized subpatterns.
935
936
937 .SH ONCE-ONLY SUBPATTERNS
938 The facility described in this section is available only when the PCRE_EXTRA
939 option is set at compile time. It is an extension to Perl regular expressions.
940
941 With both maximizing and minimizing repetition, failure of what follows
942 normally causes the repeated item to be re-evaluated to see if a different
943 number of repeats allows the rest of the pattern to match. Sometimes it is
944 useful to prevent this, either to change the nature of the match, or to cause
945 it fail earlier than it otherwise might when the author of the pattern knows
946 there is no point in carrying on.
947
948 Consider, for example, the pattern \\d+foo when applied to the subject line
949
950 123456bar
951
952 After matching all 6 digits and then failing to match "foo", the normal
953 action of the matcher is to try again with only 5 digits matching the \\d+
954 item, and then with 4, and so on, before ultimately failing. Once-only
955 subpatterns provide the means for specifying that once a portion of the pattern
956 has matched, it is not to be re-evaluated in this way, so the matcher would
957 give up immediately on failing to match "foo" the first time. The notation is
958 another kind of special parenthesis, starting with (?> as in this example:
959
960 (?>\d+)bar
961
962 This kind of parenthesis "locks up" the part of the pattern it contains once
963 it has matched, and a failure further into the pattern is prevented from
964 backtracking into it. Backtracking past it to previous items, however, works as
965 normal.
966
967 For simple cases such as the above example, this feature can be though of as a
968 maximizing repeat that must swallow everything it can. So, while both \\d+ and
969 \\d+? are prepared to adjust the number of digits they match in order to make
970 the rest of the pattern match, (?>\\d+) can only match an entire sequence of
971 digits.
972
973 This construction can of course contain arbitrarily complicated subpatterns,
974 and it can be nested. Contrast with the \\X assertion, which is a Prolog-like
975 "cut".
976
977
978 .SH COMMENTS
979 The sequence (?# marks the start of a comment which continues up to the next
980 closing parenthesis. Nested parentheses are not permitted. The characters
981 that make up a comment play no part in the pattern matching at all.
982
983 If the PCRE_EXTENDED option is set, an unescaped # character outside a
984 character class introduces a comment that continues up to the next newline
985 character in the pattern.
986
987
988 .SH INTERNAL FLAG SETTING
989 If the sequence (?i) occurs anywhere in a pattern, it has the effect of setting
990 the PCRE_CASELESS option, that is, all letters are matched in a
991 case-independent manner. The option applies to the whole pattern, not just to
992 the portion that follows it.
993
994 If the sequence (?m) occurs anywhere in a pattern, it has the effect of setting
995 the PCRE_MULTILINE option, that is, subject strings matched by this pattern are
996 treated as consisting of multiple lines.
997
998 If the sequence (?s) occurs anywhere in a pattern, it has the effect of setting
999 the PCRE_DOTALL option, so that dot metacharacters match newlines as well as
1000 all other characters.
1001
1002 If the sequence (?x) occurs anywhere in a pattern, it has the effect of setting
1003 the PCRE_EXTENDED option, that is, whitespace is ignored and # introduces a
1004 comment that lasts till the next newline. The option applies to the whole
1005 pattern, not just to the portion that follows it.
1006
1007 If the sequence (?U) occurs anywhere in a pattern, it has the effect of setting
1008 the PCRE_UNGREEDY option which inverts the greediness of quantifiers. This is
1009 an extension to Perl's facilities.
1010
1011 If the sequence (?X) occurs in a pattern, it has the effect of setting the
1012 PCRE_EXTRA flag, which turns on some additional features not found in Perl.
1013 This flag setting is special in that it must occur earlier in the pattern than
1014 any of the additional features. It is best put at the start.
1015
1016 If more than one option is required, they can be specified jointly, for example
1017 as (?ix) or (?mi).
1018
1019
1020 .SH PERFORMANCE
1021 Certain items that may appear in patterns are more efficient than others. It is
1022 more efficient to use a character class like [aeiou] than a set of alternatives
1023 such as (a|e|i|o|u). In general, the simplest construction that provides the
1024 required behaviour is usually the most efficient. Jeffrey Friedl's book
1025 contains a lot of discussion about optimizing regular expressions for efficient
1026 performance.
1027
1028 The use of PCRE_MULTILINE causes additional processing and should be avoided
1029 when it is not necessary. Caseless matching of character classes is more
1030 efficient if PCRE_CASELESS is set when the pattern is compiled.
1031
1032
1033 .SH AUTHOR
1034 Philip Hazel <ph10@cam.ac.uk>
1035 .br
1036 University Computing Service,
1037 .br
1038 New Museums Site,
1039 .br
1040 Cambridge CB2 3QG, England.
1041 .br
1042 Phone: +44 1223 334714
1043
1044 Copyright (c) 1998 University of Cambridge.

webmaster@exim.org
ViewVC Help
Powered by ViewVC 1.1.12