/[pcre]/code/trunk/pcre.3
ViewVC logotype

Contents of /code/trunk/pcre.3

Parent Directory Parent Directory | Revision Log Revision Log


Revision 9 - (show annotations) (download)
Sat Feb 24 21:38:13 2007 UTC (7 years, 4 months ago) by nigel
File size: 45313 byte(s)
Load pcre-1.03 into code/trunk.

1 .TH PCRE 3
2 .SH NAME
3 pcre - Perl-compatible regular expressions.
4 .SH SYNOPSIS
5 .B #include <pcre.h>
6 .PP
7 .SM
8 .br
9 .B pcre *pcre_compile(const char *\fIpattern\fR, int \fIoptions\fR,
10 .ti +5n
11 .B char **\fIerrptr\fR, int *\fIerroffset\fR);
12 .PP
13 .br
14 .B pcre_extra *pcre_study(const pcre *\fIcode\fR, int \fIoptions\fR,
15 .ti +5n
16 .B char **\fIerrptr\fR);
17 .PP
18 .br
19 .B int pcre_exec(const pcre *\fIcode\fR, "const pcre_extra *\fIextra\fR,"
20 .ti +5n
21 .B "const char *\fIsubject\fR," int \fIlength\fR, int \fIoptions\fR,
22 .ti +5n
23 .B int *\fIovector\fR, int \fIovecsize\fR);
24 .PP
25 .br
26 .B int pcre_info(const pcre *\fIcode\fR, int *\fIoptptr\fR, int
27 .B *\fIfirstcharptr\fR);
28 .PP
29 .br
30 .B char *pcre_version(void);
31 .PP
32 .br
33 .B void *(*pcre_malloc)(size_t);
34 .PP
35 .br
36 .B void (*pcre_free)(void *);
37 .PP
38 .br
39 .B unsigned char *pcre_cbits[128];
40 .PP
41 .br
42 .B unsigned char *pcre_ctypes[256];
43 .PP
44 .br
45 .B unsigned char *pcre_fcc[256];
46 .PP
47 .br
48 .B unsigned char *pcre_lcc[256];
49
50
51
52 .SH DESCRIPTION
53 The PCRE library is a set of functions that implement regular expression
54 pattern matching using the same syntax and semantics as Perl 5, with just a few
55 differences (see below). The current implementation corresponds to Perl 5.004.
56
57 PCRE has its own native API, which is described in this man page. There is also
58 a set of wrapper functions that correspond to the POSIX API. See
59 \fBpcreposix (3)\fR.
60
61 The three functions \fBpcre_compile()\fR, \fBpcre_study()\fR, and
62 \fBpcre_exec()\fR are used for compiling and matching regular expressions. The
63 function \fBpcre_info()\fR is used to find out information about a compiled
64 pattern, while the function \fBpcre_version()\fR returns a pointer to a string
65 containing the version of PCRE and its date of release.
66
67 The global variables \fBpcre_malloc\fR and \fBpcre_free\fR initially contain
68 the entry points of the standard \fBmalloc()\fR and \fBfree()\fR functions
69 respectively. PCRE calls the memory management functions via these variables,
70 so a calling program can replace them if it wishes to intercept the calls. This
71 should be done before calling any PCRE functions.
72
73 The other global variables are character tables. They are initialized when PCRE
74 is compiled, from source that is generated by reference to the C character type
75 functions, but which the maintainer of PCRE is free to modify. In principle
76 they could also be modified at runtime. See PCRE's README file for more
77 details.
78
79
80 .SH MULTI-THREADING
81 The PCRE functions can be used in multi-threading applications, with the
82 proviso that the character tables and the memory management functions pointed
83 to by \fBpcre_malloc\fR and \fBpcre_free\fR will be shared by all threads.
84
85 The compiled form of a regular expression is not altered during matching, so
86 the same compiled pattern can safely be used by several threads at once.
87
88
89 .SH COMPILING A PATTERN
90 The function \fBpcre_compile()\fR is called to compile a pattern into an
91 internal form. The pattern is a C string terminated by a binary zero, and
92 is passed in the argument \fIpattern\fR. A pointer to the compiled code block
93 is returned. The \fBpcre\fR type is defined for this for convenience, but in
94 fact \fBpcre\fR is just a typedef for \fBvoid\fR, since the contents of the
95 block are not defined.
96 .PP
97 The size of a compiled pattern is roughly proportional to the length of the
98 pattern string, except that each character class (other than those containing
99 just a single character, negated or not) requires 33 bytes, and repeat
100 quantifiers with a minimum greater than one or a bounded maximum cause the
101 relevant portions of the compiled pattern to be replicated.
102 .PP
103 The \fIoptions\fR argument contains independent bits that affect the
104 compilation. It should be zero if no options are required. Those options that
105 are compabible with Perl can also be set at compile time from within the
106 pattern (see the detailed description of regular expressions below) and all
107 options except PCRE_EXTENDED and PCRE_EXTRA can be set at the time of matching.
108 .PP
109 If \fIerrptr\fR is NULL, \fBpcre_compile()\fR returns NULL immediately.
110 Otherwise, if compilation of a pattern fails, \fBpcre_compile()\fR returns
111 NULL, and sets the variable pointed to by \fIerrptr\fR to point to a textual
112 error message.
113
114 The offset from the start of the pattern to the character where the error was
115 discovered is placed in the variable pointed to by \fIerroffset\fR, which must
116 not be NULL. If it is, an immediate error is given.
117 .PP
118 The following option bits are defined in the header file:
119
120 PCRE_ANCHORED
121
122 If this bit is set, the pattern is forced to be "anchored", that is, it is
123 constrained to match only at the start of the string which is being searched
124 (the "subject string"). This effect can also be achieved by appropriate
125 constructs in the pattern itself, which is the only way to do it in Perl.
126
127 PCRE_CASELESS
128
129 If this bit is set, letters in the pattern match both upper and lower case
130 letters in any subject string. It is equivalent to Perl's /i option.
131
132 PCRE_DOLLAR_ENDONLY
133
134 If this bit is set, a dollar metacharacter in the pattern matches only at the
135 end of the subject string. By default, it also matches immediately before the
136 final character if it is a newline (but not before any other newlines). The
137 PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. There is no
138 equivalent to this option in Perl.
139
140 PCRE_DOTALL
141
142 If this bit is set, a dot metacharater in the pattern matches all characters,
143 including newlines. By default, newlines are excluded. This option is
144 equivalent to Perl's /s option. A negative class such as [^a] always matches a
145 newline character, independent of the setting of this option.
146
147 PCRE_EXTENDED
148
149 If this bit is set, whitespace characters in the pattern are totally ignored
150 except when escaped or inside a character class, and characters between an
151 unescaped # outside a character class and the next newline character,
152 inclusive, are also ignored. This is equivalent to Perl's /x option, and makes
153 it possible to include comments inside complicated patterns.
154
155 PCRE_MULTILINE
156
157 By default, PCRE treats the subject string as consisting of a single "line" of
158 characters (even if it actually contains several newlines). The "start of line"
159 metacharacter (^) matches only at the start of the string, while the "end of
160 line" metacharacter ($) matches only at the end of the string, or before a
161 terminating newline. This is the same as Perl.
162
163 When PCRE_MULTILINE it is set, the "start of line" and "end of line" constructs
164 match immediately following or immediately before any newline in the subject
165 string, respectively, as well as at the very start and end. This is equivalent
166 to Perl's /m option. If there are no "\\n" characters in a subject string, or
167 no occurrences of ^ or $ in a pattern, setting PCRE_MULTILINE has no
168 effect.
169
170 PCRE_EXTRA
171
172 This option turns on additional functionality of PCRE that is incompatible with
173 Perl. Any backslash in a pattern that is followed by a letter that has no
174 special meaning causes an error, thus reserving these combinations for future
175 expansion. By default, as in Perl, a backslash followed by a letter with no
176 special meaning is treated as a literal. There are two extra features currently
177 provided, and both are in some sense experimental additions that are useful for
178 influencing the progress of a match.
179
180 (1) The sequence \\X inserts a Prolog-like "cut" into the expression.
181
182 (2) Once a subpattern enclosed in (?>subpat) brackets has matched,
183 backtracking never goes back into the pattern.
184
185 See below for further details of both of these.
186
187
188
189 .SH STUDYING A PATTERN
190 When a pattern is going to be used several times, it is worth spending more
191 time analyzing it in order to speed up the time taken for matching. The
192 function \fBpcre_study()\fR takes a pointer to a compiled pattern as its first
193 argument, and returns a pointer to a \fBpcre_extra\fR block (another \fBvoid\fR
194 typedef) containing additional information about the pattern; this can be
195 passed to \fBpcre_exec()\fR. If no additional information is available, NULL
196 is returned.
197
198 The second argument contains option bits. The only one currently supported is
199 PCRE_CASELESS. It forces the studying to be done in a caseless manner, even if
200 the original pattern was compiled without PCRE_CASELESS. When the result of
201 \fBpcre_study()\fR is passed to \fBpcre_exec()\fR, it is used only if its
202 caseless state is the same as that of the matching process. A pattern that is
203 compiled without PCRE_CASELESS can be studied with and without PCRE_CASELESS,
204 and the appropriate data passed to \fBpcre_exec()\fR with and without the
205 PCRE_CASELESS flag.
206
207 The third argument for \fBpcre_study()\fR is a pointer to an error message. If
208 studying succeeds (even if no data is returned), the variable it points to is
209 set to NULL. Otherwise it points to a textual error message.
210
211 At present, studying a pattern is useful only for non-anchored patterns that do
212 not have a single fixed starting character. A bitmap of possible starting
213 characters is created.
214
215
216 .SH MATCHING A PATTERN
217 The function \fBpcre_exec()\fR is called to match a subject string against a
218 pre-compiled pattern, which is passed in the \fIcode\fR argument. If the
219 pattern has been studied, the result of the study should be passed in the
220 \fIextra\fR argument. Otherwise this must be NULL.
221
222 The subject string is passed as a pointer in \fIsubject\fR and a length in
223 \fIlength\fR. Unlike the pattern string, it may contain binary zero characters.
224
225 The options PCRE_ANCHORED, PCRE_CASELESS, PCRE_DOLLAR_ENDONLY, PCRE_DOTALL, and
226 PCRE_MULTILINE can be passed in the \fIoptions\fR argument, whose unused bits
227 must be zero. However, if a pattern is compiled with any of these options, they
228 cannot be unset when it is obeyed.
229
230 There are also two further options that can be set only at matching time:
231
232 PCRE_NOTBOL
233
234 The first character of the string is not the beginning of a line, so the
235 circumflex metacharacter should not match before it. Setting this without
236 PCRE_MULTILINE (at either compile or match time) causes circumflex never to
237 match.
238
239 PCRE_NOTEOL
240
241 The end of the string is not the end of a line, so the dollar metacharacter
242 should not match it. Setting this without PCRE_MULTILINE (at either compile or
243 match time) causes dollar never to match.
244
245 In general, a pattern matches a certain portion of the subject, and in
246 addition, further substrings from the subject may be picked out by parts of the
247 pattern. Following the usage in Jeffrey Friedl's book, this is called
248 "capturing" in what follows, and the phrase "capturing subpattern" is used for
249 a fragment of a pattern that picks out a substring. PCRE supports several other
250 kinds of parenthesized subpattern that do not cause substrings to be captured.
251
252 Captured substrings are returned to the caller via a vector of integer offsets
253 whose address is passed in \fIovector\fR. The number of elements in the vector
254 is passed in \fIovecsize\fR. This should always be an even number, because the
255 elements are used in pairs. If an odd number is passed, it is rounded down.
256
257 The first element of a pair is set to the offset of the first character in a
258 substring, and the second is set to the offset of the first character after the
259 end of a substring. The first pair, \fIovector[0]\fR and \fIovector[1]\fR,
260 identify the portion of the subject string matched by the entire pattern. The
261 next pair is used for the first capturing subpattern, and so on. The value
262 returned by \fBpcre_exec()\fR is the number of pairs that have been set. If
263 there are no capturing subpatterns, the return value from a successful match
264 is 1, indicating that just the first pair of offsets has been set.
265
266 It is possible for an capturing subpattern number \fIn+1\fR to match some
267 part of the subject when subpattern \fIn\fR has not been used at all. For
268 example, if the string "abc" is matched against the pattern "(a|(z))(bc)",
269 subpatterns 1 and 3 are matched, but 2 is not. When this happens, both offset
270 values corresponding to the unused subpattern are set to -1.
271
272 If a capturing subpattern is matched repeatedly, it is the last portion of the
273 string that it matched that gets returned.
274
275 If the vector is too small to hold all the captured substrings, it is used as
276 far as possible, and the function returns a value of zero. In particular, if
277 the substring offsets are not of interest, \fBpcre_exec()\fR may be called with
278 \fIovector\fR passed as NULL and \fIovecsize\fR as zero. However, if the
279 pattern contains back references and the \fIovector\fR isn't big enough to
280 remember the related substrings, PCRE has to get additional memory for use
281 during matching. Thus it is usually advisable to supply an \fIovector\fR.
282
283 Note that \fBpcre_info()\fR can be used to find out how many capturing
284 subpatterns there are in a compiled pattern.
285
286 If \fBpcre_exec()\fR fails, it returns a negative number. The following are
287 defined in the header file:
288
289 PCRE_ERROR_NOMATCH (-1)
290
291 The subject string did not match the pattern.
292
293 PCRE_ERROR_BADREF (-2)
294
295 There was a back-reference in the pattern to a capturing subpattern that had
296 not previously been set.
297
298 PCRE_ERROR_NULL (-3)
299
300 Either \fIcode\fR or \fIsubject\fR was passed as NULL, or \fIovector\fR was
301 NULL and \fIovecsize\fR was not zero.
302
303 PCRE_ERROR_BADOPTION (-4)
304
305 An unrecognized bit was set in the \fIoptions\fR argument.
306
307 PCRE_ERROR_BADMAGIC (-5)
308
309 PCRE stores a 4-byte "magic number" at the start of the compiled code, to catch
310 the case when it is passed a junk pointer. This is the error it gives when the
311 magic number isn't present.
312
313 PCRE_ERROR_UNKNOWN_NODE (-6)
314
315 While running the pattern match, an unknown item was encountered in the
316 compiled pattern. This error could be caused by a bug in PCRE or by overwriting
317 of the compiled pattern.
318
319 PCRE_ERROR_NOMEMORY (-7)
320
321 If a pattern contains back references, but the \fIovector\fR that is passed to
322 \fBpcre_exec()\fR is not big enough to remember the referenced substrings, PCRE
323 gets a block of memory at the start of matching to use for this purpose. If the
324 call via \fBpcre_malloc()\fR fails, this error is given. The memory is freed at
325 the end of matching.
326
327
328 .SH INFORMATION ABOUT A PATTERN
329 The \fBpcre_info()\fR function returns information about a compiled pattern.
330 Its yield is the number of capturing subpatterns, or one of the following
331 negative numbers:
332
333 PCRE_ERROR_NULL the argument \fIcode\fR was NULL
334 PCRE_ERROR_BADMAGIC the "magic number" was not found
335
336 If the \fIoptptr\fR argument is not NULL, a copy of the options with which the
337 pattern was compiled is placed in the integer it points to.
338
339 If the \fIfirstcharptr\fR argument is not NULL, is is used to pass back
340 information about the first character of any matched string. If there is a
341 fixed first character, e.g. from a pattern such as (cat|cow|coyote), then it is
342 returned in the integer pointed to by \fIfirstcharptr\fR. Otherwise, if the
343 pattern was compiled with the PCRE_MULTILINE option, and every branch started
344 with "^", then -1 is returned, indicating that the pattern will match at the
345 start of a subject string or after any "\\n" within the string. Otherwise -2 is
346 returned.
347
348
349 .SH LIMITATIONS
350 There are some size limitations in PCRE but it is hoped that they will never in
351 practice be relevant.
352 The maximum length of a compiled pattern is 65539 (sic) bytes.
353 All values in repeating quantifiers must be less than 65536.
354 The maximum number of capturing subpatterns is 99.
355 The maximum number of all parenthesized subpatterns, including capturing
356 subpatterns and assertions, is 200.
357
358 The maximum length of a subject string is the largest positive number that an
359 integer variable can hold. However, PCRE uses recursion to handle subpatterns
360 and indefinite repetition. This means that the available stack space may limit
361 the size of a subject string that can be processed by certain patterns.
362
363
364 .SH DIFFERENCES FROM PERL
365 The differences described here are with respect to Perl 5.004.
366
367 1. By default, a whitespace character is any character that the C library
368 function \fBisspace()\fR recognizes, though it is possible to compile PCRE with
369 alternative character type tables. Normally \fBisspace()\fR matches space,
370 formfeed, newline, carriage return, horizontal tab, and vertical tab. Perl 5
371 no longer includes vertical tab in its set of whitespace characters. The \\v
372 escape that was in the Perl documentation for a long time was never in fact
373 recognized. However, the character itself was treated as whitespace at least
374 up to 5.002. In 5.004 it does not match \\s.
375
376 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl permits
377 them, but they do not mean what you might think. For example, "(?!a){3}" does
378 not assert that the next three characters are not "a". It just asserts that the
379 next character is not "a" three times.
380
381 3. Capturing subpatterns that occur inside negative lookahead assertions are
382 counted, but their entries in the offsets vector are never set. Perl sets its
383 numerical variables from any such patterns that are matched before the
384 assertion fails to match something (thereby succeeding), but only if the
385 negative lookahead assertion contains just one branch.
386
387 4. Though binary zero characters are supported in the subject string, they are
388 not allowed in a pattern string because it is passed as a normal C string,
389 terminated by zero. The escape sequence "\\0" can be used in the pattern to
390 represent a binary zero.
391
392 5. The following Perl escape sequences are not supported: \\l, \\u, \\L, \\U,
393 \\E, \\Q. In fact these are implemented by Perl's general string-handling and
394 are not part of its pattern matching engine.
395
396 6. The Perl \\G assertion is not supported as it is not relevant to single
397 pattern matches.
398
399 7. If a backreference can never be matched, PCRE diagnoses an error. In a case
400 like
401
402 /(123)\\2/
403
404 the error occurs at compile time. Perl gives no compile time error; version
405 5.004 either always fails to match, or gives a segmentation fault at runtime.
406 In more complicated cases such as
407
408 /(1)(2)(3)(4)(5)(6)(7)(8)(9)(10\\10)/
409
410 PCRE returns PCRE_ERROR_BADREF at run time. Perl always fails to match.
411
412 8. PCRE provides some extensions to the Perl regular expression facilities:
413
414 (a) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ meta-
415 character matches only at the very end of the string.
416
417 (b) If PCRE_EXTRA is set, the \\X assertion (a Prolog-like "cut") is
418 recognized, and a backslash followed by a letter with no special meaning is
419 faulted. There is also a new kind of parenthesized subpattern starting with (?>
420 which has a block on backtracking into it once it has matched.
421
422
423 .SH REGULAR EXPRESSION DETAILS
424 The syntax and semantics of the regular expressions supported by PCRE are
425 described below. Regular expressions are also described in the Perl
426 documentation and in a number of other books, some of which have copious
427 examples. Jeffrey Friedl's "Mastering Regular Expressions", published by
428 O'Reilly (ISBN 1-56592-257-3), covers them in great detail. The description
429 here is intended as reference documentation.
430
431 A regular expression is a pattern that is matched against a subject string from
432 left to right. Most characters stand for themselves in a pattern, and match the
433 corresponding characters in the subject. As a trivial example, the pattern
434
435 The quick brown fox
436
437 matches a portion of a subject string that is identical to itself. The power of
438 regular expressions comes from the ability to include alternatives and
439 repetitions in the pattern. These are encoded in the pattern by the use of
440 \fImeta-characters\fR, which do not stand for themselves but instead are
441 interpreted in some special way.
442
443 There are two different sets of meta-characters: those that are recognized
444 anywhere in the pattern except within square brackets, and those that are
445 recognized in square brackets. Outside square brackets, the meta-characters are
446 as follows:
447
448 \\ general escape character with several uses
449 ^ assert start of subject (or line, in multiline mode)
450 $ assert end of subject (or line, in multiline mode)
451 . match any character except newline (by default)
452 [ start character class definition
453 | start of alternative branch
454 ( start subpattern
455 ) end subpattern
456 ? extends the meaning of (
457 also 0 or 1 quantifier
458 also quantifier minimizer
459 * 0 or more quantifier
460 + 1 or more quantifier
461 { start min/max quantifier
462
463 Part of a pattern that is in square brackets is called a "character class". In
464 a character class the only meta-characters are:
465
466 \\ general escape character
467 ^ negate the class, but only if the first character
468 - indicates character range
469 ] terminates the character class
470
471 The following sections describe the use of each of the meta-characters.
472
473
474 .SH BACKSLASH
475 The backslash character has several uses. Firstly, if it is followed by a
476 non-alphameric character, it takes away any special meaning that character may
477 have. This use of backslash as an escape character applies both inside and
478 outside character classes.
479
480 For example, if you want to match a "*" character, you write "\\*" in the
481 pattern. This applies whether or not the following character would otherwise be
482 interpreted as a meta-character, so it is always safe to precede a
483 non-alphameric with "\\" to specify that it stands for itself. In particular,
484 if you want to match a backslash, you write "\\\\".
485
486 If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the
487 pattern and characters between a "#" outside a character class and the next
488 newline character are ignored. An escaping backslash can be used to include a
489 whitespace or "#" character as part of the pattern.
490
491 A second use of backslash provides a way of encoding non-printing characters
492 in patterns in a visible manner. There is no restriction on the appearance of
493 non-printing characters, apart from the binary zero that terminates a pattern,
494 but when a pattern is being prepared by text editing, it is usually easier to
495 use one of the following escape sequences than the binary character it
496 represents:
497
498 \\a alarm, that is, the BEL character (hex 07)
499 \\cx "control-x", where x is any character
500 \\e escape (hex 1B)
501 \\f formfeed (hex 0C)
502 \\n newline (hex 0A)
503 \\r carriage return (hex 0D)
504 \\t tab (hex 09)
505 \\xhh character with hex code hh
506 \\ddd character with octal code ddd or backreference
507
508 The precise effect of "\\cx" is as follows: if "x" is a lower case letter, it
509 is converted to upper case. Then bit 6 of the character (hex 40) is inverted.
510 Thus "\\cz" becomes hex 1A, but "\\c{" becomes hex 3B, while "\\c;" becomes hex
511 7B.
512
513 After "\\x", up to two hexadecimal digits are read (letters can be in upper or
514 lower case).
515
516 After "\\0" up to two further octal digits are read. In both cases, if there
517 are fewer than two digits, just those that are present are used. Thus the
518 sequence "\\0\\x\\07" specifies two binary zeros followed by a BEL character.
519 Make sure you supply two digits if the character that follows could otherwise
520 be taken as another digit.
521
522 The handling of a backslash followed by a digit other than 0 is complicated.
523 Outside a character class, PCRE reads it and any following digits as a decimal
524 number. If the number is less than 10, or if there have been at least that many
525 previous capturing left parentheses in the expression, the entire sequence is
526 taken as a \fIback reference\fR. A description of how this works is given
527 later, following the discussion of parenthesized subpatterns.
528
529 Inside a character class, or if the decimal number is greater than 9 and there
530 have not been that many capturing subpatterns, PCRE re-reads up to three octal
531 digits following the backslash, and generates a single byte from the least
532 significant 8 bits of the value. Any subsequent digits stand for themselves.
533 For example:
534
535 \\040 is another way of writing a space
536 \\40 is the same, provided there are fewer than 40
537 previous capturing subpatterns
538 \\7 is always a back reference
539 \\11 might be a back reference, or another way of
540 writing a tab
541 \\011 is always a tab
542 \\0113 is a tab followed by the character "3"
543 \\113 is the character with octal code 113 (since there
544 can be no more than 99 back references)
545 \\377 is a byte consisting entirely of 1 bits
546 \\81 is either a back reference, or a binary zero
547 followed by the two characters "8" and "1"
548
549 Note that octal values of 100 or greater must not be introduced by a leading
550 zero, because no more than three octal digits are ever read.
551
552 All the sequences that define a single byte value can be used both inside and
553 outside character classes. In addition, inside a character class, the sequence
554 "\\b" is interpreted as the backspace character (hex 08). Outside a character
555 class it has a different meaning (see below).
556
557 The third use of backslash is for specifying generic character types:
558
559 \\d any decimal digit
560 \\D any character that is not a decimal digit
561 \\s any whitespace character
562 \\S any character that is not a whitespace character
563 \\w any "word" character
564 \\W any "non-word" character
565
566 Each pair of escape sequences partitions the complete set of characters into
567 two disjoint sets. Any given character matches one, and only one, of each pair.
568
569 A "word" character is any letter or digit or the underscore character, that is,
570 any character which can be part of a Perl "word". These character type
571 sequences can appear both inside and outside character classes. They each match
572 one character of the appropriate type. If the current matching point is at the
573 end of the subject string, all of them fail, since there is no character to
574 match.
575
576 The fourth use of backslash is for certain assertions. An assertion specifies a
577 condition that has to be met at a particular point in a match, without
578 consuming any characters from the subject string. The backslashed assertions
579 are
580
581 \\b word boundary
582 \\B not a word boundary
583 \\A start of subject (independent of multiline mode)
584 \\Z end of subject (independent of multiline mode)
585
586 Assertions may not appear in character classes (but note that "\\b" has a
587 different meaning, namely the backspace character, inside a character class).
588
589 A word boundary is a position in the subject string where the current character
590 and the previous character do not both match "\\w" or "\\W" (i.e. one matches
591 "\\w" and the other matches "\\W"), or the start or end of the string if the
592 first or last character matches "\\w", respectively. More complicated
593 assertions are also supported (see below).
594
595 The "\\A" and "\\Z" assertions differ from the traditional "^" and "$"
596 (described below) in that they only ever match at the very start and end of the
597 subject string, respectively, whatever options are set.
598
599 When the PCRE_EXTRA flag is set on a call to \fBpcre_compile()\fR, the
600 additional assertion \\X, which has no equivalent in Perl, is recognized.
601 This operates like the "cut" operation in Prolog: it prevents the matching
602 operation from backtracking past it. For example, if the expression
603
604 .*/foo
605
606 is matched against the string "/this/string/is/not" then after the greedy .*
607 has swallowed the whole string, PCRE keeps backtracking all the way to the
608 beginning before failing. If, on the other hand, the expression is
609
610 .*/\\Xfoo
611
612 then once it has discovered that "/not" is not "/foo", backtracking ceases, and
613 the match fails. See also the section on "once-only" subpatterns below.
614
615
616
617 .SH CIRCUMFLEX AND DOLLAR
618 Outside a character class, the circumflex character is an assertion which is
619 true only if the current matching point is at the start of the subject string,
620 in the default matching mode. Inside a character class, circumflex has an
621 entirely different meaning (see below).
622
623 Circumflex need not be the first character of the pattern if a number of
624 alternatives are involved, but it should be the first thing in each alternative
625 in which it appears if the pattern is ever to match that branch. If all
626 possible alternatives start with a circumflex, that is, if the pattern is
627 constrained to match only at the start of the subject, it is said to be an
628 "anchored" pattern. (There are also other constructs that can cause a pattern
629 to be anchored.)
630
631 A dollar character is an assertion which is true only if the current matching
632 point is at the end of the subject string, or immediately before a newline
633 character that is the last character in the string (by default). Dollar need
634 not be the last character of the pattern if a number of alternatives are
635 involved, but it should be the last item in any branch in which it appears.
636 Dollar has no special meaning in a character class.
637
638 The meaning of dollar can be changed so that it matches only at the very end of
639 the string, by setting the PCRE_DOLLAR_ENDONLY option at compile or matching
640 time.
641
642 The meanings of the circumflex and dollar characters are changed if the
643 PCRE_MULTILINE option is set at compile or matching time. When this is the
644 case, they match immediately after and immediately before an internal "\\n"
645 character, respectively, in addition to matching at the start and end of the
646 subject string. For example, the pattern /^abc$/ matches the subject string
647 "def\\nabc" in multiline mode, but not otherwise. Consequently, patterns that
648 are anchored in single line mode because all branches start with "^" are not
649 anchored in multiline mode. The PCRE_DOLLAR_ENDONLY option is ignored if
650 PCRE_MULTILINE is set.
651
652 Note that the sequences "\\A" and "\\Z" can be used to match the start and end
653 of the subject in both modes, and if all branches of a pattern start with "\\A"
654 is it always anchored.
655
656
657 .SH FULL STOP (PERIOD, DOT)
658 Outside a character class, a dot in the pattern matches any one character in
659 the subject, including a non-printing character, but not (by default) newline.
660 If the PCRE_DOTALL option is set, then dots match newlines as well. The
661 handling of dot is entirely independent of the handling of circumflex and
662 dollar, the only relationship being that they both involve newline characters.
663 Dot has no special meaning in a character class.
664
665
666 .SH SQUARE BRACKETS
667 An opening square bracket introduces a character class, terminated by a closing
668 square bracket. A closing square bracket on its own is not special. If a
669 closing square bracket is required as a member of the class, it should be the
670 first data character in the class (after an initial circumflex, if present) or
671 escaped with \\.
672
673 A character class matches a single character in the subject; the character must
674 be in the set of characters defined by the class, unless the first character in
675 the class is a circumflex, in which case the subject character must not be in
676 the set defined by the class. If a circumflex is actually required as a member
677 of the class, ensure it is not the first character, or escape it with \\.
678
679 For example, the character class [aeiou] matches any lower case vowel, while
680 [^aeiou] matches any character that is not a lower case vowel. Note that a
681 circumflex is just a convenient notation for specifying the characters which
682 are in the class by enumerating those that are not. It is not an assertion: it
683 still consumes a character from the subject string, and fails if the current
684 pointer is at the end of the string.
685
686 The newline character is never treated in any special way in character classes,
687 whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE options is. A class
688 such as [^a] will always match a newline.
689
690 The minus (hyphen) character can be used to specify a range of characters in a
691 character class. For example, [d-m] matches any letter between d and m,
692 inclusive. If a minus character is required in a class, it must be escaped with
693 \\ or appear in a position where it cannot be interpreted as indicating a
694 range, typically as the first or last character in the class. It is not
695 possible to have the character "]" as the end character of a range, since a
696 sequence such as [w-] is interpreted as a class of two characters. The octal or
697 hexadecimal representation of "]" can, however, be used to end a range.
698
699 Ranges operate in ASCII collating sequence. They can also be used for
700 characters specified numerically, for example [\\000-\\037]. If a range such as
701 [W-c] is used when PCRE_CASELESS is set, it matches the letters involved in
702 either case.
703
704 The character types \\d, \\D, \\s, \\S, \\w, and \\W may also appear in a
705 character class, and add the characters that they match to the class. For
706 example, the class [^\\W_] matches any letter or digit.
707
708 All non-alphameric characters other than \\, -, ^ (at the start) and the
709 terminating ] are non-special in character classes, but it does no harm if they
710 are escaped.
711
712
713 .SH VERTICAL BAR
714 Vertical bar characters are used to separate alternative patterns. The matching
715 process tries all the alternatives in turn. For example, the pattern
716
717 gilbert|sullivan
718
719 matches either "gilbert" or "sullivan". Any number of alternatives can be used,
720 and an empty alternative is permitted (matching the empty string).
721
722
723 .SH SUBPATTERNS
724 Subpatterns are delimited by parentheses (round brackets), which can be nested.
725 Marking part of a pattern as a subpattern does two things:
726
727 1. It localizes a set of alternatives. For example, the pattern
728
729 cat(aract|erpillar|)
730
731 matches one of the words "cat", "cataract", or "caterpillar". Without the
732 parentheses, it would match "cataract", "erpillar" or the empty string.
733
734 2. It sets up the subpattern as a capturing subpattern (as defined above).
735 When the whole pattern matches, that portion of the subject string that matched
736 the subpattern is passed back to the caller via the \fIovector\fR argument of
737 \fBpcre_exec()\fR. Opening parentheses are counted from left to right (starting
738 from 1) to obtain the numbers of the capturing subpatterns.
739
740 For example, if the string "the red king" is matched against the pattern
741
742 the ((red|white) (king|queen))
743
744 the captured substrings are "red king", "red", and "king", and are numbered 1,
745 2, and 3.
746
747 The fact that plain parentheses fulfil two functions is not always helpful.
748 There are often times when a grouping subpattern is required without a
749 capturing requirement. If an opening parenthesis is followed by "?:", the
750 subpattern does not do any capturing, and is not counted when computing the
751 number of any subsequent capturing subpatterns. For example, if the string "the
752 white queen" is matched against the pattern
753
754 the ((?:red|white) (king|queen))
755
756 the captured substrings are "white queen" and "queen", and are numbered 1 and
757 2. The maximum number of captured substrings is 99, and the maximum number of
758 all subpatterns, both capturing and non-capturing, is 200.
759
760
761 .SH BACK REFERENCES
762 Outside a character class, a backslash followed by a digit greater than 0 (and
763 possibly further digits) is a back reference to a capturing subpattern earlier
764 (i.e. to its left) in the pattern, provided there have been that many previous
765 capturing left parentheses. However, if the decimal number following the
766 backslash is less than 10, it is always taken as a back reference, and causes
767 an error if there have not been that many previous capturing left parentheses.
768 See the section entitled "Backslash" above for further details of the handling
769 of digits following a backslash.
770
771 A back reference matches whatever actually matched the capturing subpattern in
772 the current subject string, rather than anything matching the subpattern
773 itself. So the pattern
774
775 (sens|respons)e and \\1ibility
776
777 matches "sense and sensibility" and "response and responsibility", but not
778 "sense and responsibility".
779
780 There may be more than one back reference to the same subpattern. If a
781 subpattern has not actually been used in a particular match, then any back
782 references to it always fail. For example, the pattern
783
784 (a|(bc))\\2
785
786 always fails if it starts to match "a" rather than "bc". Because there may be
787 up to 99 back references, all digits following the backslash are taken
788 as part of a potential back reference number. If the pattern continues with a
789 digit character, then some delimiter must be used to terminate the back
790 reference. If the PCRE_EXTENDED option is set, this can be whitespace.
791 Otherwise an empty comment can be used.
792
793
794 .SH REPETITION
795 Repetition is specified by quantifiers, which can follow any of the following
796 items:
797
798 a single character, possibly escaped
799 the . metacharacter
800 a character class
801 a back reference
802 a parenthesized subpattern
803
804 The general repetition quantifier specifies a minimum and maximum number of
805 permitted matches, by giving the two numbers in curly brackets (braces),
806 separated by a comma. The numbers must be less than 65536, and the first must
807 be less than or equal to the second. For example:
808
809 z{2,4}
810
811 matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special
812 character. If the second number is omitted, but the comma is present, there is
813 no upper limit; if the second number and the comma are both omitted, the
814 quantifier specifies an exact number of required matches. Thus
815
816 [aeiou]{3,}
817
818 matches at least 3 successive vowels, but may match many more, while
819
820 \\d{8}
821
822 matches exactly 8 digits. An opening curly bracket that appears in a position
823 where a quantifier is not allowed, or one that does not match the syntax of a
824 quantifier, is taken as a literal character. For example, "{,6}" is not a
825 quantifier, but a literal string of four characters.
826
827 The quantifier {0} is permitted, causing the expression to behave as if the
828 previous item and the quantifier were not present.
829
830 For convenience (and historical compatibility) the three most common
831 quantifiers have single-character abbreviations:
832
833 * is equivalent to {0,}
834 + is equivalent to {1,}
835 ? is equivalent to {0,1}
836
837 By default, the quantifiers are "greedy", that is, they match as much as
838 possible (up to the maximum number of permitted times), without causing the
839 rest of the pattern to fail. The classic example of where this gives problems
840 is in trying to match comments in C programs. These appear between the
841 sequences /* and */ and within the sequence, individual * and / characters may
842 appear. An attempt to match C comments by applying the pattern
843
844 /\\*.*\\*/
845
846 to the string
847
848 /* first command */ not comment /* second comment */
849
850 fails, because it matches the entire string due to the greediness of the .*
851 item.
852
853 However, if a quantifier is followed by a question mark, then it ceases to be
854 greedy, and instead matches the minimum number of times possible, so the
855 pattern
856
857 /\\*.*?\\*/
858
859 does the right thing with the C comments. The meaning of the various
860 quantifiers is not otherwise changed, just the preferred number of matches.
861 Do not confuse this use of question mark with its use as a quantifier in its
862 own right. Because it has two uses, it can sometimes appear doubled, as in
863
864 \\d??\\d
865
866 which matches one digit by preference, but can match two if that is the only
867 way the rest of the pattern matches.
868
869 When a parenthesized subpattern is quantified with a minimum repeat count that
870 is greater than 1 or with a limited maximum, more store is required for the
871 compiled pattern, in proportion to the size of the minimum or maximum.
872
873 If a pattern starts with .* then it is implicitly anchored, since whatever
874 follows will be tried against every character position in the subject string.
875 PCRE treats this as though it were preceded by \\A.
876
877 When a capturing subpattern is repeated, the value captured is the substring
878 that matched the final iteration. For example,
879
880 (\s*tweedle[dume]{3})+\\1
881
882 matches "tweedledum tweedledee tweedledee" but not "tweedledum tweedledee
883 tweedledum".
884
885
886 .SH ASSERTIONS
887 An assertion is a test on the characters following the current matching point
888 that does not actually consume any of those characters. The simple assertions
889 coded as \\b, \\B, \\A, \\Z, ^ and $ are described above. More complicated
890 assertions are coded as subpatterns starting with (?= for positive assertions,
891 and (?! for negative assertions. For example,
892
893 \\w+(?=;)
894
895 matches a word followed by a semicolon, but does not include the semicolon in
896 the match, and
897
898 foo(?!bar)
899
900 matches any occurrence of "foo" that is not followed by "bar". Note that the
901 apparently similar pattern
902
903 (?!foo)bar
904
905 does not find an occurrence of "bar" that is preceded by something other than
906 "foo"; it finds any occurrence of "bar" whatsoever, because the assertion
907 (?!foo) is always true when the next three characters are "bar".
908
909 Assertion subpatterns are not capturing subpatterns, and may not be repeated,
910 because it makes no sense to assert the same thing several times. If an
911 assertion contains capturing subpatterns within it, these are always counted
912 for the purposes of numbering the capturing subpatterns in the whole pattern.
913 Substring capturing is carried out for positive assertions, but it does not
914 make sense for negative assertions.
915
916 Assertions count towards the maximum of 200 parenthesized subpatterns.
917
918
919 .SH ONCE-ONLY SUBPATTERNS
920 The facility described in this section is available only when the PCRE_EXTRA
921 option is set at compile time. It is an extension to Perl regular expressions.
922
923 With both maximizing and minimizing repetition, failure of what follows
924 normally causes the repeated item to be re-evaluated to see if a different
925 number of repeats allows the rest of the pattern to match. Sometimes it is
926 useful to prevent this, either to change the nature of the match, or to cause
927 it fail earlier than it otherwise might when the author of the pattern knows
928 there is no point in carrying on.
929
930 Consider, for example, the pattern \\d+foo when applied to the subject line
931
932 123456bar
933
934 After matching all 6 digits and then failing to match "foo", the normal
935 action of the matcher is to try again with only 5 digits matching the \\d+
936 item, and then with 4, and so on, before ultimately failing. Once-only
937 subpatterns provide the means for specifying that once a portion of the pattern
938 has matched, it is not to be re-evaluated in this way, so the matcher would
939 give up immediately on failing to match "foo" the first time. The notation is
940 another kind of special parenthesis, starting with (?> as in this example:
941
942 (?>\d+)bar
943
944 This kind of parenthesis "locks up" the part of the pattern it contains once
945 it has matched, and a failure further into the pattern is prevented from
946 backtracking into it. Backtracking past it to previous items, however, works as
947 normal.
948
949 For simple cases such as the above example, this feature can be though of as a
950 maximizing repeat that must swallow everything it can. So, while both \\d+ and
951 \\d+? are prepared to adjust the number of digits they match in order to make
952 the rest of the pattern match, (?>\\d+) can only match an entire sequence of
953 digits.
954
955 This construction can of course contain arbitrarily complicated subpatterns,
956 and it can be nested. Contrast with the \\X assertion, which is a Prolog-like
957 "cut".
958
959
960 .SH COMMENTS
961 The sequence (?# marks the start of a comment which continues up to the next
962 closing parenthesis. Nested parentheses are not permitted. The characters
963 that make up a comment play no part in the pattern matching at all.
964
965 If the PCRE_EXTENDED option is set, an unescaped # character outside a
966 character class introduces a comment that continues up to the next newline
967 character in the pattern.
968
969
970 .SH INTERNAL FLAG SETTING
971 If the sequence (?i) occurs anywhere in a pattern, it has the effect of setting
972 the PCRE_CASELESS option, that is, all letters are matched in a
973 case-independent manner. The option applies to the whole pattern, not just to
974 the portion that follows it.
975
976 If the sequence (?m) occurs anywhere in a pattern, it has the effect of setting
977 the PCRE_MULTILINE option, that is, subject strings matched by this pattern are
978 treated as consisting of multiple lines.
979
980 If the sequence (?s) occurs anywhere in a pattern, it has the effect of setting
981 the PCRE_DOTALL option, so that dot metacharacters match newlines as well as
982 all other characters.
983
984 If the sequence (?x) occurs anywhere in a pattern, it has the effect of setting
985 the PCRE_EXTENDED option, that is, whitespace is ignored and # introduces a
986 comment that lasts till the next newline. The option applies to the whole
987 pattern, not just to the portion that follows it.
988
989 If more than one option is required, they can be specified jointly, for example
990 as (?ix) or (?mi).
991
992
993 .SH PERFORMANCE
994 Certain items that may appear in patterns are more efficient than others. It is
995 more efficient to use a character class like [aeiou] than a set of alternatives
996 such as (a|e|i|o|u). In general, the simplest construction that provides the
997 required behaviour is usually the most efficient. Jeffrey Friedl's book
998 contains a lot of discussion about optimizing regular expressions for efficient
999 performance.
1000
1001 The use of PCRE_MULTILINE causes additional processing and should be avoided
1002 when it is not necessary. Caseless matching of character classes is more
1003 efficient if PCRE_CASELESS is set when the pattern is compiled.
1004
1005
1006 .SH AUTHOR
1007 Philip Hazel <ph10@cam.ac.uk>
1008 .br
1009 University Computing Service,
1010 .br
1011 New Museums Site,
1012 .br
1013 Cambridge CB2 3QG, England.
1014 .br
1015 Phone: +44 1223 334714
1016
1017 Copyright (c) 1997 University of Cambridge.

webmaster@exim.org
ViewVC Help
Powered by ViewVC 1.1.12