ViewVC logotype

Contents of /code/trunk/doc/html/pcretest.html

Parent Directory Parent Directory | Revision Log Revision Log

Revision 75 - (hide annotations) (download) (as text)
Sat Feb 24 21:40:37 2007 UTC (8 years, 1 month ago) by nigel
File MIME type: text/html
File size: 21062 byte(s)
Load pcre-5.0 into code/trunk.

1 nigel 63 <html>
2     <head>
3     <title>pcretest specification</title>
4     </head>
5     <body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB">
6 nigel 75 <h1>pcretest man page</h1>
7     <p>
8     Return to the <a href="index.html">PCRE index page</a>.
9     </p>
10     <p>
11     This page is part of the PCRE HTML documentation. It was generated automatically
12     from the original man page. If there is any nonsense in it, please consult the
13     man page, in case the conversion went wrong.
14     <br>
15 nigel 63 <ul>
16     <li><a name="TOC1" href="#SEC1">SYNOPSIS</a>
17     <li><a name="TOC2" href="#SEC2">OPTIONS</a>
18     <li><a name="TOC3" href="#SEC3">DESCRIPTION</a>
19     <li><a name="TOC4" href="#SEC4">PATTERN MODIFIERS</a>
20 nigel 75 <li><a name="TOC5" href="#SEC5">DATA LINES</a>
21     <li><a name="TOC6" href="#SEC6">OUTPUT FROM PCRETEST</a>
22     <li><a name="TOC7" href="#SEC7">CALLOUTS</a>
23     <li><a name="TOC8" href="#SEC8">SAVING AND RELOADING COMPILED PATTERNS</a>
24     <li><a name="TOC9" href="#SEC9">AUTHOR</a>
25 nigel 63 </ul>
26     <br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br>
27     <P>
28 nigel 75 <b>pcretest [-C] [-d] [-i] [-m] [-o osize] [-p] [-t] [source]</b>
29     <b>[destination]</b>
30 nigel 63 </P>
31     <P>
32     <b>pcretest</b> was written as a test program for the PCRE regular expression
33     library itself, but it can also be used for experimenting with regular
34     expressions. This document describes the features of the test program; for
35     details of the regular expressions themselves, see the
36     <a href="pcrepattern.html"><b>pcrepattern</b></a>
37 nigel 75 documentation. For details of the PCRE library function calls and their
38     options, see the
39 nigel 63 <a href="pcreapi.html"><b>pcreapi</b></a>
40     documentation.
41     </P>
42     <br><a name="SEC2" href="#TOC1">OPTIONS</a><br>
43     <P>
44     <b>-C</b>
45     Output the version number of the PCRE library, and all available information
46     about the optional features that are included, and then exit.
47     </P>
48     <P>
49     <b>-d</b>
50 nigel 75 Behave as if each regex had the <b>/D</b> (debug) modifier; the internal
51 nigel 63 form is output after compilation.
52     </P>
53     <P>
54     <b>-i</b>
55     Behave as if each regex had the <b>/I</b> modifier; information about the
56     compiled pattern is given after compilation.
57     </P>
58     <P>
59     <b>-m</b>
60     Output the size of each compiled pattern after it has been compiled. This is
61 nigel 75 equivalent to adding <b>/M</b> to each regular expression. For compatibility
62     with earlier versions of pcretest, <b>-s</b> is a synonym for <b>-m</b>.
63 nigel 63 </P>
64     <P>
65     <b>-o</b> <i>osize</i>
66 nigel 75 Set the number of elements in the output vector that is used when calling
67     <b>pcre_exec()</b> to be <i>osize</i>. The default value is 45, which is enough
68     for 14 capturing subexpressions. The vector size can be changed for individual
69     matching calls by including \O in the data line (see below).
70 nigel 63 </P>
71     <P>
72     <b>-p</b>
73     Behave as if each regex has <b>/P</b> modifier; the POSIX wrapper API is used
74     to call PCRE. None of the other options has any effect when <b>-p</b> is set.
75     </P>
76     <P>
77     <b>-t</b>
78     Run each compile, study, and match many times with a timer, and output
79 nigel 75 resulting time per compile or match (in milliseconds). Do not set <b>-m</b> with
80     <b>-t</b>, because you will then get the size output a zillion times, and the
81     timing will be distorted.
82 nigel 63 </P>
83     <br><a name="SEC3" href="#TOC1">DESCRIPTION</a><br>
84     <P>
85     If <b>pcretest</b> is given two filename arguments, it reads from the first and
86     writes to the second. If it is given only one filename argument, it reads from
87     that file and writes to stdout. Otherwise, it reads from stdin and writes to
88     stdout, and prompts for each line of input, using "re&#62;" to prompt for regular
89     expressions, and "data&#62;" to prompt for data lines.
90     </P>
91     <P>
92     The program handles any number of sets of input on a single input file. Each
93     set starts with a regular expression, and continues with any number of data
94     lines to be matched against the pattern.
95     </P>
96     <P>
97 nigel 75 Each data line is matched separately and independently. If you want to do
98 nigel 63 multiple-line matches, you have to use the \n escape sequence in a single line
99     of input to encode the newline characters. The maximum length of data line is
100     30,000 characters.
101     </P>
102     <P>
103     An empty line signals the end of the data lines, at which point a new regular
104     expression is read. The regular expressions are given enclosed in any
105 nigel 75 non-alphanumeric delimiters other than backslash, for example
106 nigel 63 <pre>
107     /(a|bc)x+yz/
108 nigel 75 </pre>
109 nigel 63 White space before the initial delimiter is ignored. A regular expression may
110     be continued over several input lines, in which case the newline characters are
111     included within it. It is possible to include the delimiter within the pattern
112     by escaping it, for example
113     <pre>
114     /abc\/def/
115 nigel 75 </pre>
116 nigel 63 If you do so, the escape and the delimiter form part of the pattern, but since
117 nigel 75 delimiters are always non-alphanumeric, this does not affect its interpretation.
118 nigel 63 If the terminating delimiter is immediately followed by a backslash, for
119     example,
120     <pre>
121     /abc/\
122 nigel 75 </pre>
123 nigel 63 then a backslash is added to the end of the pattern. This is done to provide a
124     way of testing the error condition that arises if a pattern finishes with a
125     backslash, because
126     <pre>
127     /abc\/
128 nigel 75 </pre>
129 nigel 63 is interpreted as the first line of a pattern that starts with "abc/", causing
130     pcretest to read the next line as a continuation of the regular expression.
131     </P>
132     <br><a name="SEC4" href="#TOC1">PATTERN MODIFIERS</a><br>
133     <P>
134 nigel 75 A pattern may be followed by any number of modifiers, which are mostly single
135     characters. Following Perl usage, these are referred to below as, for example,
136     "the <b>/i</b> modifier", even though the delimiter of the pattern need not
137     always be a slash, and no slash is used when writing modifiers. Whitespace may
138     appear between the final pattern delimiter and the first modifier, and between
139     the modifiers themselves.
140 nigel 63 </P>
141     <P>
142 nigel 75 The <b>/i</b>, <b>/m</b>, <b>/s</b>, and <b>/x</b> modifiers set the PCRE_CASELESS,
143     PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when
144     <b>pcre_compile()</b> is called. These four modifier letters have the same
145     effect as they do in Perl. For example:
146 nigel 63 <pre>
147     /caseless/i
148 nigel 75 </pre>
149     The following table shows additional modifiers for setting PCRE options that do
150     not correspond to anything in Perl:
151     <pre>
152     <b>/A</b> PCRE_ANCHORED
153     <b>/C</b> PCRE_AUTO_CALLOUT
154     <b>/E</b> PCRE_DOLLAR_ENDONLY
155     <b>/N</b> PCRE_NO_AUTO_CAPTURE
156     <b>/U</b> PCRE_UNGREEDY
157     <b>/X</b> PCRE_EXTRA
158     </pre>
159 nigel 63 Searching for all possible matches within each subject string can be requested
160     by the <b>/g</b> or <b>/G</b> modifier. After finding a match, PCRE is called
161     again to search the remainder of the subject string. The difference between
162     <b>/g</b> and <b>/G</b> is that the former uses the <i>startoffset</i> argument to
163     <b>pcre_exec()</b> to start searching at a new point within the entire string
164     (which is in effect what Perl does), whereas the latter passes over a shortened
165     substring. This makes a difference to the matching process if the pattern
166     begins with a lookbehind assertion (including \b or \B).
167     </P>
168     <P>
169     If any call to <b>pcre_exec()</b> in a <b>/g</b> or <b>/G</b> sequence matches an
170     empty string, the next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED
171     flags set in order to search for another, non-empty, match at the same point.
172     If this second match fails, the start offset is advanced by one, and the normal
173     match is retried. This imitates the way Perl handles such cases when using the
174     <b>/g</b> modifier or the <b>split()</b> function.
175     </P>
176     <P>
177 nigel 75 There are yet more modifiers for controlling the way <b>pcretest</b>
178 nigel 63 operates.
179     </P>
180     <P>
181     The <b>/+</b> modifier requests that as well as outputting the substring that
182     matched the entire pattern, pcretest should in addition output the remainder of
183     the subject string. This is useful for tests where the subject contains
184     multiple copies of the same substring.
185     </P>
186     <P>
187     The <b>/L</b> modifier must be followed directly by the name of a locale, for
188     example,
189     <pre>
190 nigel 75 /pattern/Lfr_FR
191     </pre>
192     For this reason, it must be the last modifier. The given locale is set,
193 nigel 63 <b>pcre_maketables()</b> is called to build a set of character tables for the
194     locale, and this is then passed to <b>pcre_compile()</b> when compiling the
195     regular expression. Without an <b>/L</b> modifier, NULL is passed as the tables
196     pointer; that is, <b>/L</b> applies only to the expression on which it appears.
197     </P>
198     <P>
199     The <b>/I</b> modifier requests that <b>pcretest</b> output information about the
200 nigel 75 compiled pattern (whether it is anchored, has a fixed first character, and
201     so on). It does this by calling <b>pcre_fullinfo()</b> after compiling a
202     pattern. If the pattern is studied, the results of that are also output.
203 nigel 63 </P>
204     <P>
205     The <b>/D</b> modifier is a PCRE debugging feature, which also assumes <b>/I</b>.
206     It causes the internal form of compiled regular expressions to be output after
207     compilation. If the pattern was studied, the information returned is also
208     output.
209     </P>
210     <P>
211 nigel 75 The <b>/F</b> modifier causes <b>pcretest</b> to flip the byte order of the
212     fields in the compiled pattern that contain 2-byte and 4-byte numbers. This
213     facility is for testing the feature in PCRE that allows it to execute patterns
214     that were compiled on a host with a different endianness. This feature is not
215     available when the POSIX interface to PCRE is being used, that is, when the
216     <b>/P</b> pattern modifier is specified. See also the section about saving and
217     reloading compiled patterns below.
218     </P>
219     <P>
220 nigel 63 The <b>/S</b> modifier causes <b>pcre_study()</b> to be called after the
221     expression has been compiled, and the results used when the expression is
222     matched.
223     </P>
224     <P>
225     The <b>/M</b> modifier causes the size of memory block used to hold the compiled
226     pattern to be output.
227     </P>
228     <P>
229     The <b>/P</b> modifier causes <b>pcretest</b> to call PCRE via the POSIX wrapper
230     API rather than its native API. When this is done, all other modifiers except
231     <b>/i</b>, <b>/m</b>, and <b>/+</b> are ignored. REG_ICASE is set if <b>/i</b> is
232     present, and REG_NEWLINE is set if <b>/m</b> is present. The wrapper functions
233     force PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set.
234     </P>
235     <P>
236     The <b>/8</b> modifier causes <b>pcretest</b> to call PCRE with the PCRE_UTF8
237     option set. This turns on support for UTF-8 character handling in PCRE,
238     provided that it was compiled with this support enabled. This modifier also
239     causes any non-printing characters in output strings to be printed using the
240     \x{hh...} notation if they are valid UTF-8 sequences.
241     </P>
242 nigel 71 <P>
243     If the <b>/?</b> modifier is used with <b>/8</b>, it causes <b>pcretest</b> to
244     call <b>pcre_compile()</b> with the PCRE_NO_UTF8_CHECK option, to suppress the
245     checking of the string for UTF-8 validity.
246     </P>
247 nigel 75 <br><a name="SEC5" href="#TOC1">DATA LINES</a><br>
248 nigel 63 <P>
249     Before each data line is passed to <b>pcre_exec()</b>, leading and trailing
250     whitespace is removed, and it is then scanned for \ escapes. Some of these are
251     pretty esoteric features, intended for checking out some of the more
252     complicated features of PCRE. If you are just testing "ordinary" regular
253     expressions, you probably don't need any of these. The following escapes are
254     recognized:
255     <pre>
256     \a alarm (= BEL)
257     \b backspace
258     \e escape
259     \f formfeed
260     \n newline
261     \r carriage return
262     \t tab
263     \v vertical tab
264     \nnn octal character (up to 3 octal digits)
265     \xhh hexadecimal character (up to 2 hex digits)
266 nigel 75 \x{hh...} hexadecimal character, any number of digits in UTF-8 mode
267 nigel 63 \A pass the PCRE_ANCHORED option to <b>pcre_exec()</b>
268     \B pass the PCRE_NOTBOL option to <b>pcre_exec()</b>
269 nigel 75 \Cdd call pcre_copy_substring() for substring dd after a successful match (number less than 32)
270     \Cname call pcre_copy_named_substring() for substring "name" after a successful match (name termin-
271 nigel 63 ated by next non alphanumeric character)
272 nigel 75 \C+ show the current captured substrings at callout time
273 nigel 63 \C- do not supply a callout function
274 nigel 75 \C!n return 1 instead of 0 when callout number n is reached
275     \C!n!m return 1 instead of 0 when callout number n is reached for the nth time
276     \C*n pass the number n (may be negative) as callout data; this is used as the callout return value
277     \Gdd call pcre_get_substring() for substring dd after a successful match (number less than 32)
278     \Gname call pcre_get_named_substring() for substring "name" after a successful match (name termin-
279 nigel 63 ated by next non-alphanumeric character)
280 nigel 75 \L call pcre_get_substringlist() after a successful match
281 nigel 63 \M discover the minimum MATCH_LIMIT setting
282     \N pass the PCRE_NOTEMPTY option to <b>pcre_exec()</b>
283 nigel 75 \Odd set the size of the output vector passed to <b>pcre_exec()</b> to dd (any number of digits)
284     \P pass the PCRE_PARTIAL option to <b>pcre_exec()</b>
285 nigel 73 \S output details of memory get/free calls during matching
286 nigel 63 \Z pass the PCRE_NOTEOL option to <b>pcre_exec()</b>
287 nigel 75 \? pass the PCRE_NO_UTF8_CHECK option to <b>pcre_exec()</b>
288     \&#62;dd start the match at offset dd (any number of digits);
289     this sets the <i>startoffset</i> argument for <b>pcre_exec()</b>
290     </pre>
291     A backslash followed by anything else just escapes the anything else. If the
292     very last character is a backslash, it is ignored. This gives a way of passing
293     an empty line as data, since a real empty line terminates the data input.
294 nigel 63 </P>
295     <P>
296     If \M is present, <b>pcretest</b> calls <b>pcre_exec()</b> several times, with
297     different values in the <i>match_limit</i> field of the <b>pcre_extra</b> data
298     structure, until it finds the minimum number that is needed for
299     <b>pcre_exec()</b> to complete. This number is a measure of the amount of
300     recursion and backtracking that takes place, and checking it out can be
301     instructive. For most simple matches, the number is quite small, but for
302     patterns with very large numbers of matching possibilities, it can become large
303     very quickly with increasing length of subject string.
304     </P>
305     <P>
306 nigel 75 When \O is used, the value specified may be higher or lower than the size set
307     by the <b>-O</b> command line option (or defaulted to 45); \O applies only to
308     the call of <b>pcre_exec()</b> for the line in which it appears.
309 nigel 63 </P>
310     <P>
311 nigel 75 If the <b>/P</b> modifier was present on the pattern, causing the POSIX wrapper
312     API to be used, only \B and \Z have any effect, causing REG_NOTBOL and
313     REG_NOTEOL to be passed to <b>regexec()</b> respectively.
314 nigel 63 </P>
315     <P>
316     The use of \x{hh...} to represent UTF-8 characters is not dependent on the use
317     of the <b>/8</b> modifier on the pattern. It is recognized always. There may be
318     any number of hexadecimal digits inside the braces. The result is from one to
319     six bytes, encoded according to the UTF-8 rules.
320     </P>
321 nigel 75 <br><a name="SEC6" href="#TOC1">OUTPUT FROM PCRETEST</a><br>
322 nigel 63 <P>
323     When a match succeeds, pcretest outputs the list of captured substrings that
324     <b>pcre_exec()</b> returns, starting with number 0 for the string that matched
325 nigel 75 the whole pattern. Otherwise, it outputs "No match" or "Partial match"
326     when <b>pcre_exec()</b> returns PCRE_ERROR_NOMATCH or PCRE_ERROR_PARTIAL,
327     respectively, and otherwise the PCRE negative error number. Here is an example
328     of an interactive pcretest run.
329 nigel 63 <pre>
330     $ pcretest
331 nigel 75 PCRE version 5.00 07-Sep-2004
333 nigel 63 re&#62; /^abc(\d+)/
334     data&#62; abc123
335     0: abc123
336     1: 123
337     data&#62; xyz
338     No match
339 nigel 75 </pre>
340 nigel 63 If the strings contain any non-printing characters, they are output as \0x
341     escapes, or as \x{...} escapes if the <b>/8</b> modifier was present on the
342 nigel 75 pattern. If the pattern has the <b>/+</b> modifier, the output for substring 0
343     is followed by the the rest of the subject string, identified by "0+" like
344     this:
345 nigel 63 <pre>
346     re&#62; /cat/+
347     data&#62; cataract
348     0: cat
349     0+ aract
350 nigel 75 </pre>
351 nigel 63 If the pattern has the <b>/g</b> or <b>/G</b> modifier, the results of successive
352     matching attempts are output in sequence, like this:
353     <pre>
354     re&#62; /\Bi(\w\w)/g
355     data&#62; Mississippi
356     0: iss
357     1: ss
358     0: iss
359     1: ss
360     0: ipp
361     1: pp
362 nigel 75 </pre>
363 nigel 63 "No match" is output only if the first match attempt fails.
364     </P>
365     <P>
366     If any of the sequences <b>\C</b>, <b>\G</b>, or <b>\L</b> are present in a
367     data line that is successfully matched, the substrings extracted by the
368     convenience functions are output with C, G, or L after the string number
369     instead of a colon. This is in addition to the normal full list. The string
370     length (that is, the return from the extraction function) is given in
371     parentheses after each string for <b>\C</b> and <b>\G</b>.
372     </P>
373     <P>
374     Note that while patterns can be continued over several lines (a plain "&#62;"
375     prompt is used for continuations), data lines may not. However newlines can be
376     included in data by means of the \n escape.
377     </P>
378 nigel 75 <br><a name="SEC7" href="#TOC1">CALLOUTS</a><br>
379 nigel 63 <P>
380 nigel 75 If the pattern contains any callout requests, <b>pcretest</b>'s callout function
381     is called during matching. By default, it displays the callout number, the
382     start and current positions in the text at the callout time, and the next
383     pattern item to be tested. For example, the output
384     <pre>
385     ---&#62;pqrabcdef
386     0 ^ ^ \d
387     </pre>
388     indicates that callout number 0 occurred for a match attempt starting at the
389     fourth character of the subject string, when the pointer was at the seventh
390     character of the data, and when the next pattern item was \d. Just one
391     circumflex is output if the start and current positions are the same.
392     </P>
393     <P>
394     Callouts numbered 255 are assumed to be automatic callouts, inserted as a
395     result of the <b>/C</b> pattern modifier. In this case, instead of showing the
396     callout number, the offset in the pattern, preceded by a plus, is output. For
397     example:
398     <pre>
399     re&#62; /\d?[A-E]\*/C
400     data&#62; E*
401     ---&#62;E*
402     +0 ^ \d?
403     +3 ^ [A-E]
404     +8 ^^ \*
405     +10 ^ ^
406     0: E*
407     </pre>
408     The callout function in <b>pcretest</b> returns zero (carry on matching) by
409     default, but you can use an \C item in a data line (as described above) to
410     change this.
411     </P>
412     <P>
413     Inserting callouts can be helpful when using <b>pcretest</b> to check
414     complicated regular expressions. For further information about callouts, see
415     the
416     <a href="pcrecallout.html"><b>pcrecallout</b></a>
417     documentation.
418     </P>
419     <br><a name="SEC8" href="#TOC1">SAVING AND RELOADING COMPILED PATTERNS</a><br>
420     <P>
421     The facilities described in this section are not available when the POSIX
422     inteface to PCRE is being used, that is, when the <b>/P</b> pattern modifier is
423     specified.
424     </P>
425     <P>
426     When the POSIX interface is not in use, you can cause <b>pcretest</b> to write a
427     compiled pattern to a file, by following the modifiers with &#62; and a file name.
428     For example:
429     <pre>
430     /pattern/im &#62;/some/file
431     </pre>
432     See the
433     <a href="pcreprecompile.html"><b>pcreprecompile</b></a>
434     documentation for a discussion about saving and re-using compiled patterns.
435     </P>
436     <P>
437     The data that is written is binary. The first eight bytes are the length of the
438     compiled pattern data followed by the length of the optional study data, each
439     written as four bytes in big-endian order (most significant byte first). If
440     there is no study data (either the pattern was not studied, or studying did not
441     return any data), the second length is zero. The lengths are followed by an
442     exact copy of the compiled pattern. If there is additional study data, this
443     follows immediately after the compiled pattern. After writing the file,
444     <b>pcretest</b> expects to read a new pattern.
445     </P>
446     <P>
447     A saved pattern can be reloaded into <b>pcretest</b> by specifing &#60; and a file
448     name instead of a pattern. The name of the file must not contain a &#60; character,
449     as otherwise <b>pcretest</b> will interpret the line as a pattern delimited by &#60;
450     characters.
451     For example:
452     <pre>
453     re&#62; &#60;/some/file
454     Compiled regex loaded from /some/file
455     No study data
456     </pre>
457     When the pattern has been loaded, <b>pcretest</b> proceeds to read data lines in
458     the usual way.
459     </P>
460     <P>
461     You can copy a file written by <b>pcretest</b> to a different host and reload it
462     there, even if the new host has opposite endianness to the one on which the
463     pattern was compiled. For example, you can compile on an i86 machine and run on
464     a SPARC machine.
465     </P>
466     <P>
467     File names for saving and reloading can be absolute or relative, but note that
468     the shell facility of expanding a file name that starts with a tilde (~) is not
469     available.
470     </P>
471     <P>
472     The ability to save and reload files in <b>pcretest</b> is intended for testing
473     and experimentation. It is not intended for production use because only a
474     single pattern can be written to a file. Furthermore, there is no facility for
475     supplying custom character tables for use with a reloaded pattern. If the
476     original pattern was compiled with custom tables, an attempt to match a subject
477     string using a reloaded pattern is likely to cause <b>pcretest</b> to crash.
478     Finally, if you attempt to load a file that is not in the correct format, the
479     result is undefined.
480     </P>
481     <br><a name="SEC9" href="#TOC1">AUTHOR</a><br>
482     <P>
483 nigel 63 Philip Hazel &#60;ph10@cam.ac.uk&#62;
484     <br>
485     University Computing Service,
486     <br>
487     Cambridge CB2 3QG, England.
488     </P>
489     <P>
490 nigel 75 Last updated: 10 September 2004
491 nigel 63 <br>
492 nigel 75 Copyright &copy; 1997-2004 University of Cambridge.
493     <p>
494     Return to the <a href="index.html">PCRE index page</a>.
495     </p>

ViewVC Help
Powered by ViewVC 1.1.12