ViewVC logotype

Contents of /code/trunk/doc/html/pcretest.html

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1459 - (hide annotations) (download) (as text)
Tue Mar 4 10:45:15 2014 UTC (13 months, 2 weeks ago) by ph10
File MIME type: text/html
File size: 51976 byte(s)
Preparations for next release.

1 nigel 63 <html>
2     <head>
3     <title>pcretest specification</title>
4     </head>
5     <body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB">
6 nigel 75 <h1>pcretest man page</h1>
7     <p>
8     Return to the <a href="index.html">PCRE index page</a>.
9     </p>
10 ph10 111 <p>
11 nigel 75 This page is part of the PCRE HTML documentation. It was generated automatically
12     from the original man page. If there is any nonsense in it, please consult the
13     man page, in case the conversion went wrong.
14 ph10 111 <br>
15 nigel 63 <ul>
16     <li><a name="TOC1" href="#SEC1">SYNOPSIS</a>
17 ph10 1320 <li><a name="TOC2" href="#SEC2">INPUT DATA FORMAT</a>
18     <li><a name="TOC3" href="#SEC3">PCRE's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a>
19     <li><a name="TOC4" href="#SEC4">COMMAND LINE OPTIONS</a>
20     <li><a name="TOC5" href="#SEC5">DESCRIPTION</a>
21     <li><a name="TOC6" href="#SEC6">PATTERN MODIFIERS</a>
22     <li><a name="TOC7" href="#SEC7">DATA LINES</a>
23     <li><a name="TOC8" href="#SEC8">THE ALTERNATIVE MATCHING FUNCTION</a>
24     <li><a name="TOC9" href="#SEC9">DEFAULT OUTPUT FROM PCRETEST</a>
26     <li><a name="TOC11" href="#SEC11">RESTARTING AFTER A PARTIAL MATCH</a>
27     <li><a name="TOC12" href="#SEC12">CALLOUTS</a>
28     <li><a name="TOC13" href="#SEC13">NON-PRINTING CHARACTERS</a>
29     <li><a name="TOC14" href="#SEC14">SAVING AND RELOADING COMPILED PATTERNS</a>
30     <li><a name="TOC15" href="#SEC15">SEE ALSO</a>
31     <li><a name="TOC16" href="#SEC16">AUTHOR</a>
32     <li><a name="TOC17" href="#SEC17">REVISION</a>
33 nigel 63 </ul>
34     <br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br>
35     <P>
36 ph10 654 <b>pcretest [options] [input file [output file]]</b>
37 nigel 91 <br>
38     <br>
39 nigel 63 <b>pcretest</b> was written as a test program for the PCRE regular expression
40     library itself, but it can also be used for experimenting with regular
41     expressions. This document describes the features of the test program; for
42     details of the regular expressions themselves, see the
43     <a href="pcrepattern.html"><b>pcrepattern</b></a>
44 nigel 75 documentation. For details of the PCRE library function calls and their
45     options, see the
46 nigel 63 <a href="pcreapi.html"><b>pcreapi</b></a>
47 ph10 1194 ,
48     <a href="pcre16.html"><b>pcre16</b></a>
49 ph10 869 and
50 ph10 1194 <a href="pcre32.html"><b>pcre32</b></a>
51 ph10 1221 documentation.
52 nigel 63 </P>
53     <P>
54 ph10 1194 The input for <b>pcretest</b> is a sequence of regular expression patterns and
55     strings to be matched, as described below. The output shows the result of each
56     match. Options on the command line and the patterns control PCRE options and
57     exactly what is output.
58     </P>
59     <P>
60     As PCRE has evolved, it has acquired many different features, and as a result,
61     <b>pcretest</b> now has rather a lot of obscure options for testing every
62     possible feature. Some of these options are specifically designed for use in
63     conjunction with the test script and data files that are distributed as part of
64 ph10 1221 PCRE, and are unlikely to be of use otherwise. They are all documented here,
65 ph10 1194 but without much justification.
66     </P>
67 ph10 1320 <br><a name="SEC2" href="#TOC1">INPUT DATA FORMAT</a><br>
68 ph10 1194 <P>
69 ph10 1320 Input to <b>pcretest</b> is processed line by line, either by calling the C
70 ph10 1335 library's <b>fgets()</b> function, or via the <b>libreadline</b> library (see
71     below). In Unix-like environments, <b>fgets()</b> treats any bytes other than
72     newline as data characters. However, in some Windows environments character 26
73     (hex 1A) causes an immediate end of file, and no further data is read. For
74     maximum portability, therefore, it is safest to use only ASCII characters in
75 ph10 1320 <b>pcretest</b> input files.
76     </P>
77     <br><a name="SEC3" href="#TOC1">PCRE's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a><br>
78     <P>
79 ph10 903 From release 8.30, two separate PCRE libraries can be built. The original one
80     supports 8-bit character strings, whereas the newer 16-bit library supports
81 ph10 1320 character strings encoded in 16-bit units. From release 8.32, a third library
82     can be built, supporting character strings encoded in 32-bit units. The
83     <b>pcretest</b> program can be used to test all three libraries. However, it is
84     itself still an 8-bit program, reading 8-bit input and writing 8-bit output.
85     When testing the 16-bit or 32-bit library, the patterns and data strings are
86     converted to 16- or 32-bit format before being passed to the PCRE library
87     functions. Results are converted to 8-bit for output.
88 ph10 869 </P>
89     <P>
90 ph10 1194 References to functions and structures of the form <b>pcre[16|32]_xx</b> below
91 ph10 1320 mean "<b>pcre_xx</b> when using the 8-bit library, <b>pcre16_xx</b> when using
92     the 16-bit library, or <b>pcre32_xx</b> when using the 32-bit library".
93 ph10 869 </P>
94 ph10 1320 <br><a name="SEC4" href="#TOC1">COMMAND LINE OPTIONS</a><br>
95 ph10 869 <P>
96 ph10 1194 <b>-8</b>
97     If both the 8-bit library has been built, this option causes the 8-bit library
98     to be used (which is the default); if the 8-bit library has not been built,
99 ph10 869 this option causes an error.
100     </P>
101     <P>
102 ph10 1194 <b>-16</b>
103     If both the 8-bit or the 32-bit, and the 16-bit libraries have been built, this
104     option causes the 16-bit library to be used. If only the 16-bit library has been
105     built, this is the default (so has no effect). If only the 8-bit or the 32-bit
106     library has been built, this option causes an error.
107     </P>
108     <P>
109     <b>-32</b>
110     If both the 8-bit or the 16-bit, and the 32-bit libraries have been built, this
111     option causes the 32-bit library to be used. If only the 32-bit library has been
112     built, this is the default (so has no effect). If only the 8-bit or the 16-bit
113     library has been built, this option causes an error.
114     </P>
115     <P>
116 nigel 93 <b>-b</b>
117 ph10 654 Behave as if each pattern has the <b>/B</b> (show byte code) modifier; the
118     internal form is output after compilation.
119 nigel 93 </P>
120     <P>
121 nigel 63 <b>-C</b>
122     Output the version number of the PCRE library, and all available information
123 ph10 1335 about the optional features that are included, and then exit with zero exit
124 ph10 1298 code. All other options are ignored.
125 nigel 63 </P>
126     <P>
127 ph10 869 <b>-C</b> <i>option</i>
128 ph10 903 Output information about a specific build-time option, then exit. This
129     functionality is intended for use in scripts such as <b>RunTest</b>. The
130 ph10 1298 following options output the value and set the exit code as indicated:
131 ph10 869 <pre>
132 ph10 1194 ebcdic-nl the code for LF (= NL) in an EBCDIC environment:
133 ph10 1221 0x15 or 0x25
134 ph10 1194 0 if used in an ASCII environment
135 ph10 1335 exit code is always 0
136 ph10 1298 linksize the configured internal link size (2, 3, or 4)
137 ph10 1335 exit code is set to the link size
138 ph10 903 newline the default newline setting:
139     CR, LF, CRLF, ANYCRLF, or ANY
140 ph10 1335 exit code is always 0
141 ph10 1459 bsr the default setting for what \R matches:
142     ANYCRLF or ANY
143     exit code is always 0
144 ph10 869 </pre>
145 ph10 1335 The following options output 1 for true or 0 for false, and set the exit code
146 ph10 1298 to the same value:
147 ph10 869 <pre>
148 ph10 1194 ebcdic compiled for an EBCDIC environment
149 ph10 869 jit just-in-time support is available
150     pcre16 the 16-bit library was built
151 ph10 1194 pcre32 the 32-bit library was built
152 ph10 869 pcre8 the 8-bit library was built
153 ph10 903 ucp Unicode property support is available
154 ph10 1335 utf UTF-8 and/or UTF-16 and/or UTF-32 support
155 ph10 1298 is available
156     </pre>
157 ph10 1335 If an unknown option is given, an error message is output; the exit code is 0.
158 ph10 869 </P>
159     <P>
160 nigel 63 <b>-d</b>
161 ph10 654 Behave as if each pattern has the <b>/D</b> (debug) modifier; the internal
162 nigel 93 form and information about the compiled pattern is output after compilation;
163     <b>-d</b> is equivalent to <b>-b -i</b>.
164 nigel 63 </P>
165     <P>
166 nigel 77 <b>-dfa</b>
167     Behave as if each data line contains the \D escape sequence; this causes the
168 ph10 1194 alternative matching function, <b>pcre[16|32]_dfa_exec()</b>, to be used instead
169     of the standard <b>pcre[16|32]_exec()</b> function (more detail is given below).
170 nigel 77 </P>
171     <P>
172 nigel 93 <b>-help</b>
173     Output a brief summary these options and then exit.
174     </P>
175     <P>
176 nigel 63 <b>-i</b>
177 ph10 654 Behave as if each pattern has the <b>/I</b> modifier; information about the
178 nigel 63 compiled pattern is given after compilation.
179     </P>
180     <P>
181 ph10 392 <b>-M</b>
182     Behave as if each data line contains the \M escape sequence; this causes
183     PCRE to discover the minimum MATCH_LIMIT and MATCH_LIMIT_RECURSION settings by
184 ph10 1194 calling <b>pcre[16|32]_exec()</b> repeatedly with different limits.
185 ph10 392 </P>
186     <P>
187 nigel 63 <b>-m</b>
188     Output the size of each compiled pattern after it has been compiled. This is
189 ph10 903 equivalent to adding <b>/M</b> to each regular expression. The size is given in
190 ph10 869 bytes for both libraries.
191 nigel 63 </P>
192     <P>
193 ph10 1404 <b>-O</b>
194     Behave as if each pattern has the <b>/O</b> modifier, that is disable
195     auto-possessification for all patterns.
196     </P>
197     <P>
198 nigel 63 <b>-o</b> <i>osize</i>
199 nigel 75 Set the number of elements in the output vector that is used when calling
200 ph10 1194 <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b> to be <i>osize</i>. The
201 ph10 869 default value is 45, which is enough for 14 capturing subexpressions for
202 ph10 1194 <b>pcre[16|32]_exec()</b> or 22 different matches for
203     <b>pcre[16|32]_dfa_exec()</b>.
204 ph10 869 The vector size can be changed for individual matching calls by including \O
205     in the data line (see below).
206 nigel 63 </P>
207     <P>
208     <b>-p</b>
209 ph10 654 Behave as if each pattern has the <b>/P</b> modifier; the POSIX wrapper API is
210 nigel 77 used to call PCRE. None of the other options has any effect when <b>-p</b> is
211 ph10 869 set. This option can be used only with the 8-bit library.
212 nigel 63 </P>
213     <P>
214 nigel 91 <b>-q</b>
215 nigel 87 Do not output the version number of <b>pcretest</b> at the start of execution.
216     </P>
217     <P>
218 nigel 91 <b>-S</b> <i>size</i>
219 ph10 654 On Unix-like systems, set the size of the run-time stack to <i>size</i>
220 nigel 91 megabytes.
221     </P>
222     <P>
223 ph10 691 <b>-s</b> or <b>-s+</b>
224 ph10 654 Behave as if each pattern has the <b>/S</b> modifier; in other words, force each
225 ph10 930 pattern to be studied. If <b>-s+</b> is used, all the JIT compile options are
226 ph10 1194 passed to <b>pcre[16|32]_study()</b>, causing just-in-time optimization to be set
227 ph10 975 up if it is available, for both full and partial matching. Specific JIT compile
228     options can be selected by following <b>-s+</b> with a digit in the range 1 to
229 ph10 930 7, which selects the JIT compile modes as follows:
230     <pre>
231     1 normal match only
232     2 soft partial match only
233     3 normal match and soft partial match
234     4 hard partial match only
235     6 soft and hard partial match
236     7 all three modes (default)
237     </pre>
238     If <b>-s++</b> is used instead of <b>-s+</b> (with or without a following digit),
239     the text "(JIT)" is added to the first output line after a match or no match
240     when JIT-compiled code was actually used.
241 ph10 1194 <br>
242     <br>
243 ph10 1221 Note that there are pattern options that can override <b>-s</b>, either
244 ph10 1194 specifying no studying at all, or suppressing JIT compilation.
245     <br>
246     <br>
247 ph10 930 If the <b>/I</b> or <b>/D</b> option is present on a pattern (requesting output
248     about the compiled pattern), information about the result of studying is not
249     included when studying is caused only by <b>-s</b> and neither <b>-i</b> nor
250     <b>-d</b> is present on the command line. This behaviour means that the output
251     from tests that are run with and without <b>-s</b> should be identical, except
252     when options that output information about the actual running of a match are
253     set.
254 ph10 869 <br>
255     <br>
256     The <b>-M</b>, <b>-t</b>, and <b>-tm</b> options, which give information about
257     resources used, are likely to produce different output with and without
258     <b>-s</b>. Output may also differ if the <b>/C</b> option is present on an
259     individual pattern. This uses callouts to trace the the matching process, and
260     this may be different between studied and non-studied patterns. If the pattern
261     contains (*MARK) items there may also be differences, for the same reason. The
262     <b>-s</b> command line option can be overridden for specific patterns that
263     should never be studied (see the <b>/S</b> pattern modifier below).
264 ph10 654 </P>
265     <P>
266 nigel 63 <b>-t</b>
267 ph10 1404 Run each compile, study, and match many times with a timer, and output the
268     resulting times per compile, study, or match (in milliseconds). Do not set
269     <b>-m</b> with <b>-t</b>, because you will then get the size output a zillion
270     times, and the timing will be distorted. You can control the number of
271     iterations that are used for timing by following <b>-t</b> with a number (as a
272     separate item on the command line). For example, "-t 1000" iterates 1000 times.
273     The default is to iterate 500000 times.
274 nigel 63 </P>
275 nigel 93 <P>
276     <b>-tm</b>
277     This is like <b>-t</b> except that it times only the matching phase, not the
278     compile or study phases.
279     </P>
280 ph10 1404 <P>
281     <b>-T</b> <b>-TM</b>
282     These behave like <b>-t</b> and <b>-tm</b>, but in addition, at the end of a run,
283     the total times for all compiles, studies, and matches are output.
284     </P>
285 ph10 1320 <br><a name="SEC5" href="#TOC1">DESCRIPTION</a><br>
286 nigel 63 <P>
287     If <b>pcretest</b> is given two filename arguments, it reads from the first and
288     writes to the second. If it is given only one filename argument, it reads from
289     that file and writes to stdout. Otherwise, it reads from stdin and writes to
290     stdout, and prompts for each line of input, using "re&#62;" to prompt for regular
291     expressions, and "data&#62;" to prompt for data lines.
292     </P>
293     <P>
294 ph10 289 When <b>pcretest</b> is built, a configuration option can specify that it should
295     be linked with the <b>libreadline</b> library. When this is done, if the input
296     is from a terminal, it is read using the <b>readline()</b> function. This
297     provides line-editing and history facilities. The output from the <b>-help</b>
298     option states whether or not <b>readline()</b> will be used.
299     </P>
300     <P>
301 nigel 63 The program handles any number of sets of input on a single input file. Each
302     set starts with a regular expression, and continues with any number of data
303 ph10 1404 lines to be matched against that pattern.
304 nigel 63 </P>
305     <P>
306 nigel 75 Each data line is matched separately and independently. If you want to do
307 nigel 91 multi-line matches, you have to use the \n escape sequence (or \r or \r\n,
308 nigel 93 etc., depending on the newline setting) in a single line of input to encode the
309     newline sequences. There is no limit on the length of data lines; the input
310 nigel 91 buffer is automatically extended if it is too small.
311 nigel 63 </P>
312     <P>
313     An empty line signals the end of the data lines, at which point a new regular
314     expression is read. The regular expressions are given enclosed in any
315 nigel 91 non-alphanumeric delimiters other than backslash, for example:
316 nigel 63 <pre>
317     /(a|bc)x+yz/
318 nigel 75 </pre>
319 nigel 63 White space before the initial delimiter is ignored. A regular expression may
320     be continued over several input lines, in which case the newline characters are
321     included within it. It is possible to include the delimiter within the pattern
322     by escaping it, for example
323     <pre>
324     /abc\/def/
325 nigel 75 </pre>
326 nigel 63 If you do so, the escape and the delimiter form part of the pattern, but since
327 nigel 75 delimiters are always non-alphanumeric, this does not affect its interpretation.
328 nigel 63 If the terminating delimiter is immediately followed by a backslash, for
329     example,
330     <pre>
331     /abc/\
332 nigel 75 </pre>
333 nigel 63 then a backslash is added to the end of the pattern. This is done to provide a
334     way of testing the error condition that arises if a pattern finishes with a
335     backslash, because
336     <pre>
337     /abc\/
338 nigel 75 </pre>
339 nigel 63 is interpreted as the first line of a pattern that starts with "abc/", causing
340     pcretest to read the next line as a continuation of the regular expression.
341     </P>
342 ph10 1320 <br><a name="SEC6" href="#TOC1">PATTERN MODIFIERS</a><br>
343 nigel 63 <P>
344 nigel 75 A pattern may be followed by any number of modifiers, which are mostly single
345 ph10 1194 characters, though some of these can be qualified by further characters.
346     Following Perl usage, these are referred to below as, for example, "the
347     <b>/i</b> modifier", even though the delimiter of the pattern need not always be
348     a slash, and no slash is used when writing modifiers. White space may appear
349     between the final pattern delimiter and the first modifier, and between the
350     modifiers themselves. For reference, here is a complete list of modifiers. They
351     fall into several groups that are described in detail in the following
352     sections.
353     <pre>
354     <b>/8</b> set UTF mode
355 ph10 1335 <b>/9</b> set PCRE_NEVER_UTF (locks out UTF mode)
356 ph10 1194 <b>/?</b> disable UTF validity check
357     <b>/+</b> show remainder of subject after match
358     <b>/=</b> show all captures (not just those that are set)
360     <b>/A</b> set PCRE_ANCHORED
361     <b>/B</b> show compiled code
362     <b>/C</b> set PCRE_AUTO_CALLOUT
363     <b>/D</b> same as <b>/B</b> plus <b>/I</b>
364     <b>/E</b> set PCRE_DOLLAR_ENDONLY
365     <b>/F</b> flip byte order in compiled pattern
366     <b>/f</b> set PCRE_FIRSTLINE
367     <b>/G</b> find all matches (shorten string)
368     <b>/g</b> find all matches (use startoffset)
369     <b>/I</b> show information about pattern
370     <b>/i</b> set PCRE_CASELESS
371     <b>/J</b> set PCRE_DUPNAMES
372     <b>/K</b> show backtracking control names
373     <b>/L</b> set locale
374     <b>/M</b> show compiled memory size
375     <b>/m</b> set PCRE_MULTILINE
376     <b>/N</b> set PCRE_NO_AUTO_CAPTURE
377 ph10 1404 <b>/O</b> set PCRE_NO_AUTO_POSSESS
378 ph10 1194 <b>/P</b> use the POSIX wrapper
379 ph10 1459 <b>/Q</b> test external stack check function
380 ph10 1194 <b>/S</b> study the pattern after compilation
381     <b>/s</b> set PCRE_DOTALL
382     <b>/T</b> select character tables
383     <b>/U</b> set PCRE_UNGREEDY
384     <b>/W</b> set PCRE_UCP
385     <b>/X</b> set PCRE_EXTRA
386     <b>/x</b> set PCRE_EXTENDED
387     <b>/Y</b> set PCRE_NO_START_OPTIMIZE
388 ph10 1221 <b>/Z</b> don't show lengths in <b>/B</b> output
389 ph10 1194
390     <b>/&#60;any&#62;</b> set PCRE_NEWLINE_ANY
391     <b>/&#60;anycrlf&#62;</b> set PCRE_NEWLINE_ANYCRLF
392     <b>/&#60;cr&#62;</b> set PCRE_NEWLINE_CR
393     <b>/&#60;crlf&#62;</b> set PCRE_NEWLINE_CRLF
394     <b>/&#60;lf&#62;</b> set PCRE_NEWLINE_LF
395     <b>/&#60;bsr_anycrlf&#62;</b> set PCRE_BSR_ANYCRLF
396     <b>/&#60;bsr_unicode&#62;</b> set PCRE_BSR_UNICODE
397     <b>/&#60;JS&#62;</b> set PCRE_JAVASCRIPT_COMPAT
399     </PRE>
400 nigel 63 </P>
401 ph10 1194 <br><b>
402     Perl-compatible modifiers
403     </b><br>
404 nigel 63 <P>
405 nigel 75 The <b>/i</b>, <b>/m</b>, <b>/s</b>, and <b>/x</b> modifiers set the PCRE_CASELESS,
406     PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when
407 ph10 1194 <b>pcre[16|32]_compile()</b> is called. These four modifier letters have the same
408 nigel 75 effect as they do in Perl. For example:
409 nigel 63 <pre>
410     /caseless/i
411 ph10 1194
412     </PRE>
413     </P>
414     <br><b>
415     Modifiers for other PCRE options
416     </b><br>
417     <P>
418 ph10 535 The following table shows additional modifiers for setting PCRE compile-time
419     options that do not correspond to anything in Perl:
420 nigel 75 <pre>
421 ph10 903 <b>/8</b> PCRE_UTF8 ) when using the 8-bit
422 ph10 869 <b>/?</b> PCRE_NO_UTF8_CHECK ) library
424     <b>/8</b> PCRE_UTF16 ) when using the 16-bit
425     <b>/?</b> PCRE_NO_UTF16_CHECK ) library
427 ph10 1194 <b>/8</b> PCRE_UTF32 ) when using the 32-bit
428     <b>/?</b> PCRE_NO_UTF32_CHECK ) library
430 ph10 1320 <b>/9</b> PCRE_NEVER_UTF
431 ph10 231 <b>/A</b> PCRE_ANCHORED
432     <b>/C</b> PCRE_AUTO_CALLOUT
433     <b>/E</b> PCRE_DOLLAR_ENDONLY
434     <b>/f</b> PCRE_FIRSTLINE
435     <b>/J</b> PCRE_DUPNAMES
436     <b>/N</b> PCRE_NO_AUTO_CAPTURE
437 ph10 1404 <b>/O</b> PCRE_NO_AUTO_POSSESS
438 ph10 231 <b>/U</b> PCRE_UNGREEDY
439 ph10 535 <b>/W</b> PCRE_UCP
440 ph10 231 <b>/X</b> PCRE_EXTRA
441 ph10 579 <b>/Y</b> PCRE_NO_START_OPTIMIZE
442 ph10 1194 <b>/&#60;any&#62;</b> PCRE_NEWLINE_ANY
443     <b>/&#60;anycrlf&#62;</b> PCRE_NEWLINE_ANYCRLF
444 ph10 231 <b>/&#60;cr&#62;</b> PCRE_NEWLINE_CR
445 ph10 1194 <b>/&#60;crlf&#62;</b> PCRE_NEWLINE_CRLF
446 ph10 231 <b>/&#60;lf&#62;</b> PCRE_NEWLINE_LF
447     <b>/&#60;bsr_anycrlf&#62;</b> PCRE_BSR_ANYCRLF
448     <b>/&#60;bsr_unicode&#62;</b> PCRE_BSR_UNICODE
449 ph10 1194 <b>/&#60;JS&#62;</b> PCRE_JAVASCRIPT_COMPAT
450 nigel 75 </pre>
451 ph10 518 The modifiers that are enclosed in angle brackets are literal strings as shown,
452 ph10 654 including the angle brackets, but the letters within can be in either case.
453     This example sets multiline matching with CRLF as the line ending sequence:
454 nigel 93 <pre>
455 ph10 654 /^abc/m&#60;CRLF&#62;
456 nigel 93 </pre>
457 ph10 1194 As well as turning on the PCRE_UTF8/16/32 option, the <b>/8</b> modifier causes
458 ph10 869 all non-printing characters in output strings to be printed using the
459 ph10 903 \x{hh...} notation. Otherwise, those less than 0x100 are output in hex without
460 ph10 869 the curly brackets.
461     </P>
462     <P>
463     Full details of the PCRE options are given in the
464 nigel 91 <a href="pcreapi.html"><b>pcreapi</b></a>
465 ph10 535 documentation.
466 nigel 91 </P>
467     <br><b>
468     Finding all matches in a string
469     </b><br>
470     <P>
471 nigel 63 Searching for all possible matches within each subject string can be requested
472     by the <b>/g</b> or <b>/G</b> modifier. After finding a match, PCRE is called
473     again to search the remainder of the subject string. The difference between
474     <b>/g</b> and <b>/G</b> is that the former uses the <i>startoffset</i> argument to
475 ph10 1194 <b>pcre[16|32]_exec()</b> to start searching at a new point within the entire
476 ph10 869 string (which is in effect what Perl does), whereas the latter passes over a
477     shortened substring. This makes a difference to the matching process if the
478     pattern begins with a lookbehind assertion (including \b or \B).
479 nigel 63 </P>
480     <P>
481 ph10 1194 If any call to <b>pcre[16|32]_exec()</b> in a <b>/g</b> or <b>/G</b> sequence matches
482 ph10 869 an empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and
483 ph10 453 PCRE_ANCHORED flags set in order to search for another, non-empty, match at the
484 ph10 567 same point. If this second match fails, the start offset is advanced, and the
485     normal match is retried. This imitates the way Perl handles such cases when
486 ph10 579 using the <b>/g</b> modifier or the <b>split()</b> function. Normally, the start
487     offset is advanced by one character, but if the newline convention recognizes
488     CRLF as a newline, and the current character is CR followed by LF, an advance
489 ph10 567 of two is used.
490 nigel 63 </P>
491 nigel 91 <br><b>
492     Other modifiers
493     </b><br>
494 nigel 63 <P>
495 nigel 75 There are yet more modifiers for controlling the way <b>pcretest</b>
496 nigel 63 operates.
497     </P>
498     <P>
499     The <b>/+</b> modifier requests that as well as outputting the substring that
500 ph10 654 matched the entire pattern, <b>pcretest</b> should in addition output the
501     remainder of the subject string. This is useful for tests where the subject
502     contains multiple copies of the same substring. If the <b>+</b> modifier appears
503     twice, the same action is taken for captured substrings. In each case the
504     remainder is output on the following line with a plus character following the
505 ph10 691 capture number. Note that this modifier must not immediately follow the /S
506 ph10 930 modifier because /S+ and /S++ have other meanings.
507 nigel 63 </P>
508     <P>
509 ph10 654 The <b>/=</b> modifier requests that the values of all potential captured
510 ph10 869 parentheses be output after a match. By default, only those up to the highest
511     one actually used in the match are output (corresponding to the return code
512 ph10 1194 from <b>pcre[16|32]_exec()</b>). Values in the offsets vector corresponding to
513 ph10 869 higher numbers should be set to -1, and these are output as "&#60;unset&#62;". This
514     modifier gives a way of checking that this is happening.
515 ph10 654 </P>
516     <P>
517 nigel 93 The <b>/B</b> modifier is a debugging feature. It requests that <b>pcretest</b>
518 ph10 869 output a representation of the compiled code after compilation. Normally this
519     information contains length and offset values; however, if <b>/Z</b> is also
520     present, this data is replaced by spaces. This is a special feature for use in
521     the automatic test scripts; it ensures that the same output is generated for
522     different internal link sizes.
523 nigel 93 </P>
524     <P>
525     The <b>/D</b> modifier is a PCRE debugging feature, and is equivalent to
526 ph10 148 <b>/BI</b>, that is, both the <b>/B</b> and the <b>/I</b> modifiers.
527 nigel 63 </P>
528     <P>
529 nigel 75 The <b>/F</b> modifier causes <b>pcretest</b> to flip the byte order of the
530 ph10 869 2-byte and 4-byte fields in the compiled pattern. This facility is for testing
531     the feature in PCRE that allows it to execute patterns that were compiled on a
532     host with a different endianness. This feature is not available when the POSIX
533     interface to PCRE is being used, that is, when the <b>/P</b> pattern modifier is
534     specified. See also the section about saving and reloading compiled patterns
535     below.
536 nigel 75 </P>
537     <P>
538 ph10 512 The <b>/I</b> modifier requests that <b>pcretest</b> output information about the
539     compiled pattern (whether it is anchored, has a fixed first character, and
540 ph10 1194 so on). It does this by calling <b>pcre[16|32]_fullinfo()</b> after compiling a
541 ph10 1459 pattern. If the pattern is studied, the results of that are also output. In
542     this output, the word "char" means a non-UTF character, that is, the value of a
543     single data item (8-bit, 16-bit, or 32-bit, depending on the library that is
544     being tested).
545 nigel 63 </P>
546     <P>
547 ph10 512 The <b>/K</b> modifier requests <b>pcretest</b> to show names from backtracking
548 ph10 1194 control verbs that are returned from calls to <b>pcre[16|32]_exec()</b>. It causes
549     <b>pcretest</b> to create a <b>pcre[16|32]_extra</b> block if one has not already
550     been created by a call to <b>pcre[16|32]_study()</b>, and to set the
551 ph10 869 PCRE_EXTRA_MARK flag and the <b>mark</b> field within it, every time that
552 ph10 1194 <b>pcre[16|32]_exec()</b> is called. If the variable that the <b>mark</b> field
553 ph10 869 points to is non-NULL for a match, non-match, or partial match, <b>pcretest</b>
554     prints the string to which it points. For a match, this is shown on a line by
555     itself, tagged with "MK:". For a non-match it is added to the message.
556 ph10 512 </P>
557     <P>
558     The <b>/L</b> modifier must be followed directly by the name of a locale, for
559     example,
560     <pre>
561     /pattern/Lfr_FR
562     </pre>
563     For this reason, it must be the last modifier. The given locale is set,
564 ph10 1194 <b>pcre[16|32]_maketables()</b> is called to build a set of character tables for
565     the locale, and this is then passed to <b>pcre[16|32]_compile()</b> when compiling
566 ph10 869 the regular expression. Without an <b>/L</b> (or <b>/T</b>) modifier, NULL is
567     passed as the tables pointer; that is, <b>/L</b> applies only to the expression
568     on which it appears.
569 ph10 512 </P>
570     <P>
571 ph10 869 The <b>/M</b> modifier causes the size in bytes of the memory block used to hold
572     the compiled pattern to be output. This does not include the size of the
573 ph10 1194 <b>pcre[16|32]</b> block; it is just the actual compiled data. If the pattern is
574 ph10 869 successfully studied with the PCRE_STUDY_JIT_COMPILE option, the size of the
575     JIT compiled code is also output.
576 nigel 63 </P>
577     <P>
578 ph10 1459 The <b>/Q</b> modifier is used to test the use of <b>pcre_stack_guard</b>. It
579     must be followed by '0' or '1', specifying the return code to be given from an
580     external function that is passed to PCRE and used for stack checking during
581     compilation (see the
582     <a href="pcreapi.html"><b>pcreapi</b></a>
583     documentation for details).
584     </P>
585     <P>
586 ph10 1194 The <b>/S</b> modifier causes <b>pcre[16|32]_study()</b> to be called after the
587     expression has been compiled, and the results used when the expression is
588 ph10 1221 matched. There are a number of qualifying characters that may follow <b>/S</b>.
589 ph10 1194 They may appear in any order.
590     </P>
591     <P>
592 ph10 1404 If <b>/S</b> is followed by an exclamation mark, <b>pcre[16|32]_study()</b> is
593     called with the PCRE_STUDY_EXTRA_NEEDED option, causing it always to return a
594 ph10 1194 <b>pcre_extra</b> block, even when studying discovers no useful information.
595     </P>
596     <P>
597     If <b>/S</b> is followed by a second S character, it suppresses studying, even
598 ph10 654 if it was requested externally by the <b>-s</b> command line option. This makes
599     it possible to specify that certain patterns are always studied, and others are
600     never studied, independently of <b>-s</b>. This feature is used in the test
601     files in a few cases where the output is different when the pattern is studied.
602 nigel 63 </P>
603 ph10 545 <P>
604 ph10 1194 If the <b>/S</b> modifier is followed by a + character, the call to
605     <b>pcre[16|32]_study()</b> is made with all the JIT study options, requesting
606 ph10 930 just-in-time optimization support if it is available, for both normal and
607 ph10 975 partial matching. If you want to restrict the JIT compiling modes, you can
608 ph10 930 follow <b>/S+</b> with a digit in the range 1 to 7:
609     <pre>
610     1 normal match only
611     2 soft partial match only
612     3 normal match and soft partial match
613     4 hard partial match only
614     6 soft and hard partial match
615     7 all three modes (default)
616     </pre>
617     If <b>/S++</b> is used instead of <b>/S+</b> (with or without a following digit),
618     the text "(JIT)" is added to the first output line after a match or no match
619     when JIT-compiled code was actually used.
620     </P>
621     <P>
622     Note that there is also an independent <b>/+</b> modifier; it must not be given
623 ph10 975 immediately after <b>/S</b> or <b>/S+</b> because this will be misinterpreted.
624 ph10 930 </P>
625     <P>
626     If JIT studying is successful, the compiled JIT code will automatically be used
627 ph10 1194 when <b>pcre[16|32]_exec()</b> is run, except when incompatible run-time options
628 ph10 930 are specified. For more details, see the
629 ph10 691 <a href="pcrejit.html"><b>pcrejit</b></a>
630     documentation. See also the <b>\J</b> escape sequence below for a way of
631     setting the size of the JIT stack.
632     </P>
633     <P>
634 ph10 1194 Finally, if <b>/S</b> is followed by a minus character, JIT compilation is
635     suppressed, even if it was requested externally by the <b>-s</b> command line
636     option. This makes it possible to specify that JIT is never to be used for
637     certain patterns.
638     </P>
639     <P>
640 ph10 545 The <b>/T</b> modifier must be followed by a single digit. It causes a specific
641 ph10 1194 set of built-in character tables to be passed to <b>pcre[16|32]_compile()</b>. It
642 ph10 869 is used in the standard PCRE tests to check behaviour with different character
643 ph10 545 tables. The digit specifies the tables as follows:
644     <pre>
645     0 the default ASCII tables, as distributed in
646     pcre_chartables.c.dist
647     1 a set of tables defining ISO 8859 characters
648     </pre>
649     In table 1, some characters whose codes are greater than 128 are identified as
650     letters, digits, spaces, etc.
651     </P>
652 ph10 535 <br><b>
653     Using the POSIX wrapper API
654     </b><br>
655     <P>
656     The <b>/P</b> modifier causes <b>pcretest</b> to call PCRE via the POSIX wrapper
657 ph10 869 API rather than its native API. This supports only the 8-bit library. When
658     <b>/P</b> is set, the following modifiers set options for the <b>regcomp()</b>
659     function:
660 ph10 535 <pre>
661     /i REG_ICASE
662     /m REG_NEWLINE
663     /N REG_NOSUB
664     /s REG_DOTALL )
665     /U REG_UNGREEDY ) These options are not part of
666     /W REG_UCP ) the POSIX standard
667     /8 REG_UTF8 )
668     </pre>
669     The <b>/+</b> modifier works as described above. All other modifiers are
670     ignored.
671     </P>
672 ph10 1404 <br><b>
673     Locking out certain modifiers
674     </b><br>
675     <P>
676     PCRE can be compiled with or without support for certain features such as
677     UTF-8/16/32 or Unicode properties. Accordingly, the standard tests are split up
678     into a number of different files that are selected for running depending on
679     which features are available. When updating the tests, it is all too easy to
680     put a new test into the wrong file by mistake; for example, to put a test that
681     requires UTF support into a file that is used when it is not available. To help
682     detect such mistakes as early as possible, there is a facility for locking out
683     specific modifiers. If an input line for <b>pcretest</b> starts with the string
684     "&#60; forbid " the following sequence of characters is taken as a list of
685     forbidden modifiers. For example, in the test files that must not use UTF or
686     Unicode property support, this line appears:
687     <pre>
688     &#60; forbid 8W
689     </pre>
690     This locks out the /8 and /W modifiers. An immediate error is given if they are
691     subsequently encountered. If the character string contains &#60; but not &#62;, all the
692     multi-character modifiers that begin with &#60; are locked out. Otherwise, such
693     modifiers must be explicitly listed, for example:
694     <pre>
695     &#60; forbid &#60;JS&#62;&#60;cr&#62;
696     </pre>
697     There must be a single space between &#60; and "forbid" for this feature to be
698     recognised. If there is not, the line is interpreted either as a request to
699     re-load a pre-compiled pattern (see "SAVING AND RELOADING COMPILED PATTERNS"
700     below) or, if there is a another &#60; character, as a pattern that uses &#60; as its
701     delimiter.
702     </P>
703 ph10 1320 <br><a name="SEC7" href="#TOC1">DATA LINES</a><br>
704 nigel 63 <P>
705 ph10 1194 Before each data line is passed to <b>pcre[16|32]_exec()</b>, leading and trailing
706 ph10 654 white space is removed, and it is then scanned for \ escapes. Some of these
707     are pretty esoteric features, intended for checking out some of the more
708 nigel 63 complicated features of PCRE. If you are just testing "ordinary" regular
709     expressions, you probably don't need any of these. The following escapes are
710     recognized:
711     <pre>
712 nigel 93 \a alarm (BEL, \x07)
713     \b backspace (\x08)
714     \e escape (\x27)
715 ph10 654 \f form feed (\x0c)
716 nigel 93 \n newline (\x0a)
717 nigel 91 \qdd set the PCRE_MATCH_LIMIT limit to dd (any number of digits)
718 nigel 93 \r carriage return (\x0d)
719     \t tab (\x09)
720     \v vertical tab (\x0b)
721 ph10 869 \nnn octal character (up to 3 octal digits); always
722 ph10 1194 a byte unless &#62; 255 in UTF-8 or 16-bit or 32-bit mode
723 ph10 1404 \o{dd...} octal character (any number of octal digits}
724 ph10 572 \xhh hexadecimal byte (up to 2 hex digits)
725 ph10 869 \x{hh...} hexadecimal character (any number of hex digits)
726 ph10 1194 \A pass the PCRE_ANCHORED option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
727     \B pass the PCRE_NOTBOL option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
728     \Cdd call pcre[16|32]_copy_substring() for substring dd after a successful match (number less than 32)
729     \Cname call pcre[16|32]_copy_named_substring() for substring "name" after a successful match (name termin-
730 nigel 63 ated by next non alphanumeric character)
731 nigel 75 \C+ show the current captured substrings at callout time
732 nigel 63 \C- do not supply a callout function
733 nigel 75 \C!n return 1 instead of 0 when callout number n is reached
734     \C!n!m return 1 instead of 0 when callout number n is reached for the nth time
735     \C*n pass the number n (may be negative) as callout data; this is used as the callout return value
736 ph10 1194 \D use the <b>pcre[16|32]_dfa_exec()</b> match function
737     \F only shortest match for <b>pcre[16|32]_dfa_exec()</b>
738     \Gdd call pcre[16|32]_get_substring() for substring dd after a successful match (number less than 32)
739     \Gname call pcre[16|32]_get_named_substring() for substring "name" after a successful match (name termin-
740 nigel 63 ated by next non-alphanumeric character)
741 ph10 691 \Jdd set up a JIT stack of dd kilobytes maximum (any number of digits)
742 ph10 1194 \L call pcre[16|32]_get_substringlist() after a successful match
743 nigel 91 \M discover the minimum MATCH_LIMIT and MATCH_LIMIT_RECURSION settings
744 ph10 1194 \N pass the PCRE_NOTEMPTY option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>; if used twice, pass the
745 ph10 461 PCRE_NOTEMPTY_ATSTART option
746 ph10 1194 \Odd set the size of the output vector passed to <b>pcre[16|32]_exec()</b> to dd (any number of digits)
747     \P pass the PCRE_PARTIAL_SOFT option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>; if used twice, pass the
748 ph10 461 PCRE_PARTIAL_HARD option
749 nigel 91 \Qdd set the PCRE_MATCH_LIMIT_RECURSION limit to dd (any number of digits)
750 ph10 1194 \R pass the PCRE_DFA_RESTART option to <b>pcre[16|32]_dfa_exec()</b>
751 nigel 73 \S output details of memory get/free calls during matching
752 ph10 1194 \Y pass the PCRE_NO_START_OPTIMIZE option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
753     \Z pass the PCRE_NOTEOL option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
754     \? pass the PCRE_NO_UTF[8|16|32]_CHECK option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
755 ph10 579 \&#62;dd start the match at offset dd (optional "-"; then any number of digits); this sets the <i>startoffset</i>
756 ph10 1194 argument for <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
757     \&#60;cr&#62; pass the PCRE_NEWLINE_CR option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
758     \&#60;lf&#62; pass the PCRE_NEWLINE_LF option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
759     \&#60;crlf&#62; pass the PCRE_NEWLINE_CRLF option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
760     \&#60;anycrlf&#62; pass the PCRE_NEWLINE_ANYCRLF option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
761     \&#60;any&#62; pass the PCRE_NEWLINE_ANY option to <b>pcre[16|32]_exec()</b> or <b>pcre[16|32]_dfa_exec()</b>
762 nigel 75 </pre>
763 ph10 869 The use of \x{hh...} is not dependent on the use of the <b>/8</b> modifier on
764     the pattern. It is recognized always. There may be any number of hexadecimal
765     digits inside the braces; invalid values provoke error messages.
766 ph10 572 </P>
767     <P>
768 ph10 878 Note that \xhh specifies one byte rather than one character in UTF-8 mode;
769     this makes it possible to construct invalid UTF-8 sequences for testing
770     purposes. On the other hand, \x{hh} is interpreted as a UTF-8 character in
771     UTF-8 mode, generating more than one byte if the value is greater than 127.
772     When testing the 8-bit library not in UTF-8 mode, \x{hh} generates one byte
773     for values less than 256, and causes an error for greater values.
774 ph10 869 </P>
775     <P>
776     In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it
777     possible to construct invalid UTF-16 sequences for testing purposes.
778     </P>
779     <P>
780 ph10 1194 In UTF-32 mode, all 4- to 8-digit \x{...} values are accepted. This makes it
781     possible to construct invalid UTF-32 sequences for testing purposes.
782     </P>
783     <P>
784 nigel 93 The escapes that specify line ending sequences are literal strings, exactly as
785     shown. No more than one newline setting should be present in any data line.
786 nigel 63 </P>
787     <P>
788 nigel 93 A backslash followed by anything else just escapes the anything else. If
789     the very last character is a backslash, it is ignored. This gives a way of
790     passing an empty line as data, since a real empty line terminates the data
791     input.
792     </P>
793     <P>
794 ph10 691 The <b>\J</b> escape provides a way of setting the maximum stack size that is
795     used by the just-in-time optimization code. It is ignored if JIT optimization
796     is not being used. Providing a stack that is larger than the default 32K is
797     necessary only for very complicated patterns.
798     </P>
799     <P>
800 ph10 1194 If \M is present, <b>pcretest</b> calls <b>pcre[16|32]_exec()</b> several times,
801 ph10 869 with different values in the <i>match_limit</i> and <i>match_limit_recursion</i>
802 ph10 1194 fields of the <b>pcre[16|32]_extra</b> data structure, until it finds the minimum
803     numbers for each parameter that allow <b>pcre[16|32]_exec()</b> to complete without
804 ph10 691 error. Because this is testing a specific feature of the normal interpretive
805 ph10 1194 <b>pcre[16|32]_exec()</b> execution, the use of any JIT optimization that might
806 ph10 869 have been set up by the <b>/S+</b> qualifier of <b>-s+</b> option is disabled.
807 nigel 63 </P>
808     <P>
809 ph10 691 The <i>match_limit</i> number is a measure of the amount of backtracking
810     that takes place, and checking it out can be instructive. For most simple
811     matches, the number is quite small, but for patterns with very large numbers of
812     matching possibilities, it can become large very quickly with increasing length
813     of subject string. The <i>match_limit_recursion</i> number is a measure of how
814     much stack (or, if PCRE is compiled with NO_RECURSE, how much heap) memory is
815     needed to complete the match attempt.
816     </P>
817     <P>
818 nigel 75 When \O is used, the value specified may be higher or lower than the size set
819     by the <b>-O</b> command line option (or defaulted to 45); \O applies only to
820 ph10 1194 the call of <b>pcre[16|32]_exec()</b> for the line in which it appears.
821 nigel 63 </P>
822     <P>
823 nigel 75 If the <b>/P</b> modifier was present on the pattern, causing the POSIX wrapper
824 ph10 535 API to be used, the only option-setting sequences that have any effect are \B,
825     \N, and \Z, causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively,
826     to be passed to <b>regexec()</b>.
827 nigel 63 </P>
828 ph10 1320 <br><a name="SEC8" href="#TOC1">THE ALTERNATIVE MATCHING FUNCTION</a><br>
829 nigel 63 <P>
830 nigel 77 By default, <b>pcretest</b> uses the standard PCRE matching function,
831 ph10 1194 <b>pcre[16|32]_exec()</b> to match each data line. PCRE also supports an
832     alternative matching function, <b>pcre[16|32]_dfa_test()</b>, which operates in a
833 nigel 77 different way, and has some restrictions. The differences between the two
834     functions are described in the
835     <a href="pcrematching.html"><b>pcrematching</b></a>
836     documentation.
837     </P>
838     <P>
839     If a data line contains the \D escape sequence, or if the command line
840 ph10 869 contains the <b>-dfa</b> option, the alternative matching function is used.
841 nigel 77 This function finds all possible matches at a given point. If, however, the \F
842     escape sequence is present in the data line, it stops after the first match is
843     found. This is always the shortest possible match.
844     </P>
845 ph10 1320 <br><a name="SEC9" href="#TOC1">DEFAULT OUTPUT FROM PCRETEST</a><br>
846 nigel 77 <P>
847     This section describes the output when the normal matching function,
848 ph10 1194 <b>pcre[16|32]_exec()</b>, is being used.
849 nigel 77 </P>
850     <P>
851 ph10 654 When a match succeeds, <b>pcretest</b> outputs the list of captured substrings
852 ph10 1194 that <b>pcre[16|32]_exec()</b> returns, starting with number 0 for the string that
853 ph10 654 matched the whole pattern. Otherwise, it outputs "No match" when the return is
854 ph10 453 PCRE_ERROR_NOMATCH, and "Partial match:" followed by the partially matching
855 ph10 1194 substring when <b>pcre[16|32]_exec()</b> returns PCRE_ERROR_PARTIAL. (Note that
856 ph10 869 this is the entire substring that was inspected during the partial match; it
857     may include characters before the actual match start if a lookbehind assertion,
858 ph10 654 \K, \b, or \B was involved.) For any other return, <b>pcretest</b> outputs
859     the PCRE negative error number and a short descriptive phrase. If the error is
860 ph10 869 a failed UTF string check, the offset of the start of the failing character and
861     the reason code are also output, provided that the size of the output vector is
862     at least two. Here is an example of an interactive <b>pcretest</b> run.
863 nigel 63 <pre>
864     $ pcretest
865 ph10 654 PCRE version 8.13 2011-04-30
866 nigel 75
867 nigel 63 re&#62; /^abc(\d+)/
868     data&#62; abc123
869     0: abc123
870     1: 123
871     data&#62; xyz
872     No match
873 nigel 75 </pre>
874 ph10 654 Unset capturing substrings that are not followed by one that is set are not
875 ph10 1194 returned by <b>pcre[16|32]_exec()</b>, and are not shown by <b>pcretest</b>. In the
876 ph10 654 following example, there are two capturing substrings, but when the first data
877     line is matched, the second, unset substring is not shown. An "internal" unset
878     substring is shown as "&#60;unset&#62;", as for the second data line.
879 ph10 286 <pre>
880     re&#62; /(a)|(b)/
881     data&#62; a
882     0: a
883     1: a
884     data&#62; b
885     0: b
886     1: &#60;unset&#62;
887     2: b
888     </pre>
889 ph10 869 If the strings contain any non-printing characters, they are output as \xhh
890 ph10 903 escapes if the value is less than 256 and UTF mode is not set. Otherwise they
891 ph10 869 are output as \x{hh...} escapes. See below for the definition of non-printing
892     characters. If the pattern has the <b>/+</b> modifier, the output for substring
893     0 is followed by the the rest of the subject string, identified by "0+" like
894     this:
895 nigel 63 <pre>
896     re&#62; /cat/+
897     data&#62; cataract
898     0: cat
899     0+ aract
900 nigel 75 </pre>
901 nigel 63 If the pattern has the <b>/g</b> or <b>/G</b> modifier, the results of successive
902     matching attempts are output in sequence, like this:
903     <pre>
904     re&#62; /\Bi(\w\w)/g
905     data&#62; Mississippi
906     0: iss
907     1: ss
908     0: iss
909     1: ss
910     0: ipp
911     1: pp
912 nigel 75 </pre>
913 ph10 654 "No match" is output only if the first match attempt fails. Here is an example
914     of a failure message (the offset 4 that is specified by \&#62;4 is past the end of
915     the subject string):
916     <pre>
917     re&#62; /xyz/
918     data&#62; xyz\&#62;4
919     Error -24 (bad offset value)
920     </PRE>
921 nigel 63 </P>
922     <P>
923     If any of the sequences <b>\C</b>, <b>\G</b>, or <b>\L</b> are present in a
924     data line that is successfully matched, the substrings extracted by the
925     convenience functions are output with C, G, or L after the string number
926     instead of a colon. This is in addition to the normal full list. The string
927     length (that is, the return from the extraction function) is given in
928     parentheses after each string for <b>\C</b> and <b>\G</b>.
929     </P>
930     <P>
931 nigel 93 Note that whereas patterns can be continued over several lines (a plain "&#62;"
932 nigel 63 prompt is used for continuations), data lines may not. However newlines can be
933 nigel 93 included in data by means of the \n escape (or \r, \r\n, etc., depending on
934     the newline sequence setting).
935 nigel 63 </P>
936 ph10 1320 <br><a name="SEC10" href="#TOC1">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a><br>
937 nigel 63 <P>
938 ph10 1194 When the alternative matching function, <b>pcre[16|32]_dfa_exec()</b>, is used (by
939 nigel 77 means of the \D escape sequence or the <b>-dfa</b> command line option), the
940     output consists of a list of all the matches that start at the first point in
941     the subject where there is at least one match. For example:
942     <pre>
943     re&#62; /(tang|tangerine|tan)/
944     data&#62; yellow tangerine\D
945     0: tangerine
946     1: tang
947     2: tan
948     </pre>
949     (Using the normal matching function on this data finds only "tang".) The
950 ph10 429 longest matching string is always given first (and numbered zero). After a
951 ph10 461 PCRE_ERROR_PARTIAL return, the output is "Partial match:", followed by the
952 ph10 567 partially matching substring. (Note that this is the entire substring that was
953     inspected during the partial match; it may include characters before the actual
954     match start if a lookbehind assertion, \K, \b, or \B was involved.)
955 nigel 77 </P>
956     <P>
957 nigel 93 If <b>/g</b> is present on the pattern, the search for further matches resumes
958 nigel 77 at the end of the longest match. For example:
959     <pre>
960     re&#62; /(tang|tangerine|tan)/g
961     data&#62; yellow tangerine and tangy sultana\D
962     0: tangerine
963     1: tang
964     2: tan
965     0: tang
966     1: tan
967     0: tan
968     </pre>
969     Since the matching function does not support substring capture, the escape
970     sequences that are concerned with captured substrings are not relevant.
971     </P>
972 ph10 1320 <br><a name="SEC11" href="#TOC1">RESTARTING AFTER A PARTIAL MATCH</a><br>
973 nigel 77 <P>
974     When the alternative matching function has given the PCRE_ERROR_PARTIAL return,
975     indicating that the subject partially matched the pattern, you can restart the
976     match with additional subject data by means of the \R escape sequence. For
977     example:
978     <pre>
979     re&#62; /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
980     data&#62; 23ja\P\D
981     Partial match: 23ja
982     data&#62; n05\R\D
983     0: n05
984     </pre>
985     For further information about partial matching, see the
986     <a href="pcrepartial.html"><b>pcrepartial</b></a>
987     documentation.
988     </P>
989 ph10 1320 <br><a name="SEC12" href="#TOC1">CALLOUTS</a><br>
990 nigel 77 <P>
991 nigel 75 If the pattern contains any callout requests, <b>pcretest</b>'s callout function
992 nigel 77 is called during matching. This works with both matching functions. By default,
993     the called function displays the callout number, the start and current
994     positions in the text at the callout time, and the next pattern item to be
995 ph10 869 tested. For example:
996 nigel 75 <pre>
997     ---&#62;pqrabcdef
998     0 ^ ^ \d
999     </pre>
1000 ph10 869 This output indicates that callout number 0 occurred for a match attempt
1001     starting at the fourth character of the subject string, when the pointer was at
1002     the seventh character of the data, and when the next pattern item was \d. Just
1003     one circumflex is output if the start and current positions are the same.
1004 nigel 75 </P>
1005     <P>
1006     Callouts numbered 255 are assumed to be automatic callouts, inserted as a
1007     result of the <b>/C</b> pattern modifier. In this case, instead of showing the
1008     callout number, the offset in the pattern, preceded by a plus, is output. For
1009     example:
1010     <pre>
1011     re&#62; /\d?[A-E]\*/C
1012     data&#62; E*
1013     ---&#62;E*
1014     +0 ^ \d?
1015     +3 ^ [A-E]
1016     +8 ^^ \*
1017     +10 ^ ^
1018     0: E*
1019     </pre>
1020 ph10 654 If a pattern contains (*MARK) items, an additional line is output whenever
1021     a change of latest mark is passed to the callout function. For example:
1022     <pre>
1023     re&#62; /a(*MARK:X)bc/C
1024     data&#62; abc
1025     ---&#62;abc
1026     +0 ^ a
1027     +1 ^^ (*MARK:X)
1028     +10 ^^ b
1029     Latest Mark: X
1030     +11 ^ ^ c
1031     +12 ^ ^
1032     0: abc
1033     </pre>
1034     The mark changes between matching "a" and "b", but stays the same for the rest
1035     of the match, so nothing more is output. If, as a result of backtracking, the
1036     mark reverts to being unset, the text "&#60;unset&#62;" is output.
1037     </P>
1038     <P>
1039 nigel 75 The callout function in <b>pcretest</b> returns zero (carry on matching) by
1040 nigel 77 default, but you can use a \C item in a data line (as described above) to
1041 ph10 654 change this and other parameters of the callout.
1042 nigel 75 </P>
1043     <P>
1044     Inserting callouts can be helpful when using <b>pcretest</b> to check
1045     complicated regular expressions. For further information about callouts, see
1046     the
1047     <a href="pcrecallout.html"><b>pcrecallout</b></a>
1048     documentation.
1049     </P>
1050 ph10 1320 <br><a name="SEC13" href="#TOC1">NON-PRINTING CHARACTERS</a><br>
1051 nigel 75 <P>
1052 nigel 93 When <b>pcretest</b> is outputting text in the compiled version of a pattern,
1053     bytes other than 32-126 are always treated as non-printing characters are are
1054     therefore shown as hex escapes.
1055     </P>
1056     <P>
1057     When <b>pcretest</b> is outputting text that is a matched part of a subject
1058     string, it behaves in the same way, unless a different locale has been set for
1059     the pattern (using the <b>/L</b> modifier). In this case, the <b>isprint()</b>
1060     function to distinguish printing and non-printing characters.
1061     </P>
1062 ph10 1320 <br><a name="SEC14" href="#TOC1">SAVING AND RELOADING COMPILED PATTERNS</a><br>
1063 nigel 93 <P>
1064 nigel 75 The facilities described in this section are not available when the POSIX
1065 ph10 654 interface to PCRE is being used, that is, when the <b>/P</b> pattern modifier is
1066 nigel 75 specified.
1067     </P>
1068     <P>
1069     When the POSIX interface is not in use, you can cause <b>pcretest</b> to write a
1070     compiled pattern to a file, by following the modifiers with &#62; and a file name.
1071     For example:
1072     <pre>
1073     /pattern/im &#62;/some/file
1074     </pre>
1075     See the
1076     <a href="pcreprecompile.html"><b>pcreprecompile</b></a>
1077     documentation for a discussion about saving and re-using compiled patterns.
1078 ph10 691 Note that if the pattern was successfully studied with JIT optimization, the
1079     JIT data cannot be saved.
1080 nigel 75 </P>
1081     <P>
1082     The data that is written is binary. The first eight bytes are the length of the
1083     compiled pattern data followed by the length of the optional study data, each
1084     written as four bytes in big-endian order (most significant byte first). If
1085     there is no study data (either the pattern was not studied, or studying did not
1086     return any data), the second length is zero. The lengths are followed by an
1087     exact copy of the compiled pattern. If there is additional study data, this
1088 ph10 691 (excluding any JIT data) follows immediately after the compiled pattern. After
1089     writing the file, <b>pcretest</b> expects to read a new pattern.
1090 nigel 75 </P>
1091     <P>
1092 ph10 654 A saved pattern can be reloaded into <b>pcretest</b> by specifying &#60; and a file
1093 ph10 1404 name instead of a pattern. There must be no space between &#60; and the file name,
1094     which must not contain a &#60; character, as otherwise <b>pcretest</b> will
1095     interpret the line as a pattern delimited by &#60; characters. For example:
1096 nigel 75 <pre>
1097     re&#62; &#60;/some/file
1098 ph10 654 Compiled pattern loaded from /some/file
1099 nigel 75 No study data
1100     </pre>
1101 ph10 691 If the pattern was previously studied with the JIT optimization, the JIT
1102     information cannot be saved and restored, and so is lost. When the pattern has
1103     been loaded, <b>pcretest</b> proceeds to read data lines in the usual way.
1104 nigel 75 </P>
1105     <P>
1106     You can copy a file written by <b>pcretest</b> to a different host and reload it
1107     there, even if the new host has opposite endianness to the one on which the
1108     pattern was compiled. For example, you can compile on an i86 machine and run on
1109 ph10 903 a SPARC machine. When a pattern is reloaded on a host with different
1110 ph10 869 endianness, the confirmation message is changed to:
1111     <pre>
1112     Compiled pattern (byte-inverted) loaded from /some/file
1113     </pre>
1114 ph10 903 The test suite contains some saved pre-compiled patterns with different
1115     endianness. These are reloaded using "&#60;!" instead of just "&#60;". This suppresses
1116     the "(byte-inverted)" text so that the output is the same on all hosts. It also
1117     forces debugging output once the pattern has been reloaded.
1118 nigel 75 </P>
1119     <P>
1120     File names for saving and reloading can be absolute or relative, but note that
1121     the shell facility of expanding a file name that starts with a tilde (~) is not
1122     available.
1123     </P>
1124     <P>
1125     The ability to save and reload files in <b>pcretest</b> is intended for testing
1126     and experimentation. It is not intended for production use because only a
1127     single pattern can be written to a file. Furthermore, there is no facility for
1128     supplying custom character tables for use with a reloaded pattern. If the
1129     original pattern was compiled with custom tables, an attempt to match a subject
1130     string using a reloaded pattern is likely to cause <b>pcretest</b> to crash.
1131     Finally, if you attempt to load a file that is not in the correct format, the
1132     result is undefined.
1133     </P>
1134 ph10 1320 <br><a name="SEC15" href="#TOC1">SEE ALSO</a><br>
1135 nigel 75 <P>
1136 ph10 1194 <b>pcre</b>(3), <b>pcre16</b>(3), <b>pcre32</b>(3), <b>pcreapi</b>(3),
1137     <b>pcrecallout</b>(3),
1138 ph10 869 <b>pcrejit</b>, <b>pcrematching</b>(3), <b>pcrepartial</b>(d),
1139     <b>pcrepattern</b>(3), <b>pcreprecompile</b>(3).
1140 nigel 93 </P>
1141 ph10 1320 <br><a name="SEC16" href="#TOC1">AUTHOR</a><br>
1142 nigel 93 <P>
1143 nigel 77 Philip Hazel
1144 nigel 63 <br>
1145 ph10 99 University Computing Service
1146 nigel 63 <br>
1147 nigel 93 Cambridge CB2 3QH, England.
1148 ph10 99 <br>
1149 nigel 63 </P>
1150 ph10 1320 <br><a name="SEC17" href="#TOC1">REVISION</a><br>
1151 nigel 63 <P>
1152 ph10 1459 Last updated: 09 February 2014
1153 nigel 63 <br>
1154 ph10 1459 Copyright &copy; 1997-2014 University of Cambridge.
1155 ph10 99 <br>
1156 nigel 75 <p>
1157     Return to the <a href="index.html">PCRE index page</a>.
1158     </p>


Name Value
svn:eol-style native
svn:keywords "Author Date Id Revision Url"

ViewVC Help
Powered by ViewVC 1.1.12