/[pcre]/code/tags/pcre-4.5/README
ViewVC logotype

Contents of /code/tags/pcre-4.5/README

Parent Directory Parent Directory | Revision Log Revision Log


Revision 74 - (show annotations) (download)
Sat Feb 24 21:40:32 2007 UTC (7 years, 1 month ago) by nigel
File size: 16781 byte(s)
Tag code/trunk as code/tags/pcre-4.5.

1 README file for PCRE (Perl-compatible regular expression library)
2 -----------------------------------------------------------------
3
4 The latest release of PCRE is always available from
5
6 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
7
8 Please read the NEWS file if you are upgrading from a previous release.
9
10 PCRE has its own native API, but a set of "wrapper" functions that are based on
11 the POSIX API are also supplied in the library libpcreposix. Note that this
12 just provides a POSIX calling interface to PCRE: the regular expressions
13 themselves still follow Perl syntax and semantics. The header file
14 for the POSIX-style functions is called pcreposix.h. The official POSIX name is
15 regex.h, but I didn't want to risk possible problems with existing files of
16 that name by distributing it that way. To use it with an existing program that
17 uses the POSIX API, it will have to be renamed or pointed at by a link.
18
19 If you are using the POSIX interface to PCRE and there is already a POSIX regex
20 library installed on your system, you must take care when linking programs to
21 ensure that they link with PCRE's libpcreposix library. Otherwise they may pick
22 up the "real" POSIX functions of the same name.
23
24
25 Contributions by users of PCRE
26 ------------------------------
27
28 You can find contributions from PCRE users in the directory
29
30 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
31
32 where there is also a README file giving brief descriptions of what they are.
33 Several of them provide support for compiling PCRE on various flavours of
34 Windows systems (I myself do not use Windows). Some are complete in themselves;
35 others are pointers to URLs containing relevant files.
36
37
38 Building PCRE on a Unix-like system
39 -----------------------------------
40
41 To build PCRE on a Unix-like system, first run the "configure" command from the
42 PCRE distribution directory, with your current directory set to the directory
43 where you want the files to be created. This command is a standard GNU
44 "autoconf" configuration script, for which generic instructions are supplied in
45 INSTALL.
46
47 Most commonly, people build PCRE within its own distribution directory, and in
48 this case, on many systems, just running "./configure" is sufficient, but the
49 usual methods of changing standard defaults are available. For example,
50
51 CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
52
53 specifies that the C compiler should be run with the flags '-O2 -Wall' instead
54 of the default, and that "make install" should install PCRE under /opt/local
55 instead of the default /usr/local.
56
57 If you want to build in a different directory, just run "configure" with that
58 directory as current. For example, suppose you have unpacked the PCRE source
59 into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
60
61 cd /build/pcre/pcre-xxx
62 /source/pcre/pcre-xxx/configure
63
64 There are some optional features that can be included or omitted from the PCRE
65 library. You can read more about them in the pcrebuild man page.
66
67 . If you want to make use of the support for UTF-8 character strings in PCRE,
68 you must add --enable-utf8 to the "configure" command. Without it, the code
69 for handling UTF-8 is not included in the library. (Even when included, it
70 still has to be enabled by an option at run time.)
71
72 . You can build PCRE to recognized CR or NL as the newline character, instead
73 of whatever your compiler uses for "\n", by adding --newline-is-cr or
74 --newline-is-nl to the "configure" command, respectively. Only do this if you
75 really understand what you are doing. On traditional Unix-like systems, the
76 newline character is NL.
77
78 . When called via the POSIX interface, PCRE uses malloc() to get additional
79 storage for processing capturing parentheses if there are more than 10 of
80 them. You can increase this threshold by setting, for example,
81
82 --with-posix-malloc-threshold=20
83
84 on the "configure" command.
85
86 . PCRE has a counter which can be set to limit the amount of resources it uses.
87 If the limit is exceeded during a match, the match fails. The default is ten
88 million. You can change the default by setting, for example,
89
90 --with-match-limit=500000
91
92 on the "configure" command. This is just the default; individual calls to
93 pcre_exec() can supply their own value. There is discussion on the pcreapi
94 man page.
95
96 . The default maximum compiled pattern size is around 64K. You can increase
97 this by adding --with-link-size=3 to the "configure" command. You can
98 increase it even more by setting --with-link-size=4, but this is unlikely
99 ever to be necessary. If you build PCRE with an increased link size, test 2
100 (and 5 if you are using UTF-8) will fail. Part of the output of these tests
101 is a representation of the compiled pattern, and this changes with the link
102 size.
103
104 . You can build PCRE so that its match() function does not call itself
105 recursively. Instead, it uses blocks of data from the heap via special
106 functions pcre_stack_malloc() and pcre_stack_free() to save data that would
107 otherwise be saved on the stack. To build PCRE like this, use
108
109 --disable-stack-for-recursion
110
111 on the "configure" command. PCRE runs more slowly in this mode, but it may be
112 necessary in environments with limited stack sizes.
113
114 The "configure" script builds five files:
115
116 . libtool is a script that builds shared and/or static libraries
117 . Makefile is built by copying Makefile.in and making substitutions.
118 . config.h is built by copying config.in and making substitutions.
119 . pcre-config is built by copying pcre-config.in and making substitutions.
120 . RunTest is a script for running tests
121
122 Once "configure" has run, you can run "make". It builds two libraries called
123 libpcre and libpcreposix, a test program called pcretest, and the pcregrep
124 command. You can use "make install" to copy these, the public header files
125 pcre.h and pcreposix.h, and the man pages to appropriate live directories on
126 your system, in the normal way.
127
128 Running "make install" also installs the command pcre-config, which can be used
129 to recall information about the PCRE configuration and installation. For
130 example,
131
132 pcre-config --version
133
134 prints the version number, and
135
136 pcre-config --libs
137
138 outputs information about where the library is installed. This command can be
139 included in makefiles for programs that use PCRE, saving the programmer from
140 having to remember too many details.
141
142
143 Shared libraries on Unix-like systems
144 -------------------------------------
145
146 The default distribution builds PCRE as two shared libraries and two static
147 libraries, as long as the operating system supports shared libraries. Shared
148 library support relies on the "libtool" script which is built as part of the
149 "configure" process.
150
151 The libtool script is used to compile and link both shared and static
152 libraries. They are placed in a subdirectory called .libs when they are newly
153 built. The programs pcretest and pcregrep are built to use these uninstalled
154 libraries (by means of wrapper scripts in the case of shared libraries). When
155 you use "make install" to install shared libraries, pcregrep and pcretest are
156 automatically re-built to use the newly installed shared libraries before being
157 installed themselves. However, the versions left in the source directory still
158 use the uninstalled libraries.
159
160 To build PCRE using static libraries only you must use --disable-shared when
161 configuring it. For example
162
163 ./configure --prefix=/usr/gnu --disable-shared
164
165 Then run "make" in the usual way. Similarly, you can use --disable-static to
166 build only shared libraries.
167
168
169 Cross-compiling on a Unix-like system
170 -------------------------------------
171
172 You can specify CC and CFLAGS in the normal way to the "configure" command, in
173 order to cross-compile PCRE for some other host. However, during the building
174 process, the dftables.c source file is compiled *and run* on the local host, in
175 order to generate the default character tables (the chartables.c file). It
176 therefore needs to be compiled with the local compiler, not the cross compiler.
177 You can do this by specifying CC_FOR_BUILD (and if necessary CFLAGS_FOR_BUILD)
178 when calling the "configure" command. If they are not specified, they default
179 to the values of CC and CFLAGS.
180
181
182 Building on non-Unix systems
183 ----------------------------
184
185 For a non-Unix system, read the comments in the file NON-UNIX-USE, though if
186 the system supports the use of "configure" and "make" you may be able to build
187 PCRE in the same way as for Unix systems.
188
189 PCRE has been compiled on Windows systems and on Macintoshes, but I don't know
190 the details because I don't use those systems. It should be straightforward to
191 build PCRE on any system that has a Standard C compiler, because it uses only
192 Standard C functions.
193
194
195 Testing PCRE
196 ------------
197
198 To test PCRE on a Unix system, run the RunTest script that is created by the
199 configuring process. (This can also be run by "make runtest", "make check", or
200 "make test".) For other systems, see the instructions in NON-UNIX-USE.
201
202 The script runs the pcretest test program (which is documented in its own man
203 page) on each of the testinput files (in the testdata directory) in turn,
204 and compares the output with the contents of the corresponding testoutput file.
205 A file called testtry is used to hold the output from pcretest. To run pcretest
206 on just one of the test files, give its number as an argument to RunTest, for
207 example:
208
209 RunTest 2
210
211 The first file can also be fed directly into the perltest script to check that
212 Perl gives the same results. The only difference you should see is in the first
213 few lines, where the Perl version is given instead of the PCRE version.
214
215 The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
216 pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
217 detection, and run-time flags that are specific to PCRE, as well as the POSIX
218 wrapper API. It also uses the debugging flag to check some of the internals of
219 pcre_compile().
220
221 If you build PCRE with a locale setting that is not the standard C locale, the
222 character tables may be different (see next paragraph). In some cases, this may
223 cause failures in the second set of tests. For example, in a locale where the
224 isprint() function yields TRUE for characters in the range 128-255, the use of
225 [:isascii:] inside a character class defines a different set of characters, and
226 this shows up in this test as a difference in the compiled code, which is being
227 listed for checking. Where the comparison test output contains [\x00-\x7f] the
228 test will contain [\x00-\xff], and similarly in some other cases. This is not a
229 bug in PCRE.
230
231 The third set of tests checks pcre_maketables(), the facility for building a
232 set of character tables for a specific locale and using them instead of the
233 default tables. The tests make use of the "fr_FR" (French) locale. Before
234 running the test, the script checks for the presence of this locale by running
235 the "locale" command. If that command fails, or if it doesn't include "fr_FR"
236 in the list of available locales, the third test cannot be run, and a comment
237 is output to say why. If running this test produces instances of the error
238
239 ** Failed to set locale "fr_FR"
240
241 in the comparison output, it means that locale is not available on your system,
242 despite being listed by "locale". This does not mean that PCRE is broken.
243
244 The fourth test checks the UTF-8 support. It is not run automatically unless
245 PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
246 running "configure". This file can be also fed directly to the perltest script,
247 provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
248 commented in the script, can be be used.)
249
250 The fifth and final file tests error handling with UTF-8 encoding, and internal
251 UTF-8 features of PCRE that are not relevant to Perl.
252
253
254 Character tables
255 ----------------
256
257 PCRE uses four tables for manipulating and identifying characters. The final
258 argument of the pcre_compile() function is a pointer to a block of memory
259 containing the concatenated tables. A call to pcre_maketables() can be used to
260 generate a set of tables in the current locale. If the final argument for
261 pcre_compile() is passed as NULL, a set of default tables that is built into
262 the binary is used.
263
264 The source file called chartables.c contains the default set of tables. This is
265 not supplied in the distribution, but is built by the program dftables
266 (compiled from dftables.c), which uses the ANSI C character handling functions
267 such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table
268 sources. This means that the default C locale which is set for your system will
269 control the contents of these default tables. You can change the default tables
270 by editing chartables.c and then re-building PCRE. If you do this, you should
271 probably also edit Makefile to ensure that the file doesn't ever get
272 re-generated.
273
274 The first two 256-byte tables provide lower casing and case flipping functions,
275 respectively. The next table consists of three 32-byte bit maps which identify
276 digits, "word" characters, and white space, respectively. These are used when
277 building 32-byte bit maps that represent character classes.
278
279 The final 256-byte table has bits indicating various character types, as
280 follows:
281
282 1 white space character
283 2 letter
284 4 decimal digit
285 8 hexadecimal digit
286 16 alphanumeric or '_'
287 128 regular expression metacharacter or binary zero
288
289 You should not alter the set of characters that contain the 128 bit, as that
290 will cause PCRE to malfunction.
291
292
293 Manifest
294 --------
295
296 The distribution should contain the following files:
297
298 (A) The actual source files of the PCRE library functions and their
299 headers:
300
301 dftables.c auxiliary program for building chartables.c
302 get.c )
303 maketables.c )
304 study.c ) source of
305 pcre.c ) the functions
306 pcreposix.c )
307 printint.c )
308 pcre.in "source" for the header for the external API; pcre.h
309 is built from this by "configure"
310 pcreposix.h header for the external POSIX wrapper API
311 internal.h header for internal use
312 config.in template for config.h, which is built by configure
313
314 (B) Auxiliary files:
315
316 AUTHORS information about the author of PCRE
317 ChangeLog log of changes to the code
318 INSTALL generic installation instructions
319 LICENCE conditions for the use of PCRE
320 COPYING the same, using GNU's standard name
321 Makefile.in template for Unix Makefile, which is built by configure
322 NEWS important changes in this release
323 NON-UNIX-USE notes on building PCRE on non-Unix systems
324 README this file
325 RunTest.in template for a Unix shell script for running tests
326 config.guess ) files used by libtool,
327 config.sub ) used only when building a shared library
328 configure a configuring shell script (built by autoconf)
329 configure.in the autoconf input used to build configure
330 doc/Tech.Notes notes on the encoding
331 doc/*.3 man page sources for the PCRE functions
332 doc/*.1 man page sources for pcregrep and pcretest
333 doc/html/* HTML documentation
334 doc/pcre.txt plain text version of the man pages
335 doc/pcretest.txt plain text documentation of test program
336 doc/perltest.txt plain text documentation of Perl test program
337 install-sh a shell script for installing files
338 ltmain.sh file used to build a libtool script
339 pcretest.c comprehensive test program
340 pcredemo.c simple demonstration of coding calls to PCRE
341 perltest Perl test program
342 pcregrep.c source of a grep utility that uses PCRE
343 pcre-config.in source of script which retains PCRE information
344 testdata/testinput1 test data, compatible with Perl
345 testdata/testinput2 test data for error messages and non-Perl things
346 testdata/testinput3 test data for locale-specific tests
347 testdata/testinput4 test data for UTF-8 tests compatible with Perl
348 testdata/testinput5 test data for other UTF-8 tests
349 testdata/testoutput1 test results corresponding to testinput1
350 testdata/testoutput2 test results corresponding to testinput2
351 testdata/testoutput3 test results corresponding to testinput3
352 testdata/testoutput4 test results corresponding to testinput4
353 testdata/testoutput5 test results corresponding to testinput5
354
355 (C) Auxiliary files for Win32 DLL
356
357 dll.mk
358 pcre.def
359
360 (D) Auxiliary file for VPASCAL
361
362 makevp.bat
363
364 Philip Hazel <ph10@cam.ac.uk>
365 December 2003

webmaster@exim.org
ViewVC Help
Powered by ViewVC 1.1.12