/[pcre]/code/trunk/README
ViewVC logotype

Contents of /code/trunk/README

Parent Directory Parent Directory | Revision Log Revision Log


Revision 63 - (show annotations) (download)
Sat Feb 24 21:40:03 2007 UTC (7 years, 2 months ago) by nigel
File size: 15887 byte(s)
Load pcre-4.0 into code/trunk.

1 README file for PCRE (Perl-compatible regular expression library)
2 -----------------------------------------------------------------
3
4 The latest release of PCRE is always available from
5
6 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
7
8 Please read the NEWS file if you are upgrading from a previous release.
9
10 PCRE has its own native API, but a set of "wrapper" functions that are based on
11 the POSIX API are also supplied in the library libpcreposix. Note that this
12 just provides a POSIX calling interface to PCRE: the regular expressions
13 themselves still follow Perl syntax and semantics. The header file
14 for the POSIX-style functions is called pcreposix.h. The official POSIX name is
15 regex.h, but I didn't want to risk possible problems with existing files of
16 that name by distributing it that way. To use it with an existing program that
17 uses the POSIX API, it will have to be renamed or pointed at by a link.
18
19
20 Contributions by users of PCRE
21 ------------------------------
22
23 You can find contributions from PCRE users in the directory
24
25 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
26
27 where there is also a README file giving brief descriptions of what they are.
28 Several of them provide support for compiling PCRE on various flavours of
29 Windows systems (I myself do not use Windows). Some are complete in themselves;
30 others are pointers to URLs containing relevant files.
31
32
33 Building PCRE on a Unix-like system
34 -----------------------------------
35
36 To build PCRE on a Unix-like system, first run the "configure" command from the
37 PCRE distribution directory, with your current directory set to the directory
38 where you want the files to be created. This command is a standard GNU
39 "autoconf" configuration script, for which generic instructions are supplied in
40 INSTALL.
41
42 Most commonly, people build PCRE within its own distribution directory, and in
43 this case, on many systems, just running "./configure" is sufficient, but the
44 usual methods of changing standard defaults are available. For example,
45
46 CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
47
48 specifies that the C compiler should be run with the flags '-O2 -Wall' instead
49 of the default, and that "make install" should install PCRE under /opt/local
50 instead of the default /usr/local.
51
52 If you want to build in a different directory, just run "configure" with that
53 directory as current. For example, suppose you have unpacked the PCRE source
54 into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
55
56 cd /build/pcre/pcre-xxx
57 /source/pcre/pcre-xxx/configure
58
59 There are some optional features that can be included or omitted from the PCRE
60 library. You can read more about them in the pcrebuild man page.
61
62 . If you want to make use of the support for UTF-8 character strings in PCRE,
63 you must add --enable-utf8 to the "configure" command. Without it, the code
64 for handling UTF-8 is not included in the library. (Even when included, it
65 still has to be enabled by an option at run time.)
66
67 . You can build PCRE to recognized CR or NL as the newline character, instead
68 of whatever your compiler uses for "\n", by adding --newline-is-cr or
69 --newline-is-nl to the "configure" command, respectively. Only do this if you
70 really understand what you are doing. On traditional Unix-like systems, the
71 newline character is NL.
72
73 . When called via the POSIX interface, PCRE uses malloc() to get additional
74 storage for processing capturing parentheses if there are more than 10 of
75 them. You can increase this threshold by setting, for example,
76
77 --with-posix-malloc-threshold=20
78
79 on the "configure" command.
80
81 . PCRE has a counter which can be set to limit the amount of resources it uses.
82 If the limit is exceeded during a match, the match fails. The default is ten
83 million. You can change the default by setting, for example,
84
85 --with-match-limit=500000
86
87 on the "configure" command. This is just the default; individual calls to
88 pcre_exec() can supply their own value. There is discussion on the pcreapi
89 man page.
90
91 . The default maximum compiled pattern size is around 64K. You can increase
92 this by adding --with-link-size=3 to the "configure" command. You can
93 increase it even more by setting --with-link-size=4, but this is unlikely
94 ever to be necessary. If you build PCRE with an increased link size, test 2
95 (and 5 if you are using UTF-8) will fail. Part of the output of these tests
96 is a representation of the compiled pattern, and this changes with the link
97 size.
98
99 The "configure" script builds five files:
100
101 . libtool is a script that builds shared and/or static libraries
102 . Makefile is built by copying Makefile.in and making substitutions.
103 . config.h is built by copying config.in and making substitutions.
104 . pcre-config is built by copying pcre-config.in and making substitutions.
105 . RunTest is a script for running tests
106
107 Once "configure" has run, you can run "make". It builds two libraries called
108 libpcre and libpcreposix, a test program called pcretest, and the pcregrep
109 command. You can use "make install" to copy these, the public header files
110 pcre.h and pcreposix.h, and the man pages to appropriate live directories on
111 your system, in the normal way.
112
113 Running "make install" also installs the command pcre-config, which can be used
114 to recall information about the PCRE configuration and installation. For
115 example,
116
117 pcre-config --version
118
119 prints the version number, and
120
121 pcre-config --libs
122
123 outputs information about where the library is installed. This command can be
124 included in makefiles for programs that use PCRE, saving the programmer from
125 having to remember too many details.
126
127
128 Shared libraries on Unix-like systems
129 -------------------------------------
130
131 The default distribution builds PCRE as two shared libraries and two static
132 libraries, as long as the operating system supports shared libraries. Shared
133 library support relies on the "libtool" script which is built as part of the
134 "configure" process.
135
136 The libtool script is used to compile and link both shared and static
137 libraries. They are placed in a subdirectory called .libs when they are newly
138 built. The programs pcretest and pcregrep are built to use these uninstalled
139 libraries (by means of wrapper scripts in the case of shared libraries). When
140 you use "make install" to install shared libraries, pcregrep and pcretest are
141 automatically re-built to use the newly installed shared libraries before being
142 installed themselves. However, the versions left in the source directory still
143 use the uninstalled libraries.
144
145 To build PCRE using static libraries only you must use --disable-shared when
146 configuring it. For example
147
148 ./configure --prefix=/usr/gnu --disable-shared
149
150 Then run "make" in the usual way. Similarly, you can use --disable-static to
151 build only shared libraries.
152
153
154 Cross-compiling on a Unix-like system
155 -------------------------------------
156
157 You can specify CC and CFLAGS in the normal way to the "configure" command, in
158 order to cross-compile PCRE for some other host. However, during the building
159 process, the dftables.c source file is compiled *and run* on the local host, in
160 order to generate the default character tables (the chartables.c file). It
161 therefore needs to be compiled with the local compiler, not the cross compiler.
162 You can do this by specifying HOST_CC (and if necessary HOST_CFLAGS) when
163 calling the "configure" command. If they are not specified, they default to the
164 values of CC and CFLAGS.
165
166
167 Building on non-Unix systems
168 ----------------------------
169
170 For a non-Unix system, read the comments in the file NON-UNIX-USE. PCRE has
171 been compiled on Windows systems and on Macintoshes, but I don't know the
172 details because I don't use those systems. It should be straightforward to
173 build PCRE on any system that has a Standard C compiler, because it uses only
174 Standard C functions.
175
176
177 Testing PCRE
178 ------------
179
180 To test PCRE on a Unix system, run the RunTest script that is created by the
181 configuring process. (This can also be run by "make runtest", "make check", or
182 "make test".) For other systems, see the instruction in NON-UNIX-USE.
183
184 The script runs the pcretest test program (which is documented in its own man
185 page) on each of the testinput files (in the testdata directory) in turn,
186 and compares the output with the contents of the corresponding testoutput file.
187 A file called testtry is used to hold the output from pcretest. To run pcretest
188 on just one of the test files, give its number as an argument to RunTest, for
189 example:
190
191 RunTest 2
192
193 The first file can also be fed directly into the perltest script to check that
194 Perl gives the same results. The only difference you should see is in the first
195 few lines, where the Perl version is given instead of the PCRE version.
196
197 The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
198 pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
199 detection, and run-time flags that are specific to PCRE, as well as the POSIX
200 wrapper API. It also uses the debugging flag to check some of the internals of
201 pcre_compile().
202
203 If you build PCRE with a locale setting that is not the standard C locale, the
204 character tables may be different (see next paragraph). In some cases, this may
205 cause failures in the second set of tests. For example, in a locale where the
206 isprint() function yields TRUE for characters in the range 128-255, the use of
207 [:isascii:] inside a character class defines a different set of characters, and
208 this shows up in this test as a difference in the compiled code, which is being
209 listed for checking. Where the comparison test output contains [\x00-\x7f] the
210 test will contain [\x00-\xff], and similarly in some other cases. This is not a
211 bug in PCRE.
212
213 The third set of tests checks pcre_maketables(), the facility for building a
214 set of character tables for a specific locale and using them instead of the
215 default tables. The tests make use of the "fr" (French) locale. Before running
216 the test, the script checks for the presence of this locale by running the
217 "locale" command. If that command fails, or if it doesn't include "fr" in the
218 list of available locales, the third test cannot be run, and a comment is
219 output to say why. If running this test produces instances of the error
220
221 ** Failed to set locale "fr"
222
223 in the comparison output, it means that locale is not available on your system,
224 despite being listed by "locale". This does not mean that PCRE is broken.
225
226 The fourth test checks the UTF-8 support. It is not run automatically unless
227 PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
228 running "configure". This file can be also fed directly to the perltest script,
229 provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
230 commented in the script, can be be used.)
231
232 The fifth and final file tests error handling with UTF-8 encoding, and internal
233 UTF-8 features of PCRE that are not relevant to Perl.
234
235
236 Character tables
237 ----------------
238
239 PCRE uses four tables for manipulating and identifying characters. The final
240 argument of the pcre_compile() function is a pointer to a block of memory
241 containing the concatenated tables. A call to pcre_maketables() can be used to
242 generate a set of tables in the current locale. If the final argument for
243 pcre_compile() is passed as NULL, a set of default tables that is built into
244 the binary is used.
245
246 The source file called chartables.c contains the default set of tables. This is
247 not supplied in the distribution, but is built by the program dftables
248 (compiled from dftables.c), which uses the ANSI C character handling functions
249 such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table
250 sources. This means that the default C locale which is set for your system will
251 control the contents of these default tables. You can change the default tables
252 by editing chartables.c and then re-building PCRE. If you do this, you should
253 probably also edit Makefile to ensure that the file doesn't ever get
254 re-generated.
255
256 The first two 256-byte tables provide lower casing and case flipping functions,
257 respectively. The next table consists of three 32-byte bit maps which identify
258 digits, "word" characters, and white space, respectively. These are used when
259 building 32-byte bit maps that represent character classes.
260
261 The final 256-byte table has bits indicating various character types, as
262 follows:
263
264 1 white space character
265 2 letter
266 4 decimal digit
267 8 hexadecimal digit
268 16 alphanumeric or '_'
269 128 regular expression metacharacter or binary zero
270
271 You should not alter the set of characters that contain the 128 bit, as that
272 will cause PCRE to malfunction.
273
274
275 Manifest
276 --------
277
278 The distribution should contain the following files:
279
280 (A) The actual source files of the PCRE library functions and their
281 headers:
282
283 dftables.c auxiliary program for building chartables.c
284 get.c )
285 maketables.c )
286 study.c ) source of
287 pcre.c ) the functions
288 pcreposix.c )
289 printint.c )
290 pcre.in "source" for the header for the external API; pcre.h
291 is built from this by "configure"
292 pcreposix.h header for the external POSIX wrapper API
293 internal.h header for internal use
294 config.in template for config.h, which is built by configure
295
296 (B) Auxiliary files:
297
298 AUTHORS information about the author of PCRE
299 ChangeLog log of changes to the code
300 INSTALL generic installation instructions
301 LICENCE conditions for the use of PCRE
302 COPYING the same, using GNU's standard name
303 Makefile.in template for Unix Makefile, which is built by configure
304 NEWS important changes in this release
305 NON-UNIX-USE notes on building PCRE on non-Unix systems
306 README this file
307 RunTest.in template for a Unix shell script for running tests
308 config.guess ) files used by libtool,
309 config.sub ) used only when building a shared library
310 configure a configuring shell script (built by autoconf)
311 configure.in the autoconf input used to build configure
312 doc/Tech.Notes notes on the encoding
313 doc/*.3 man page sources for the PCRE functions
314 doc/*.1 man page sources for pcregrep and pcretest
315 doc/html/* HTML documentation
316 doc/pcre.txt plain text version of the man pages
317 doc/pcretest.txt plain text documentation of test program
318 doc/perltest.txt plain text documentation of Perl test program
319 install-sh a shell script for installing files
320 ltmain.sh file used to build a libtool script
321 pcretest.c comprehensive test program
322 pcredemo.c simple demonstration of coding calls to PCRE
323 perltest Perl test program
324 pcregrep.c source of a grep utility that uses PCRE
325 pcre-config.in source of script which retains PCRE information
326 testdata/testinput1 test data, compatible with Perl
327 testdata/testinput2 test data for error messages and non-Perl things
328 testdata/testinput3 test data for locale-specific tests
329 testdata/testinput4 test data for UTF-8 tests compatible with Perl
330 testdata/testinput5 test data for other UTF-8 tests
331 testdata/testoutput1 test results corresponding to testinput1
332 testdata/testoutput2 test results corresponding to testinput2
333 testdata/testoutput3 test results corresponding to testinput3
334 testdata/testoutput4 test results corresponding to testinput4
335 testdata/testoutput5 test results corresponding to testinput5
336
337 (C) Auxiliary files for Win32 DLL
338
339 dll.mk
340 pcre.def
341
342 (D) Auxiliary file for VPASCAL
343
344 makevp.bat
345
346 Philip Hazel <ph10@cam.ac.uk>
347 February 2003

webmaster@exim.org
ViewVC Help
Powered by ViewVC 1.1.12