ViewVC logotype

Contents of /code/trunk/README

Parent Directory Parent Directory | Revision Log Revision Log

Revision 73 - (hide annotations) (download)
Sat Feb 24 21:40:30 2007 UTC (8 years, 1 month ago) by nigel
File size: 16781 byte(s)
Load pcre-4.5 into code/trunk.

1 nigel 41 README file for PCRE (Perl-compatible regular expression library)
2     -----------------------------------------------------------------
3 nigel 3
4 nigel 43 The latest release of PCRE is always available from
6     ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
8 nigel 41 Please read the NEWS file if you are upgrading from a previous release.
9 nigel 23
10 nigel 49 PCRE has its own native API, but a set of "wrapper" functions that are based on
11     the POSIX API are also supplied in the library libpcreposix. Note that this
12     just provides a POSIX calling interface to PCRE: the regular expressions
13     themselves still follow Perl syntax and semantics. The header file
14     for the POSIX-style functions is called pcreposix.h. The official POSIX name is
15     regex.h, but I didn't want to risk possible problems with existing files of
16     that name by distributing it that way. To use it with an existing program that
17     uses the POSIX API, it will have to be renamed or pointed at by a link.
18 nigel 35
19 nigel 73 If you are using the POSIX interface to PCRE and there is already a POSIX regex
20     library installed on your system, you must take care when linking programs to
21     ensure that they link with PCRE's libpcreposix library. Otherwise they may pick
22     up the "real" POSIX functions of the same name.
23 nigel 49
24 nigel 73
25 nigel 53 Contributions by users of PCRE
26     ------------------------------
28     You can find contributions from PCRE users in the directory
30     ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
32     where there is also a README file giving brief descriptions of what they are.
33     Several of them provide support for compiling PCRE on various flavours of
34     Windows systems (I myself do not use Windows). Some are complete in themselves;
35     others are pointers to URLs containing relevant files.
38 nigel 63 Building PCRE on a Unix-like system
39     -----------------------------------
40 nigel 3
41 nigel 63 To build PCRE on a Unix-like system, first run the "configure" command from the
42     PCRE distribution directory, with your current directory set to the directory
43     where you want the files to be created. This command is a standard GNU
44     "autoconf" configuration script, for which generic instructions are supplied in
45     INSTALL.
46 nigel 3
47 nigel 53 Most commonly, people build PCRE within its own distribution directory, and in
48     this case, on many systems, just running "./configure" is sufficient, but the
49     usual methods of changing standard defaults are available. For example,
51 nigel 41 CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
53     specifies that the C compiler should be run with the flags '-O2 -Wall' instead
54     of the default, and that "make install" should install PCRE under /opt/local
55 nigel 49 instead of the default /usr/local.
56 nigel 41
57 nigel 53 If you want to build in a different directory, just run "configure" with that
58     directory as current. For example, suppose you have unpacked the PCRE source
59     into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
61     cd /build/pcre/pcre-xxx
62     /source/pcre/pcre-xxx/configure
64 nigel 63 There are some optional features that can be included or omitted from the PCRE
65     library. You can read more about them in the pcrebuild man page.
66 nigel 49
67 nigel 63 . If you want to make use of the support for UTF-8 character strings in PCRE,
68     you must add --enable-utf8 to the "configure" command. Without it, the code
69     for handling UTF-8 is not included in the library. (Even when included, it
70     still has to be enabled by an option at run time.)
72     . You can build PCRE to recognized CR or NL as the newline character, instead
73     of whatever your compiler uses for "\n", by adding --newline-is-cr or
74     --newline-is-nl to the "configure" command, respectively. Only do this if you
75     really understand what you are doing. On traditional Unix-like systems, the
76     newline character is NL.
78     . When called via the POSIX interface, PCRE uses malloc() to get additional
79     storage for processing capturing parentheses if there are more than 10 of
80     them. You can increase this threshold by setting, for example,
82     --with-posix-malloc-threshold=20
84     on the "configure" command.
86     . PCRE has a counter which can be set to limit the amount of resources it uses.
87     If the limit is exceeded during a match, the match fails. The default is ten
88     million. You can change the default by setting, for example,
90     --with-match-limit=500000
92     on the "configure" command. This is just the default; individual calls to
93     pcre_exec() can supply their own value. There is discussion on the pcreapi
94     man page.
96     . The default maximum compiled pattern size is around 64K. You can increase
97     this by adding --with-link-size=3 to the "configure" command. You can
98     increase it even more by setting --with-link-size=4, but this is unlikely
99     ever to be necessary. If you build PCRE with an increased link size, test 2
100     (and 5 if you are using UTF-8) will fail. Part of the output of these tests
101     is a representation of the compiled pattern, and this changes with the link
102     size.
104 nigel 73 . You can build PCRE so that its match() function does not call itself
105     recursively. Instead, it uses blocks of data from the heap via special
106     functions pcre_stack_malloc() and pcre_stack_free() to save data that would
107     otherwise be saved on the stack. To build PCRE like this, use
109     --disable-stack-for-recursion
111     on the "configure" command. PCRE runs more slowly in this mode, but it may be
112     necessary in environments with limited stack sizes.
114 nigel 53 The "configure" script builds five files:
115 nigel 49
116 nigel 53 . libtool is a script that builds shared and/or static libraries
117 nigel 43 . Makefile is built by copying Makefile.in and making substitutions.
118     . config.h is built by copying config.in and making substitutions.
119     . pcre-config is built by copying pcre-config.in and making substitutions.
120 nigel 49 . RunTest is a script for running tests
121 nigel 41
122     Once "configure" has run, you can run "make". It builds two libraries called
123 nigel 49 libpcre and libpcreposix, a test program called pcretest, and the pcregrep
124 nigel 53 command. You can use "make install" to copy these, the public header files
125     pcre.h and pcreposix.h, and the man pages to appropriate live directories on
126     your system, in the normal way.
127 nigel 3
128 nigel 43 Running "make install" also installs the command pcre-config, which can be used
129     to recall information about the PCRE configuration and installation. For
130     example,
131 nigel 37
132 nigel 43 pcre-config --version
134     prints the version number, and
136     pcre-config --libs
138     outputs information about where the library is installed. This command can be
139     included in makefiles for programs that use PCRE, saving the programmer from
140     having to remember too many details.
143 nigel 63 Shared libraries on Unix-like systems
144     -------------------------------------
145 nigel 53
146     The default distribution builds PCRE as two shared libraries and two static
147     libraries, as long as the operating system supports shared libraries. Shared
148     library support relies on the "libtool" script which is built as part of the
149     "configure" process.
150 nigel 39
151 nigel 53 The libtool script is used to compile and link both shared and static
152     libraries. They are placed in a subdirectory called .libs when they are newly
153     built. The programs pcretest and pcregrep are built to use these uninstalled
154     libraries (by means of wrapper scripts in the case of shared libraries). When
155     you use "make install" to install shared libraries, pcregrep and pcretest are
156     automatically re-built to use the newly installed shared libraries before being
157     installed themselves. However, the versions left in the source directory still
158     use the uninstalled libraries.
160     To build PCRE using static libraries only you must use --disable-shared when
161 nigel 43 configuring it. For example
162 nigel 3
163 nigel 43 ./configure --prefix=/usr/gnu --disable-shared
164 nigel 41
165 nigel 53 Then run "make" in the usual way. Similarly, you can use --disable-static to
166     build only shared libraries.
167 nigel 41
168 nigel 43
169 nigel 63 Cross-compiling on a Unix-like system
170     -------------------------------------
172     You can specify CC and CFLAGS in the normal way to the "configure" command, in
173     order to cross-compile PCRE for some other host. However, during the building
174     process, the dftables.c source file is compiled *and run* on the local host, in
175     order to generate the default character tables (the chartables.c file). It
176     therefore needs to be compiled with the local compiler, not the cross compiler.
177 nigel 73 You can do this by specifying CC_FOR_BUILD (and if necessary CFLAGS_FOR_BUILD)
178     when calling the "configure" command. If they are not specified, they default
179     to the values of CC and CFLAGS.
180 nigel 63
182 nigel 41 Building on non-Unix systems
183     ----------------------------
185 nigel 73 For a non-Unix system, read the comments in the file NON-UNIX-USE, though if
186     the system supports the use of "configure" and "make" you may be able to build
187     PCRE in the same way as for Unix systems.
189     PCRE has been compiled on Windows systems and on Macintoshes, but I don't know
190     the details because I don't use those systems. It should be straightforward to
191 nigel 41 build PCRE on any system that has a Standard C compiler, because it uses only
192     Standard C functions.
195     Testing PCRE
196     ------------
198 nigel 53 To test PCRE on a Unix system, run the RunTest script that is created by the
199     configuring process. (This can also be run by "make runtest", "make check", or
200 nigel 73 "make test".) For other systems, see the instructions in NON-UNIX-USE.
201 nigel 41
202 nigel 63 The script runs the pcretest test program (which is documented in its own man
203     page) on each of the testinput files (in the testdata directory) in turn,
204 nigel 53 and compares the output with the contents of the corresponding testoutput file.
205     A file called testtry is used to hold the output from pcretest. To run pcretest
206     on just one of the test files, give its number as an argument to RunTest, for
207     example:
208 nigel 41
209 nigel 63 RunTest 2
210 nigel 3
211 nigel 63 The first file can also be fed directly into the perltest script to check that
212     Perl gives the same results. The only difference you should see is in the first
213     few lines, where the Perl version is given instead of the PCRE version.
214 nigel 3
215 nigel 49 The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
216     pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
217     detection, and run-time flags that are specific to PCRE, as well as the POSIX
218     wrapper API. It also uses the debugging flag to check some of the internals of
219     pcre_compile().
220 nigel 7
221 nigel 49 If you build PCRE with a locale setting that is not the standard C locale, the
222     character tables may be different (see next paragraph). In some cases, this may
223     cause failures in the second set of tests. For example, in a locale where the
224     isprint() function yields TRUE for characters in the range 128-255, the use of
225     [:isascii:] inside a character class defines a different set of characters, and
226     this shows up in this test as a difference in the compiled code, which is being
227     listed for checking. Where the comparison test output contains [\x00-\x7f] the
228     test will contain [\x00-\xff], and similarly in some other cases. This is not a
229     bug in PCRE.
231 nigel 63 The third set of tests checks pcre_maketables(), the facility for building a
232 nigel 25 set of character tables for a specific locale and using them instead of the
233 nigel 73 default tables. The tests make use of the "fr_FR" (French) locale. Before
234     running the test, the script checks for the presence of this locale by running
235     the "locale" command. If that command fails, or if it doesn't include "fr_FR"
236     in the list of available locales, the third test cannot be run, and a comment
237     is output to say why. If running this test produces instances of the error
238 nigel 25
239 nigel 73 ** Failed to set locale "fr_FR"
240 nigel 25
241     in the comparison output, it means that locale is not available on your system,
242     despite being listed by "locale". This does not mean that PCRE is broken.
244 nigel 63 The fourth test checks the UTF-8 support. It is not run automatically unless
245     PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
246     running "configure". This file can be also fed directly to the perltest script,
247     provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
248     commented in the script, can be be used.)
249 nigel 3
250 nigel 63 The fifth and final file tests error handling with UTF-8 encoding, and internal
251     UTF-8 features of PCRE that are not relevant to Perl.
252 nigel 3
253 nigel 63
254 nigel 3 Character tables
255     ----------------
257 nigel 25 PCRE uses four tables for manipulating and identifying characters. The final
258     argument of the pcre_compile() function is a pointer to a block of memory
259 nigel 35 containing the concatenated tables. A call to pcre_maketables() can be used to
260     generate a set of tables in the current locale. If the final argument for
261     pcre_compile() is passed as NULL, a set of default tables that is built into
262     the binary is used.
263 nigel 3
264 nigel 25 The source file called chartables.c contains the default set of tables. This is
265 nigel 27 not supplied in the distribution, but is built by the program dftables
266     (compiled from dftables.c), which uses the ANSI C character handling functions
267 nigel 25 such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table
268 nigel 35 sources. This means that the default C locale which is set for your system will
269     control the contents of these default tables. You can change the default tables
270     by editing chartables.c and then re-building PCRE. If you do this, you should
271     probably also edit Makefile to ensure that the file doesn't ever get
272     re-generated.
273 nigel 3
274 nigel 25 The first two 256-byte tables provide lower casing and case flipping functions,
275     respectively. The next table consists of three 32-byte bit maps which identify
276     digits, "word" characters, and white space, respectively. These are used when
277     building 32-byte bit maps that represent character classes.
279     The final 256-byte table has bits indicating various character types, as
280 nigel 3 follows:
282     1 white space character
283     2 letter
284     4 decimal digit
285     8 hexadecimal digit
286     16 alphanumeric or '_'
287     128 regular expression metacharacter or binary zero
289     You should not alter the set of characters that contain the 128 bit, as that
290     will cause PCRE to malfunction.
293 nigel 41 Manifest
294     --------
295 nigel 3
296 nigel 41 The distribution should contain the following files:
297 nigel 3
298 nigel 41 (A) The actual source files of the PCRE library functions and their
299     headers:
300 nigel 3
301 nigel 41 dftables.c auxiliary program for building chartables.c
302     get.c )
303     maketables.c )
304     study.c ) source of
305     pcre.c ) the functions
306     pcreposix.c )
307 nigel 63 printint.c )
308 nigel 43 pcre.in "source" for the header for the external API; pcre.h
309     is built from this by "configure"
310 nigel 41 pcreposix.h header for the external POSIX wrapper API
311     internal.h header for internal use
312     config.in template for config.h, which is built by configure
313 nigel 3
314 nigel 41 (B) Auxiliary files:
315 nigel 3
316 nigel 41 AUTHORS information about the author of PCRE
317     ChangeLog log of changes to the code
318     INSTALL generic installation instructions
319     LICENCE conditions for the use of PCRE
320 nigel 43 COPYING the same, using GNU's standard name
321 nigel 41 Makefile.in template for Unix Makefile, which is built by configure
322     NEWS important changes in this release
323     NON-UNIX-USE notes on building PCRE on non-Unix systems
324     README this file
325 nigel 49 RunTest.in template for a Unix shell script for running tests
326 nigel 41 config.guess ) files used by libtool,
327     config.sub ) used only when building a shared library
328     configure a configuring shell script (built by autoconf)
329     configure.in the autoconf input used to build configure
330     doc/Tech.Notes notes on the encoding
331 nigel 63 doc/*.3 man page sources for the PCRE functions
332     doc/*.1 man page sources for pcregrep and pcretest
333     doc/html/* HTML documentation
334     doc/pcre.txt plain text version of the man pages
335     doc/pcretest.txt plain text documentation of test program
336     doc/perltest.txt plain text documentation of Perl test program
337 nigel 41 install-sh a shell script for installing files
338 nigel 53 ltmain.sh file used to build a libtool script
339     pcretest.c comprehensive test program
340     pcredemo.c simple demonstration of coding calls to PCRE
341 nigel 41 perltest Perl test program
342 nigel 49 pcregrep.c source of a grep utility that uses PCRE
343 nigel 43 pcre-config.in source of script which retains PCRE information
344 nigel 63 testdata/testinput1 test data, compatible with Perl
345 nigel 41 testdata/testinput2 test data for error messages and non-Perl things
346 nigel 63 testdata/testinput3 test data for locale-specific tests
347     testdata/testinput4 test data for UTF-8 tests compatible with Perl
348     testdata/testinput5 test data for other UTF-8 tests
349 nigel 41 testdata/testoutput1 test results corresponding to testinput1
350     testdata/testoutput2 test results corresponding to testinput2
351     testdata/testoutput3 test results corresponding to testinput3
352     testdata/testoutput4 test results corresponding to testinput4
353 nigel 49 testdata/testoutput5 test results corresponding to testinput5
354 nigel 3
355 nigel 41 (C) Auxiliary files for Win32 DLL
356 nigel 29
357 nigel 41 dll.mk
358     pcre.def
359 nigel 29
360 nigel 63 (D) Auxiliary file for VPASCAL
362     makevp.bat
364 nigel 3 Philip Hazel <ph10@cam.ac.uk>
365 nigel 73 December 2003

ViewVC Help
Powered by ViewVC 1.1.12