ViewVC logotype

Contents of /code/trunk/README

Parent Directory Parent Directory | Revision Log Revision Log

Revision 65 - (hide annotations) (download)
Sat Feb 24 21:40:08 2007 UTC (8 years, 2 months ago) by nigel
File size: 16335 byte(s)
Load pcre-4.1 into code/trunk.

1 nigel 41 README file for PCRE (Perl-compatible regular expression library)
2     -----------------------------------------------------------------
3 nigel 3
4 nigel 43 The latest release of PCRE is always available from
6     ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
8 nigel 41 Please read the NEWS file if you are upgrading from a previous release.
9 nigel 23
10 nigel 49 PCRE has its own native API, but a set of "wrapper" functions that are based on
11     the POSIX API are also supplied in the library libpcreposix. Note that this
12     just provides a POSIX calling interface to PCRE: the regular expressions
13     themselves still follow Perl syntax and semantics. The header file
14     for the POSIX-style functions is called pcreposix.h. The official POSIX name is
15     regex.h, but I didn't want to risk possible problems with existing files of
16     that name by distributing it that way. To use it with an existing program that
17     uses the POSIX API, it will have to be renamed or pointed at by a link.
18 nigel 35
19 nigel 49
20 nigel 53 Contributions by users of PCRE
21     ------------------------------
23     You can find contributions from PCRE users in the directory
25     ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
27     where there is also a README file giving brief descriptions of what they are.
28     Several of them provide support for compiling PCRE on various flavours of
29     Windows systems (I myself do not use Windows). Some are complete in themselves;
30     others are pointers to URLs containing relevant files.
33 nigel 63 Building PCRE on a Unix-like system
34     -----------------------------------
35 nigel 3
36 nigel 63 To build PCRE on a Unix-like system, first run the "configure" command from the
37     PCRE distribution directory, with your current directory set to the directory
38     where you want the files to be created. This command is a standard GNU
39     "autoconf" configuration script, for which generic instructions are supplied in
40     INSTALL.
41 nigel 3
42 nigel 53 Most commonly, people build PCRE within its own distribution directory, and in
43     this case, on many systems, just running "./configure" is sufficient, but the
44     usual methods of changing standard defaults are available. For example,
46 nigel 41 CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
48     specifies that the C compiler should be run with the flags '-O2 -Wall' instead
49     of the default, and that "make install" should install PCRE under /opt/local
50 nigel 49 instead of the default /usr/local.
51 nigel 41
52 nigel 53 If you want to build in a different directory, just run "configure" with that
53     directory as current. For example, suppose you have unpacked the PCRE source
54     into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
56     cd /build/pcre/pcre-xxx
57     /source/pcre/pcre-xxx/configure
59 nigel 63 There are some optional features that can be included or omitted from the PCRE
60     library. You can read more about them in the pcrebuild man page.
61 nigel 49
62 nigel 63 . If you want to make use of the support for UTF-8 character strings in PCRE,
63     you must add --enable-utf8 to the "configure" command. Without it, the code
64     for handling UTF-8 is not included in the library. (Even when included, it
65     still has to be enabled by an option at run time.)
67     . You can build PCRE to recognized CR or NL as the newline character, instead
68     of whatever your compiler uses for "\n", by adding --newline-is-cr or
69     --newline-is-nl to the "configure" command, respectively. Only do this if you
70     really understand what you are doing. On traditional Unix-like systems, the
71     newline character is NL.
73     . When called via the POSIX interface, PCRE uses malloc() to get additional
74     storage for processing capturing parentheses if there are more than 10 of
75     them. You can increase this threshold by setting, for example,
77     --with-posix-malloc-threshold=20
79     on the "configure" command.
81     . PCRE has a counter which can be set to limit the amount of resources it uses.
82     If the limit is exceeded during a match, the match fails. The default is ten
83     million. You can change the default by setting, for example,
85     --with-match-limit=500000
87     on the "configure" command. This is just the default; individual calls to
88     pcre_exec() can supply their own value. There is discussion on the pcreapi
89     man page.
91     . The default maximum compiled pattern size is around 64K. You can increase
92     this by adding --with-link-size=3 to the "configure" command. You can
93     increase it even more by setting --with-link-size=4, but this is unlikely
94     ever to be necessary. If you build PCRE with an increased link size, test 2
95     (and 5 if you are using UTF-8) will fail. Part of the output of these tests
96     is a representation of the compiled pattern, and this changes with the link
97     size.
99 nigel 53 The "configure" script builds five files:
100 nigel 49
101 nigel 53 . libtool is a script that builds shared and/or static libraries
102 nigel 43 . Makefile is built by copying Makefile.in and making substitutions.
103     . config.h is built by copying config.in and making substitutions.
104     . pcre-config is built by copying pcre-config.in and making substitutions.
105 nigel 49 . RunTest is a script for running tests
106 nigel 41
107     Once "configure" has run, you can run "make". It builds two libraries called
108 nigel 49 libpcre and libpcreposix, a test program called pcretest, and the pcregrep
109 nigel 53 command. You can use "make install" to copy these, the public header files
110     pcre.h and pcreposix.h, and the man pages to appropriate live directories on
111     your system, in the normal way.
112 nigel 3
113 nigel 43 Running "make install" also installs the command pcre-config, which can be used
114     to recall information about the PCRE configuration and installation. For
115     example,
116 nigel 37
117 nigel 43 pcre-config --version
119     prints the version number, and
121     pcre-config --libs
123     outputs information about where the library is installed. This command can be
124     included in makefiles for programs that use PCRE, saving the programmer from
125     having to remember too many details.
128 nigel 65 Cross-compiling PCRE on a Unix-like system
129     ------------------------------------------
131     PCRE needs to compile and run an auxiliary program as part of the building
132     process. Obviously, if the real compilation is for some other system, it can't
133     use the same CC and CFLAGS values when it is doing this. For cross compilation,
134     therefore, you must set CC_FOR_BUILD to the local host's compiler, and you can
135     set flags in CFLAGS_FOR_BUILD if you need to.
138 nigel 63 Shared libraries on Unix-like systems
139     -------------------------------------
140 nigel 53
141     The default distribution builds PCRE as two shared libraries and two static
142     libraries, as long as the operating system supports shared libraries. Shared
143     library support relies on the "libtool" script which is built as part of the
144     "configure" process.
145 nigel 39
146 nigel 53 The libtool script is used to compile and link both shared and static
147     libraries. They are placed in a subdirectory called .libs when they are newly
148     built. The programs pcretest and pcregrep are built to use these uninstalled
149     libraries (by means of wrapper scripts in the case of shared libraries). When
150     you use "make install" to install shared libraries, pcregrep and pcretest are
151     automatically re-built to use the newly installed shared libraries before being
152     installed themselves. However, the versions left in the source directory still
153     use the uninstalled libraries.
155     To build PCRE using static libraries only you must use --disable-shared when
156 nigel 43 configuring it. For example
157 nigel 3
158 nigel 43 ./configure --prefix=/usr/gnu --disable-shared
159 nigel 41
160 nigel 53 Then run "make" in the usual way. Similarly, you can use --disable-static to
161     build only shared libraries.
162 nigel 41
163 nigel 43
164 nigel 63 Cross-compiling on a Unix-like system
165     -------------------------------------
167     You can specify CC and CFLAGS in the normal way to the "configure" command, in
168     order to cross-compile PCRE for some other host. However, during the building
169     process, the dftables.c source file is compiled *and run* on the local host, in
170     order to generate the default character tables (the chartables.c file). It
171     therefore needs to be compiled with the local compiler, not the cross compiler.
172     You can do this by specifying HOST_CC (and if necessary HOST_CFLAGS) when
173     calling the "configure" command. If they are not specified, they default to the
174     values of CC and CFLAGS.
177 nigel 41 Building on non-Unix systems
178     ----------------------------
180     For a non-Unix system, read the comments in the file NON-UNIX-USE. PCRE has
181     been compiled on Windows systems and on Macintoshes, but I don't know the
182     details because I don't use those systems. It should be straightforward to
183     build PCRE on any system that has a Standard C compiler, because it uses only
184     Standard C functions.
187     Testing PCRE
188     ------------
190 nigel 53 To test PCRE on a Unix system, run the RunTest script that is created by the
191     configuring process. (This can also be run by "make runtest", "make check", or
192     "make test".) For other systems, see the instruction in NON-UNIX-USE.
193 nigel 41
194 nigel 63 The script runs the pcretest test program (which is documented in its own man
195     page) on each of the testinput files (in the testdata directory) in turn,
196 nigel 53 and compares the output with the contents of the corresponding testoutput file.
197     A file called testtry is used to hold the output from pcretest. To run pcretest
198     on just one of the test files, give its number as an argument to RunTest, for
199     example:
200 nigel 41
201 nigel 63 RunTest 2
202 nigel 3
203 nigel 63 The first file can also be fed directly into the perltest script to check that
204     Perl gives the same results. The only difference you should see is in the first
205     few lines, where the Perl version is given instead of the PCRE version.
206 nigel 3
207 nigel 49 The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
208     pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
209     detection, and run-time flags that are specific to PCRE, as well as the POSIX
210     wrapper API. It also uses the debugging flag to check some of the internals of
211     pcre_compile().
212 nigel 7
213 nigel 49 If you build PCRE with a locale setting that is not the standard C locale, the
214     character tables may be different (see next paragraph). In some cases, this may
215     cause failures in the second set of tests. For example, in a locale where the
216     isprint() function yields TRUE for characters in the range 128-255, the use of
217     [:isascii:] inside a character class defines a different set of characters, and
218     this shows up in this test as a difference in the compiled code, which is being
219     listed for checking. Where the comparison test output contains [\x00-\x7f] the
220     test will contain [\x00-\xff], and similarly in some other cases. This is not a
221     bug in PCRE.
223 nigel 63 The third set of tests checks pcre_maketables(), the facility for building a
224 nigel 25 set of character tables for a specific locale and using them instead of the
225     default tables. The tests make use of the "fr" (French) locale. Before running
226     the test, the script checks for the presence of this locale by running the
227     "locale" command. If that command fails, or if it doesn't include "fr" in the
228 nigel 63 list of available locales, the third test cannot be run, and a comment is
229 nigel 25 output to say why. If running this test produces instances of the error
231     ** Failed to set locale "fr"
233     in the comparison output, it means that locale is not available on your system,
234     despite being listed by "locale". This does not mean that PCRE is broken.
236 nigel 63 The fourth test checks the UTF-8 support. It is not run automatically unless
237     PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
238     running "configure". This file can be also fed directly to the perltest script,
239     provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
240     commented in the script, can be be used.)
241 nigel 3
242 nigel 63 The fifth and final file tests error handling with UTF-8 encoding, and internal
243     UTF-8 features of PCRE that are not relevant to Perl.
244 nigel 3
245 nigel 63
246 nigel 3 Character tables
247     ----------------
249 nigel 25 PCRE uses four tables for manipulating and identifying characters. The final
250     argument of the pcre_compile() function is a pointer to a block of memory
251 nigel 35 containing the concatenated tables. A call to pcre_maketables() can be used to
252     generate a set of tables in the current locale. If the final argument for
253     pcre_compile() is passed as NULL, a set of default tables that is built into
254     the binary is used.
255 nigel 3
256 nigel 25 The source file called chartables.c contains the default set of tables. This is
257 nigel 27 not supplied in the distribution, but is built by the program dftables
258     (compiled from dftables.c), which uses the ANSI C character handling functions
259 nigel 25 such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table
260 nigel 35 sources. This means that the default C locale which is set for your system will
261     control the contents of these default tables. You can change the default tables
262     by editing chartables.c and then re-building PCRE. If you do this, you should
263     probably also edit Makefile to ensure that the file doesn't ever get
264     re-generated.
265 nigel 3
266 nigel 25 The first two 256-byte tables provide lower casing and case flipping functions,
267     respectively. The next table consists of three 32-byte bit maps which identify
268     digits, "word" characters, and white space, respectively. These are used when
269     building 32-byte bit maps that represent character classes.
271     The final 256-byte table has bits indicating various character types, as
272 nigel 3 follows:
274     1 white space character
275     2 letter
276     4 decimal digit
277     8 hexadecimal digit
278     16 alphanumeric or '_'
279     128 regular expression metacharacter or binary zero
281     You should not alter the set of characters that contain the 128 bit, as that
282     will cause PCRE to malfunction.
285 nigel 41 Manifest
286     --------
287 nigel 3
288 nigel 41 The distribution should contain the following files:
289 nigel 3
290 nigel 41 (A) The actual source files of the PCRE library functions and their
291     headers:
292 nigel 3
293 nigel 41 dftables.c auxiliary program for building chartables.c
294     get.c )
295     maketables.c )
296     study.c ) source of
297     pcre.c ) the functions
298     pcreposix.c )
299 nigel 63 printint.c )
300 nigel 43 pcre.in "source" for the header for the external API; pcre.h
301     is built from this by "configure"
302 nigel 41 pcreposix.h header for the external POSIX wrapper API
303     internal.h header for internal use
304     config.in template for config.h, which is built by configure
305 nigel 3
306 nigel 41 (B) Auxiliary files:
307 nigel 3
308 nigel 41 AUTHORS information about the author of PCRE
309     ChangeLog log of changes to the code
310     INSTALL generic installation instructions
311     LICENCE conditions for the use of PCRE
312 nigel 43 COPYING the same, using GNU's standard name
313 nigel 41 Makefile.in template for Unix Makefile, which is built by configure
314     NEWS important changes in this release
315     NON-UNIX-USE notes on building PCRE on non-Unix systems
316     README this file
317 nigel 49 RunTest.in template for a Unix shell script for running tests
318 nigel 41 config.guess ) files used by libtool,
319     config.sub ) used only when building a shared library
320     configure a configuring shell script (built by autoconf)
321     configure.in the autoconf input used to build configure
322     doc/Tech.Notes notes on the encoding
323 nigel 63 doc/*.3 man page sources for the PCRE functions
324     doc/*.1 man page sources for pcregrep and pcretest
325     doc/html/* HTML documentation
326     doc/pcre.txt plain text version of the man pages
327     doc/pcretest.txt plain text documentation of test program
328     doc/perltest.txt plain text documentation of Perl test program
329 nigel 41 install-sh a shell script for installing files
330 nigel 53 ltmain.sh file used to build a libtool script
331     pcretest.c comprehensive test program
332     pcredemo.c simple demonstration of coding calls to PCRE
333 nigel 41 perltest Perl test program
334 nigel 49 pcregrep.c source of a grep utility that uses PCRE
335 nigel 43 pcre-config.in source of script which retains PCRE information
336 nigel 63 testdata/testinput1 test data, compatible with Perl
337 nigel 41 testdata/testinput2 test data for error messages and non-Perl things
338 nigel 63 testdata/testinput3 test data for locale-specific tests
339     testdata/testinput4 test data for UTF-8 tests compatible with Perl
340     testdata/testinput5 test data for other UTF-8 tests
341 nigel 41 testdata/testoutput1 test results corresponding to testinput1
342     testdata/testoutput2 test results corresponding to testinput2
343     testdata/testoutput3 test results corresponding to testinput3
344     testdata/testoutput4 test results corresponding to testinput4
345 nigel 49 testdata/testoutput5 test results corresponding to testinput5
346 nigel 3
347 nigel 41 (C) Auxiliary files for Win32 DLL
348 nigel 29
349 nigel 41 dll.mk
350     pcre.def
351 nigel 29
352 nigel 63 (D) Auxiliary file for VPASCAL
354     makevp.bat
356 nigel 3 Philip Hazel <ph10@cam.ac.uk>
357 nigel 63 February 2003

ViewVC Help
Powered by ViewVC 1.1.12