ViewVC logotype

Contents of /code/trunk/README

Parent Directory Parent Directory | Revision Log Revision Log

Revision 63 - (hide annotations) (download)
Sat Feb 24 21:40:03 2007 UTC (8 years, 1 month ago) by nigel
File size: 15887 byte(s)
Load pcre-4.0 into code/trunk.

1 nigel 41 README file for PCRE (Perl-compatible regular expression library)
2     -----------------------------------------------------------------
3 nigel 3
4 nigel 43 The latest release of PCRE is always available from
6     ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
8 nigel 41 Please read the NEWS file if you are upgrading from a previous release.
9 nigel 23
10 nigel 49 PCRE has its own native API, but a set of "wrapper" functions that are based on
11     the POSIX API are also supplied in the library libpcreposix. Note that this
12     just provides a POSIX calling interface to PCRE: the regular expressions
13     themselves still follow Perl syntax and semantics. The header file
14     for the POSIX-style functions is called pcreposix.h. The official POSIX name is
15     regex.h, but I didn't want to risk possible problems with existing files of
16     that name by distributing it that way. To use it with an existing program that
17     uses the POSIX API, it will have to be renamed or pointed at by a link.
18 nigel 35
19 nigel 49
20 nigel 53 Contributions by users of PCRE
21     ------------------------------
23     You can find contributions from PCRE users in the directory
25     ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
27     where there is also a README file giving brief descriptions of what they are.
28     Several of them provide support for compiling PCRE on various flavours of
29     Windows systems (I myself do not use Windows). Some are complete in themselves;
30     others are pointers to URLs containing relevant files.
33 nigel 63 Building PCRE on a Unix-like system
34     -----------------------------------
35 nigel 3
36 nigel 63 To build PCRE on a Unix-like system, first run the "configure" command from the
37     PCRE distribution directory, with your current directory set to the directory
38     where you want the files to be created. This command is a standard GNU
39     "autoconf" configuration script, for which generic instructions are supplied in
40     INSTALL.
41 nigel 3
42 nigel 53 Most commonly, people build PCRE within its own distribution directory, and in
43     this case, on many systems, just running "./configure" is sufficient, but the
44     usual methods of changing standard defaults are available. For example,
46 nigel 41 CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
48     specifies that the C compiler should be run with the flags '-O2 -Wall' instead
49     of the default, and that "make install" should install PCRE under /opt/local
50 nigel 49 instead of the default /usr/local.
51 nigel 41
52 nigel 53 If you want to build in a different directory, just run "configure" with that
53     directory as current. For example, suppose you have unpacked the PCRE source
54     into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
56     cd /build/pcre/pcre-xxx
57     /source/pcre/pcre-xxx/configure
59 nigel 63 There are some optional features that can be included or omitted from the PCRE
60     library. You can read more about them in the pcrebuild man page.
61 nigel 49
62 nigel 63 . If you want to make use of the support for UTF-8 character strings in PCRE,
63     you must add --enable-utf8 to the "configure" command. Without it, the code
64     for handling UTF-8 is not included in the library. (Even when included, it
65     still has to be enabled by an option at run time.)
67     . You can build PCRE to recognized CR or NL as the newline character, instead
68     of whatever your compiler uses for "\n", by adding --newline-is-cr or
69     --newline-is-nl to the "configure" command, respectively. Only do this if you
70     really understand what you are doing. On traditional Unix-like systems, the
71     newline character is NL.
73     . When called via the POSIX interface, PCRE uses malloc() to get additional
74     storage for processing capturing parentheses if there are more than 10 of
75     them. You can increase this threshold by setting, for example,
77     --with-posix-malloc-threshold=20
79     on the "configure" command.
81     . PCRE has a counter which can be set to limit the amount of resources it uses.
82     If the limit is exceeded during a match, the match fails. The default is ten
83     million. You can change the default by setting, for example,
85     --with-match-limit=500000
87     on the "configure" command. This is just the default; individual calls to
88     pcre_exec() can supply their own value. There is discussion on the pcreapi
89     man page.
91     . The default maximum compiled pattern size is around 64K. You can increase
92     this by adding --with-link-size=3 to the "configure" command. You can
93     increase it even more by setting --with-link-size=4, but this is unlikely
94     ever to be necessary. If you build PCRE with an increased link size, test 2
95     (and 5 if you are using UTF-8) will fail. Part of the output of these tests
96     is a representation of the compiled pattern, and this changes with the link
97     size.
99 nigel 53 The "configure" script builds five files:
100 nigel 49
101 nigel 53 . libtool is a script that builds shared and/or static libraries
102 nigel 43 . Makefile is built by copying Makefile.in and making substitutions.
103     . config.h is built by copying config.in and making substitutions.
104     . pcre-config is built by copying pcre-config.in and making substitutions.
105 nigel 49 . RunTest is a script for running tests
106 nigel 41
107     Once "configure" has run, you can run "make". It builds two libraries called
108 nigel 49 libpcre and libpcreposix, a test program called pcretest, and the pcregrep
109 nigel 53 command. You can use "make install" to copy these, the public header files
110     pcre.h and pcreposix.h, and the man pages to appropriate live directories on
111     your system, in the normal way.
112 nigel 3
113 nigel 43 Running "make install" also installs the command pcre-config, which can be used
114     to recall information about the PCRE configuration and installation. For
115     example,
116 nigel 37
117 nigel 43 pcre-config --version
119     prints the version number, and
121     pcre-config --libs
123     outputs information about where the library is installed. This command can be
124     included in makefiles for programs that use PCRE, saving the programmer from
125     having to remember too many details.
128 nigel 63 Shared libraries on Unix-like systems
129     -------------------------------------
130 nigel 53
131     The default distribution builds PCRE as two shared libraries and two static
132     libraries, as long as the operating system supports shared libraries. Shared
133     library support relies on the "libtool" script which is built as part of the
134     "configure" process.
135 nigel 39
136 nigel 53 The libtool script is used to compile and link both shared and static
137     libraries. They are placed in a subdirectory called .libs when they are newly
138     built. The programs pcretest and pcregrep are built to use these uninstalled
139     libraries (by means of wrapper scripts in the case of shared libraries). When
140     you use "make install" to install shared libraries, pcregrep and pcretest are
141     automatically re-built to use the newly installed shared libraries before being
142     installed themselves. However, the versions left in the source directory still
143     use the uninstalled libraries.
145     To build PCRE using static libraries only you must use --disable-shared when
146 nigel 43 configuring it. For example
147 nigel 3
148 nigel 43 ./configure --prefix=/usr/gnu --disable-shared
149 nigel 41
150 nigel 53 Then run "make" in the usual way. Similarly, you can use --disable-static to
151     build only shared libraries.
152 nigel 41
153 nigel 43
154 nigel 63 Cross-compiling on a Unix-like system
155     -------------------------------------
157     You can specify CC and CFLAGS in the normal way to the "configure" command, in
158     order to cross-compile PCRE for some other host. However, during the building
159     process, the dftables.c source file is compiled *and run* on the local host, in
160     order to generate the default character tables (the chartables.c file). It
161     therefore needs to be compiled with the local compiler, not the cross compiler.
162     You can do this by specifying HOST_CC (and if necessary HOST_CFLAGS) when
163     calling the "configure" command. If they are not specified, they default to the
164     values of CC and CFLAGS.
167 nigel 41 Building on non-Unix systems
168     ----------------------------
170     For a non-Unix system, read the comments in the file NON-UNIX-USE. PCRE has
171     been compiled on Windows systems and on Macintoshes, but I don't know the
172     details because I don't use those systems. It should be straightforward to
173     build PCRE on any system that has a Standard C compiler, because it uses only
174     Standard C functions.
177     Testing PCRE
178     ------------
180 nigel 53 To test PCRE on a Unix system, run the RunTest script that is created by the
181     configuring process. (This can also be run by "make runtest", "make check", or
182     "make test".) For other systems, see the instruction in NON-UNIX-USE.
183 nigel 41
184 nigel 63 The script runs the pcretest test program (which is documented in its own man
185     page) on each of the testinput files (in the testdata directory) in turn,
186 nigel 53 and compares the output with the contents of the corresponding testoutput file.
187     A file called testtry is used to hold the output from pcretest. To run pcretest
188     on just one of the test files, give its number as an argument to RunTest, for
189     example:
190 nigel 41
191 nigel 63 RunTest 2
192 nigel 3
193 nigel 63 The first file can also be fed directly into the perltest script to check that
194     Perl gives the same results. The only difference you should see is in the first
195     few lines, where the Perl version is given instead of the PCRE version.
196 nigel 3
197 nigel 49 The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
198     pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
199     detection, and run-time flags that are specific to PCRE, as well as the POSIX
200     wrapper API. It also uses the debugging flag to check some of the internals of
201     pcre_compile().
202 nigel 7
203 nigel 49 If you build PCRE with a locale setting that is not the standard C locale, the
204     character tables may be different (see next paragraph). In some cases, this may
205     cause failures in the second set of tests. For example, in a locale where the
206     isprint() function yields TRUE for characters in the range 128-255, the use of
207     [:isascii:] inside a character class defines a different set of characters, and
208     this shows up in this test as a difference in the compiled code, which is being
209     listed for checking. Where the comparison test output contains [\x00-\x7f] the
210     test will contain [\x00-\xff], and similarly in some other cases. This is not a
211     bug in PCRE.
213 nigel 63 The third set of tests checks pcre_maketables(), the facility for building a
214 nigel 25 set of character tables for a specific locale and using them instead of the
215     default tables. The tests make use of the "fr" (French) locale. Before running
216     the test, the script checks for the presence of this locale by running the
217     "locale" command. If that command fails, or if it doesn't include "fr" in the
218 nigel 63 list of available locales, the third test cannot be run, and a comment is
219 nigel 25 output to say why. If running this test produces instances of the error
221     ** Failed to set locale "fr"
223     in the comparison output, it means that locale is not available on your system,
224     despite being listed by "locale". This does not mean that PCRE is broken.
226 nigel 63 The fourth test checks the UTF-8 support. It is not run automatically unless
227     PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
228     running "configure". This file can be also fed directly to the perltest script,
229     provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
230     commented in the script, can be be used.)
231 nigel 3
232 nigel 63 The fifth and final file tests error handling with UTF-8 encoding, and internal
233     UTF-8 features of PCRE that are not relevant to Perl.
234 nigel 3
235 nigel 63
236 nigel 3 Character tables
237     ----------------
239 nigel 25 PCRE uses four tables for manipulating and identifying characters. The final
240     argument of the pcre_compile() function is a pointer to a block of memory
241 nigel 35 containing the concatenated tables. A call to pcre_maketables() can be used to
242     generate a set of tables in the current locale. If the final argument for
243     pcre_compile() is passed as NULL, a set of default tables that is built into
244     the binary is used.
245 nigel 3
246 nigel 25 The source file called chartables.c contains the default set of tables. This is
247 nigel 27 not supplied in the distribution, but is built by the program dftables
248     (compiled from dftables.c), which uses the ANSI C character handling functions
249 nigel 25 such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table
250 nigel 35 sources. This means that the default C locale which is set for your system will
251     control the contents of these default tables. You can change the default tables
252     by editing chartables.c and then re-building PCRE. If you do this, you should
253     probably also edit Makefile to ensure that the file doesn't ever get
254     re-generated.
255 nigel 3
256 nigel 25 The first two 256-byte tables provide lower casing and case flipping functions,
257     respectively. The next table consists of three 32-byte bit maps which identify
258     digits, "word" characters, and white space, respectively. These are used when
259     building 32-byte bit maps that represent character classes.
261     The final 256-byte table has bits indicating various character types, as
262 nigel 3 follows:
264     1 white space character
265     2 letter
266     4 decimal digit
267     8 hexadecimal digit
268     16 alphanumeric or '_'
269     128 regular expression metacharacter or binary zero
271     You should not alter the set of characters that contain the 128 bit, as that
272     will cause PCRE to malfunction.
275 nigel 41 Manifest
276     --------
277 nigel 3
278 nigel 41 The distribution should contain the following files:
279 nigel 3
280 nigel 41 (A) The actual source files of the PCRE library functions and their
281     headers:
282 nigel 3
283 nigel 41 dftables.c auxiliary program for building chartables.c
284     get.c )
285     maketables.c )
286     study.c ) source of
287     pcre.c ) the functions
288     pcreposix.c )
289 nigel 63 printint.c )
290 nigel 43 pcre.in "source" for the header for the external API; pcre.h
291     is built from this by "configure"
292 nigel 41 pcreposix.h header for the external POSIX wrapper API
293     internal.h header for internal use
294     config.in template for config.h, which is built by configure
295 nigel 3
296 nigel 41 (B) Auxiliary files:
297 nigel 3
298 nigel 41 AUTHORS information about the author of PCRE
299     ChangeLog log of changes to the code
300     INSTALL generic installation instructions
301     LICENCE conditions for the use of PCRE
302 nigel 43 COPYING the same, using GNU's standard name
303 nigel 41 Makefile.in template for Unix Makefile, which is built by configure
304     NEWS important changes in this release
305     NON-UNIX-USE notes on building PCRE on non-Unix systems
306     README this file
307 nigel 49 RunTest.in template for a Unix shell script for running tests
308 nigel 41 config.guess ) files used by libtool,
309     config.sub ) used only when building a shared library
310     configure a configuring shell script (built by autoconf)
311     configure.in the autoconf input used to build configure
312     doc/Tech.Notes notes on the encoding
313 nigel 63 doc/*.3 man page sources for the PCRE functions
314     doc/*.1 man page sources for pcregrep and pcretest
315     doc/html/* HTML documentation
316     doc/pcre.txt plain text version of the man pages
317     doc/pcretest.txt plain text documentation of test program
318     doc/perltest.txt plain text documentation of Perl test program
319 nigel 41 install-sh a shell script for installing files
320 nigel 53 ltmain.sh file used to build a libtool script
321     pcretest.c comprehensive test program
322     pcredemo.c simple demonstration of coding calls to PCRE
323 nigel 41 perltest Perl test program
324 nigel 49 pcregrep.c source of a grep utility that uses PCRE
325 nigel 43 pcre-config.in source of script which retains PCRE information
326 nigel 63 testdata/testinput1 test data, compatible with Perl
327 nigel 41 testdata/testinput2 test data for error messages and non-Perl things
328 nigel 63 testdata/testinput3 test data for locale-specific tests
329     testdata/testinput4 test data for UTF-8 tests compatible with Perl
330     testdata/testinput5 test data for other UTF-8 tests
331 nigel 41 testdata/testoutput1 test results corresponding to testinput1
332     testdata/testoutput2 test results corresponding to testinput2
333     testdata/testoutput3 test results corresponding to testinput3
334     testdata/testoutput4 test results corresponding to testinput4
335 nigel 49 testdata/testoutput5 test results corresponding to testinput5
336 nigel 3
337 nigel 41 (C) Auxiliary files for Win32 DLL
338 nigel 29
339 nigel 41 dll.mk
340     pcre.def
341 nigel 29
342 nigel 63 (D) Auxiliary file for VPASCAL
344     makevp.bat
346 nigel 3 Philip Hazel <ph10@cam.ac.uk>
347 nigel 63 February 2003

ViewVC Help
Powered by ViewVC 1.1.12