| 1 |
nigel |
41 |
README file for PCRE (Perl-compatible regular expression library) |
| 2 |
|
|
----------------------------------------------------------------- |
| 3 |
nigel |
3 |
|
| 4 |
nigel |
43 |
The latest release of PCRE is always available from |
| 5 |
|
|
|
| 6 |
|
|
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz |
| 7 |
|
|
|
| 8 |
nigel |
41 |
Please read the NEWS file if you are upgrading from a previous release. |
| 9 |
nigel |
23 |
|
| 10 |
nigel |
49 |
PCRE has its own native API, but a set of "wrapper" functions that are based on |
| 11 |
|
|
the POSIX API are also supplied in the library libpcreposix. Note that this |
| 12 |
|
|
just provides a POSIX calling interface to PCRE: the regular expressions |
| 13 |
|
|
themselves still follow Perl syntax and semantics. The header file |
| 14 |
|
|
for the POSIX-style functions is called pcreposix.h. The official POSIX name is |
| 15 |
|
|
regex.h, but I didn't want to risk possible problems with existing files of |
| 16 |
|
|
that name by distributing it that way. To use it with an existing program that |
| 17 |
|
|
uses the POSIX API, it will have to be renamed or pointed at by a link. |
| 18 |
nigel |
35 |
|
| 19 |
nigel |
49 |
|
| 20 |
nigel |
53 |
Contributions by users of PCRE |
| 21 |
|
|
------------------------------ |
| 22 |
|
|
|
| 23 |
|
|
You can find contributions from PCRE users in the directory |
| 24 |
|
|
|
| 25 |
|
|
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib |
| 26 |
|
|
|
| 27 |
|
|
where there is also a README file giving brief descriptions of what they are. |
| 28 |
|
|
Several of them provide support for compiling PCRE on various flavours of |
| 29 |
|
|
Windows systems (I myself do not use Windows). Some are complete in themselves; |
| 30 |
|
|
others are pointers to URLs containing relevant files. |
| 31 |
|
|
|
| 32 |
|
|
|
| 33 |
nigel |
63 |
Building PCRE on a Unix-like system |
| 34 |
|
|
----------------------------------- |
| 35 |
nigel |
3 |
|
| 36 |
nigel |
63 |
To build PCRE on a Unix-like system, first run the "configure" command from the |
| 37 |
|
|
PCRE distribution directory, with your current directory set to the directory |
| 38 |
|
|
where you want the files to be created. This command is a standard GNU |
| 39 |
|
|
"autoconf" configuration script, for which generic instructions are supplied in |
| 40 |
|
|
INSTALL. |
| 41 |
nigel |
3 |
|
| 42 |
nigel |
53 |
Most commonly, people build PCRE within its own distribution directory, and in |
| 43 |
|
|
this case, on many systems, just running "./configure" is sufficient, but the |
| 44 |
|
|
usual methods of changing standard defaults are available. For example, |
| 45 |
|
|
|
| 46 |
nigel |
41 |
CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local |
| 47 |
|
|
|
| 48 |
|
|
specifies that the C compiler should be run with the flags '-O2 -Wall' instead |
| 49 |
|
|
of the default, and that "make install" should install PCRE under /opt/local |
| 50 |
nigel |
49 |
instead of the default /usr/local. |
| 51 |
nigel |
41 |
|
| 52 |
nigel |
53 |
If you want to build in a different directory, just run "configure" with that |
| 53 |
|
|
directory as current. For example, suppose you have unpacked the PCRE source |
| 54 |
|
|
into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx: |
| 55 |
|
|
|
| 56 |
|
|
cd /build/pcre/pcre-xxx |
| 57 |
|
|
/source/pcre/pcre-xxx/configure |
| 58 |
|
|
|
| 59 |
nigel |
63 |
There are some optional features that can be included or omitted from the PCRE |
| 60 |
|
|
library. You can read more about them in the pcrebuild man page. |
| 61 |
nigel |
49 |
|
| 62 |
nigel |
63 |
. If you want to make use of the support for UTF-8 character strings in PCRE, |
| 63 |
|
|
you must add --enable-utf8 to the "configure" command. Without it, the code |
| 64 |
|
|
for handling UTF-8 is not included in the library. (Even when included, it |
| 65 |
|
|
still has to be enabled by an option at run time.) |
| 66 |
|
|
|
| 67 |
|
|
. You can build PCRE to recognized CR or NL as the newline character, instead |
| 68 |
|
|
of whatever your compiler uses for "\n", by adding --newline-is-cr or |
| 69 |
|
|
--newline-is-nl to the "configure" command, respectively. Only do this if you |
| 70 |
|
|
really understand what you are doing. On traditional Unix-like systems, the |
| 71 |
|
|
newline character is NL. |
| 72 |
|
|
|
| 73 |
|
|
. When called via the POSIX interface, PCRE uses malloc() to get additional |
| 74 |
|
|
storage for processing capturing parentheses if there are more than 10 of |
| 75 |
|
|
them. You can increase this threshold by setting, for example, |
| 76 |
|
|
|
| 77 |
|
|
--with-posix-malloc-threshold=20 |
| 78 |
|
|
|
| 79 |
|
|
on the "configure" command. |
| 80 |
|
|
|
| 81 |
|
|
. PCRE has a counter which can be set to limit the amount of resources it uses. |
| 82 |
|
|
If the limit is exceeded during a match, the match fails. The default is ten |
| 83 |
|
|
million. You can change the default by setting, for example, |
| 84 |
|
|
|
| 85 |
|
|
--with-match-limit=500000 |
| 86 |
|
|
|
| 87 |
|
|
on the "configure" command. This is just the default; individual calls to |
| 88 |
|
|
pcre_exec() can supply their own value. There is discussion on the pcreapi |
| 89 |
|
|
man page. |
| 90 |
|
|
|
| 91 |
|
|
. The default maximum compiled pattern size is around 64K. You can increase |
| 92 |
|
|
this by adding --with-link-size=3 to the "configure" command. You can |
| 93 |
|
|
increase it even more by setting --with-link-size=4, but this is unlikely |
| 94 |
|
|
ever to be necessary. If you build PCRE with an increased link size, test 2 |
| 95 |
|
|
(and 5 if you are using UTF-8) will fail. Part of the output of these tests |
| 96 |
|
|
is a representation of the compiled pattern, and this changes with the link |
| 97 |
|
|
size. |
| 98 |
|
|
|
| 99 |
nigel |
53 |
The "configure" script builds five files: |
| 100 |
nigel |
49 |
|
| 101 |
nigel |
53 |
. libtool is a script that builds shared and/or static libraries |
| 102 |
nigel |
43 |
. Makefile is built by copying Makefile.in and making substitutions. |
| 103 |
|
|
. config.h is built by copying config.in and making substitutions. |
| 104 |
|
|
. pcre-config is built by copying pcre-config.in and making substitutions. |
| 105 |
nigel |
49 |
. RunTest is a script for running tests |
| 106 |
nigel |
41 |
|
| 107 |
|
|
Once "configure" has run, you can run "make". It builds two libraries called |
| 108 |
nigel |
49 |
libpcre and libpcreposix, a test program called pcretest, and the pcregrep |
| 109 |
nigel |
53 |
command. You can use "make install" to copy these, the public header files |
| 110 |
|
|
pcre.h and pcreposix.h, and the man pages to appropriate live directories on |
| 111 |
|
|
your system, in the normal way. |
| 112 |
nigel |
3 |
|
| 113 |
nigel |
43 |
Running "make install" also installs the command pcre-config, which can be used |
| 114 |
|
|
to recall information about the PCRE configuration and installation. For |
| 115 |
|
|
example, |
| 116 |
nigel |
37 |
|
| 117 |
nigel |
43 |
pcre-config --version |
| 118 |
|
|
|
| 119 |
|
|
prints the version number, and |
| 120 |
|
|
|
| 121 |
|
|
pcre-config --libs |
| 122 |
|
|
|
| 123 |
|
|
outputs information about where the library is installed. This command can be |
| 124 |
|
|
included in makefiles for programs that use PCRE, saving the programmer from |
| 125 |
|
|
having to remember too many details. |
| 126 |
|
|
|
| 127 |
|
|
|
| 128 |
nigel |
63 |
Shared libraries on Unix-like systems |
| 129 |
|
|
------------------------------------- |
| 130 |
nigel |
53 |
|
| 131 |
|
|
The default distribution builds PCRE as two shared libraries and two static |
| 132 |
|
|
libraries, as long as the operating system supports shared libraries. Shared |
| 133 |
|
|
library support relies on the "libtool" script which is built as part of the |
| 134 |
|
|
"configure" process. |
| 135 |
nigel |
39 |
|
| 136 |
nigel |
53 |
The libtool script is used to compile and link both shared and static |
| 137 |
|
|
libraries. They are placed in a subdirectory called .libs when they are newly |
| 138 |
|
|
built. The programs pcretest and pcregrep are built to use these uninstalled |
| 139 |
|
|
libraries (by means of wrapper scripts in the case of shared libraries). When |
| 140 |
|
|
you use "make install" to install shared libraries, pcregrep and pcretest are |
| 141 |
|
|
automatically re-built to use the newly installed shared libraries before being |
| 142 |
|
|
installed themselves. However, the versions left in the source directory still |
| 143 |
|
|
use the uninstalled libraries. |
| 144 |
|
|
|
| 145 |
|
|
To build PCRE using static libraries only you must use --disable-shared when |
| 146 |
nigel |
43 |
configuring it. For example |
| 147 |
nigel |
3 |
|
| 148 |
nigel |
43 |
./configure --prefix=/usr/gnu --disable-shared |
| 149 |
nigel |
41 |
|
| 150 |
nigel |
53 |
Then run "make" in the usual way. Similarly, you can use --disable-static to |
| 151 |
|
|
build only shared libraries. |
| 152 |
nigel |
41 |
|
| 153 |
nigel |
43 |
|
| 154 |
nigel |
63 |
Cross-compiling on a Unix-like system |
| 155 |
|
|
------------------------------------- |
| 156 |
|
|
|
| 157 |
|
|
You can specify CC and CFLAGS in the normal way to the "configure" command, in |
| 158 |
|
|
order to cross-compile PCRE for some other host. However, during the building |
| 159 |
|
|
process, the dftables.c source file is compiled *and run* on the local host, in |
| 160 |
|
|
order to generate the default character tables (the chartables.c file). It |
| 161 |
|
|
therefore needs to be compiled with the local compiler, not the cross compiler. |
| 162 |
|
|
You can do this by specifying HOST_CC (and if necessary HOST_CFLAGS) when |
| 163 |
|
|
calling the "configure" command. If they are not specified, they default to the |
| 164 |
|
|
values of CC and CFLAGS. |
| 165 |
|
|
|
| 166 |
|
|
|
| 167 |
nigel |
41 |
Building on non-Unix systems |
| 168 |
|
|
---------------------------- |
| 169 |
|
|
|
| 170 |
|
|
For a non-Unix system, read the comments in the file NON-UNIX-USE. PCRE has |
| 171 |
|
|
been compiled on Windows systems and on Macintoshes, but I don't know the |
| 172 |
|
|
details because I don't use those systems. It should be straightforward to |
| 173 |
|
|
build PCRE on any system that has a Standard C compiler, because it uses only |
| 174 |
|
|
Standard C functions. |
| 175 |
|
|
|
| 176 |
|
|
|
| 177 |
|
|
Testing PCRE |
| 178 |
|
|
------------ |
| 179 |
|
|
|
| 180 |
nigel |
53 |
To test PCRE on a Unix system, run the RunTest script that is created by the |
| 181 |
|
|
configuring process. (This can also be run by "make runtest", "make check", or |
| 182 |
|
|
"make test".) For other systems, see the instruction in NON-UNIX-USE. |
| 183 |
nigel |
41 |
|
| 184 |
nigel |
63 |
The script runs the pcretest test program (which is documented in its own man |
| 185 |
|
|
page) on each of the testinput files (in the testdata directory) in turn, |
| 186 |
nigel |
53 |
and compares the output with the contents of the corresponding testoutput file. |
| 187 |
|
|
A file called testtry is used to hold the output from pcretest. To run pcretest |
| 188 |
|
|
on just one of the test files, give its number as an argument to RunTest, for |
| 189 |
|
|
example: |
| 190 |
nigel |
41 |
|
| 191 |
nigel |
63 |
RunTest 2 |
| 192 |
nigel |
3 |
|
| 193 |
nigel |
63 |
The first file can also be fed directly into the perltest script to check that |
| 194 |
|
|
Perl gives the same results. The only difference you should see is in the first |
| 195 |
|
|
few lines, where the Perl version is given instead of the PCRE version. |
| 196 |
nigel |
3 |
|
| 197 |
nigel |
49 |
The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(), |
| 198 |
|
|
pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error |
| 199 |
|
|
detection, and run-time flags that are specific to PCRE, as well as the POSIX |
| 200 |
|
|
wrapper API. It also uses the debugging flag to check some of the internals of |
| 201 |
|
|
pcre_compile(). |
| 202 |
nigel |
7 |
|
| 203 |
nigel |
49 |
If you build PCRE with a locale setting that is not the standard C locale, the |
| 204 |
|
|
character tables may be different (see next paragraph). In some cases, this may |
| 205 |
|
|
cause failures in the second set of tests. For example, in a locale where the |
| 206 |
|
|
isprint() function yields TRUE for characters in the range 128-255, the use of |
| 207 |
|
|
[:isascii:] inside a character class defines a different set of characters, and |
| 208 |
|
|
this shows up in this test as a difference in the compiled code, which is being |
| 209 |
|
|
listed for checking. Where the comparison test output contains [\x00-\x7f] the |
| 210 |
|
|
test will contain [\x00-\xff], and similarly in some other cases. This is not a |
| 211 |
|
|
bug in PCRE. |
| 212 |
|
|
|
| 213 |
nigel |
63 |
The third set of tests checks pcre_maketables(), the facility for building a |
| 214 |
nigel |
25 |
set of character tables for a specific locale and using them instead of the |
| 215 |
|
|
default tables. The tests make use of the "fr" (French) locale. Before running |
| 216 |
|
|
the test, the script checks for the presence of this locale by running the |
| 217 |
|
|
"locale" command. If that command fails, or if it doesn't include "fr" in the |
| 218 |
nigel |
63 |
list of available locales, the third test cannot be run, and a comment is |
| 219 |
nigel |
25 |
output to say why. If running this test produces instances of the error |
| 220 |
|
|
|
| 221 |
|
|
** Failed to set locale "fr" |
| 222 |
|
|
|
| 223 |
|
|
in the comparison output, it means that locale is not available on your system, |
| 224 |
|
|
despite being listed by "locale". This does not mean that PCRE is broken. |
| 225 |
|
|
|
| 226 |
nigel |
63 |
The fourth test checks the UTF-8 support. It is not run automatically unless |
| 227 |
|
|
PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when |
| 228 |
|
|
running "configure". This file can be also fed directly to the perltest script, |
| 229 |
|
|
provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch, |
| 230 |
|
|
commented in the script, can be be used.) |
| 231 |
nigel |
3 |
|
| 232 |
nigel |
63 |
The fifth and final file tests error handling with UTF-8 encoding, and internal |
| 233 |
|
|
UTF-8 features of PCRE that are not relevant to Perl. |
| 234 |
nigel |
3 |
|
| 235 |
nigel |
63 |
|
| 236 |
nigel |
3 |
Character tables |
| 237 |
|
|
---------------- |
| 238 |
|
|
|
| 239 |
nigel |
25 |
PCRE uses four tables for manipulating and identifying characters. The final |
| 240 |
|
|
argument of the pcre_compile() function is a pointer to a block of memory |
| 241 |
nigel |
35 |
containing the concatenated tables. A call to pcre_maketables() can be used to |
| 242 |
|
|
generate a set of tables in the current locale. If the final argument for |
| 243 |
|
|
pcre_compile() is passed as NULL, a set of default tables that is built into |
| 244 |
|
|
the binary is used. |
| 245 |
nigel |
3 |
|
| 246 |
nigel |
25 |
The source file called chartables.c contains the default set of tables. This is |
| 247 |
nigel |
27 |
not supplied in the distribution, but is built by the program dftables |
| 248 |
|
|
(compiled from dftables.c), which uses the ANSI C character handling functions |
| 249 |
nigel |
25 |
such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table |
| 250 |
nigel |
35 |
sources. This means that the default C locale which is set for your system will |
| 251 |
|
|
control the contents of these default tables. You can change the default tables |
| 252 |
|
|
by editing chartables.c and then re-building PCRE. If you do this, you should |
| 253 |
|
|
probably also edit Makefile to ensure that the file doesn't ever get |
| 254 |
|
|
re-generated. |
| 255 |
nigel |
3 |
|
| 256 |
nigel |
25 |
The first two 256-byte tables provide lower casing and case flipping functions, |
| 257 |
|
|
respectively. The next table consists of three 32-byte bit maps which identify |
| 258 |
|
|
digits, "word" characters, and white space, respectively. These are used when |
| 259 |
|
|
building 32-byte bit maps that represent character classes. |
| 260 |
|
|
|
| 261 |
|
|
The final 256-byte table has bits indicating various character types, as |
| 262 |
nigel |
3 |
follows: |
| 263 |
|
|
|
| 264 |
|
|
1 white space character |
| 265 |
|
|
2 letter |
| 266 |
|
|
4 decimal digit |
| 267 |
|
|
8 hexadecimal digit |
| 268 |
|
|
16 alphanumeric or '_' |
| 269 |
|
|
128 regular expression metacharacter or binary zero |
| 270 |
|
|
|
| 271 |
|
|
You should not alter the set of characters that contain the 128 bit, as that |
| 272 |
|
|
will cause PCRE to malfunction. |
| 273 |
|
|
|
| 274 |
|
|
|
| 275 |
nigel |
41 |
Manifest |
| 276 |
|
|
-------- |
| 277 |
nigel |
3 |
|
| 278 |
nigel |
41 |
The distribution should contain the following files: |
| 279 |
nigel |
3 |
|
| 280 |
nigel |
41 |
(A) The actual source files of the PCRE library functions and their |
| 281 |
|
|
headers: |
| 282 |
nigel |
3 |
|
| 283 |
nigel |
41 |
dftables.c auxiliary program for building chartables.c |
| 284 |
|
|
get.c ) |
| 285 |
|
|
maketables.c ) |
| 286 |
|
|
study.c ) source of |
| 287 |
|
|
pcre.c ) the functions |
| 288 |
|
|
pcreposix.c ) |
| 289 |
nigel |
63 |
printint.c ) |
| 290 |
nigel |
43 |
pcre.in "source" for the header for the external API; pcre.h |
| 291 |
|
|
is built from this by "configure" |
| 292 |
nigel |
41 |
pcreposix.h header for the external POSIX wrapper API |
| 293 |
|
|
internal.h header for internal use |
| 294 |
|
|
config.in template for config.h, which is built by configure |
| 295 |
nigel |
3 |
|
| 296 |
nigel |
41 |
(B) Auxiliary files: |
| 297 |
nigel |
3 |
|
| 298 |
nigel |
41 |
AUTHORS information about the author of PCRE |
| 299 |
|
|
ChangeLog log of changes to the code |
| 300 |
|
|
INSTALL generic installation instructions |
| 301 |
|
|
LICENCE conditions for the use of PCRE |
| 302 |
nigel |
43 |
COPYING the same, using GNU's standard name |
| 303 |
nigel |
41 |
Makefile.in template for Unix Makefile, which is built by configure |
| 304 |
|
|
NEWS important changes in this release |
| 305 |
|
|
NON-UNIX-USE notes on building PCRE on non-Unix systems |
| 306 |
|
|
README this file |
| 307 |
nigel |
49 |
RunTest.in template for a Unix shell script for running tests |
| 308 |
nigel |
41 |
config.guess ) files used by libtool, |
| 309 |
|
|
config.sub ) used only when building a shared library |
| 310 |
|
|
configure a configuring shell script (built by autoconf) |
| 311 |
|
|
configure.in the autoconf input used to build configure |
| 312 |
|
|
doc/Tech.Notes notes on the encoding |
| 313 |
nigel |
63 |
doc/*.3 man page sources for the PCRE functions |
| 314 |
|
|
doc/*.1 man page sources for pcregrep and pcretest |
| 315 |
|
|
doc/html/* HTML documentation |
| 316 |
|
|
doc/pcre.txt plain text version of the man pages |
| 317 |
|
|
doc/pcretest.txt plain text documentation of test program |
| 318 |
|
|
doc/perltest.txt plain text documentation of Perl test program |
| 319 |
nigel |
41 |
install-sh a shell script for installing files |
| 320 |
nigel |
53 |
ltmain.sh file used to build a libtool script |
| 321 |
|
|
pcretest.c comprehensive test program |
| 322 |
|
|
pcredemo.c simple demonstration of coding calls to PCRE |
| 323 |
nigel |
41 |
perltest Perl test program |
| 324 |
nigel |
49 |
pcregrep.c source of a grep utility that uses PCRE |
| 325 |
nigel |
43 |
pcre-config.in source of script which retains PCRE information |
| 326 |
nigel |
63 |
testdata/testinput1 test data, compatible with Perl |
| 327 |
nigel |
41 |
testdata/testinput2 test data for error messages and non-Perl things |
| 328 |
nigel |
63 |
testdata/testinput3 test data for locale-specific tests |
| 329 |
|
|
testdata/testinput4 test data for UTF-8 tests compatible with Perl |
| 330 |
|
|
testdata/testinput5 test data for other UTF-8 tests |
| 331 |
nigel |
41 |
testdata/testoutput1 test results corresponding to testinput1 |
| 332 |
|
|
testdata/testoutput2 test results corresponding to testinput2 |
| 333 |
|
|
testdata/testoutput3 test results corresponding to testinput3 |
| 334 |
|
|
testdata/testoutput4 test results corresponding to testinput4 |
| 335 |
nigel |
49 |
testdata/testoutput5 test results corresponding to testinput5 |
| 336 |
nigel |
3 |
|
| 337 |
nigel |
41 |
(C) Auxiliary files for Win32 DLL |
| 338 |
nigel |
29 |
|
| 339 |
nigel |
41 |
dll.mk |
| 340 |
|
|
pcre.def |
| 341 |
nigel |
29 |
|
| 342 |
nigel |
63 |
(D) Auxiliary file for VPASCAL |
| 343 |
|
|
|
| 344 |
|
|
makevp.bat |
| 345 |
|
|
|
| 346 |
nigel |
3 |
Philip Hazel <ph10@cam.ac.uk> |
| 347 |
nigel |
63 |
February 2003 |
Parent Directory
| 
Revision Log