| 1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
| 2 |
------------------ |
------------------ |
| 3 |
|
|
| 4 |
|
Version 6.7 04-Jul-06 |
| 5 |
|
--------------------- |
| 6 |
|
|
| 7 |
|
1. In order to handle tests when input lines are enormously long, pcretest has |
| 8 |
|
been re-factored so that it automatically extends its buffers when |
| 9 |
|
necessary. The code is crude, but this _is_ just a test program. The |
| 10 |
|
default size has been increased from 32K to 50K. |
| 11 |
|
|
| 12 |
|
2. The code in pcre_study() was using the value of the re argument before |
| 13 |
|
testing it for NULL. (Of course, in any sensible call of the function, it |
| 14 |
|
won't be NULL.) |
| 15 |
|
|
| 16 |
|
3. The memmove() emulation function in pcre_internal.h, which is used on |
| 17 |
|
systems that lack both memmove() and bcopy() - that is, hardly ever - |
| 18 |
|
was missing a "static" storage class specifier. |
| 19 |
|
|
| 20 |
|
4. When UTF-8 mode was not set, PCRE looped when compiling certain patterns |
| 21 |
|
containing an extended class (one that cannot be represented by a bitmap |
| 22 |
|
because it contains high-valued characters or Unicode property items, e.g. |
| 23 |
|
[\pZ]). Almost always one would set UTF-8 mode when processing such a |
| 24 |
|
pattern, but PCRE should not loop if you do not (it no longer does). |
| 25 |
|
[Detail: two cases were found: (a) a repeated subpattern containing an |
| 26 |
|
extended class; (b) a recursive reference to a subpattern that followed a |
| 27 |
|
previous extended class. It wasn't skipping over the extended class |
| 28 |
|
correctly when UTF-8 mode was not set.] |
| 29 |
|
|
| 30 |
|
5. A negated single-character class was not being recognized as fixed-length |
| 31 |
|
in lookbehind assertions such as (?<=[^f]), leading to an incorrect |
| 32 |
|
compile error "lookbehind assertion is not fixed length". |
| 33 |
|
|
| 34 |
|
6. The RunPerlTest auxiliary script was showing an unexpected difference |
| 35 |
|
between PCRE and Perl for UTF-8 tests. It turns out that it is hard to |
| 36 |
|
write a Perl script that can interpret lines of an input file either as |
| 37 |
|
byte characters or as UTF-8, which is what "perltest" was being required to |
| 38 |
|
do for the non-UTF-8 and UTF-8 tests, respectively. Essentially what you |
| 39 |
|
can't do is switch easily at run time between having the "use utf8;" pragma |
| 40 |
|
or not. In the end, I fudged it by using the RunPerlTest script to insert |
| 41 |
|
"use utf8;" explicitly for the UTF-8 tests. |
| 42 |
|
|
| 43 |
|
7. In multiline (/m) mode, PCRE was matching ^ after a terminating newline at |
| 44 |
|
the end of the subject string, contrary to the documentation and to what |
| 45 |
|
Perl does. This was true of both matching functions. Now it matches only at |
| 46 |
|
the start of the subject and immediately after *internal* newlines. |
| 47 |
|
|
| 48 |
|
8. A call of pcre_fullinfo() from pcretest to get the option bits was passing |
| 49 |
|
a pointer to an int instead of a pointer to an unsigned long int. This |
| 50 |
|
caused problems on 64-bit systems. |
| 51 |
|
|
| 52 |
|
9. Applied a patch from the folks at Google to pcrecpp.cc, to fix "another |
| 53 |
|
instance of the 'standard' template library not being so standard". |
| 54 |
|
|
| 55 |
|
10. There was no check on the number of named subpatterns nor the maximum |
| 56 |
|
length of a subpattern name. The product of these values is used to compute |
| 57 |
|
the size of the memory block for a compiled pattern. By supplying a very |
| 58 |
|
long subpattern name and a large number of named subpatterns, the size |
| 59 |
|
computation could be caused to overflow. This is now prevented by limiting |
| 60 |
|
the length of names to 32 characters, and the number of named subpatterns |
| 61 |
|
to 10,000. |
| 62 |
|
|
| 63 |
|
11. Subpatterns that are repeated with specific counts have to be replicated in |
| 64 |
|
the compiled pattern. The size of memory for this was computed from the |
| 65 |
|
length of the subpattern and the repeat count. The latter is limited to |
| 66 |
|
65535, but there was no limit on the former, meaning that integer overflow |
| 67 |
|
could in principle occur. The compiled length of a repeated subpattern is |
| 68 |
|
now limited to 30,000 bytes in order to prevent this. |
| 69 |
|
|
| 70 |
|
12. Added the optional facility to have named substrings with the same name. |
| 71 |
|
|
| 72 |
|
13. Added the ability to use a named substring as a condition, using the |
| 73 |
|
Python syntax: (?(name)yes|no). This overloads (?(R)... and names that |
| 74 |
|
are numbers (not recommended). Forward references are permitted. |
| 75 |
|
|
| 76 |
|
14. Added forward references in named backreferences (if you see what I mean). |
| 77 |
|
|
| 78 |
|
15. In UTF-8 mode, with the PCRE_DOTALL option set, a quantified dot in the |
| 79 |
|
pattern could run off the end of the subject. For example, the pattern |
| 80 |
|
"(?s)(.{1,5})"8 did this with the subject "ab". |
| 81 |
|
|
| 82 |
|
16. If PCRE_DOTALL or PCRE_MULTILINE were set, pcre_dfa_exec() behaved as if |
| 83 |
|
PCRE_CASELESS was set when matching characters that were quantified with ? |
| 84 |
|
or *. |
| 85 |
|
|
| 86 |
|
17. A character class other than a single negated character that had a minimum |
| 87 |
|
but no maximum quantifier - for example [ab]{6,} - was not handled |
| 88 |
|
correctly by pce_dfa_exec(). It would match only one character. |
| 89 |
|
|
| 90 |
|
18. A valid (though odd) pattern that looked like a POSIX character |
| 91 |
|
class but used an invalid character after [ (for example [[,abc,]]) caused |
| 92 |
|
pcre_compile() to give the error "Failed: internal error: code overflow" or |
| 93 |
|
in some cases to crash with a glibc free() error. This could even happen if |
| 94 |
|
the pattern terminated after [[ but there just happened to be a sequence of |
| 95 |
|
letters, a binary zero, and a closing ] in the memory that followed. |
| 96 |
|
|
| 97 |
|
19. Perl's treatment of octal escapes in the range \400 to \777 has changed |
| 98 |
|
over the years. Originally (before any Unicode support), just the bottom 8 |
| 99 |
|
bits were taken. Thus, for example, \500 really meant \100. Nowadays the |
| 100 |
|
output from "man perlunicode" includes this: |
| 101 |
|
|
| 102 |
|
The regular expression compiler produces polymorphic opcodes. That |
| 103 |
|
is, the pattern adapts to the data and automatically switches to |
| 104 |
|
the Unicode character scheme when presented with Unicode data--or |
| 105 |
|
instead uses a traditional byte scheme when presented with byte |
| 106 |
|
data. |
| 107 |
|
|
| 108 |
|
Sadly, a wide octal escape does not cause a switch, and in a string with |
| 109 |
|
no other multibyte characters, these octal escapes are treated as before. |
| 110 |
|
Thus, in Perl, the pattern /\500/ actually matches \100 but the pattern |
| 111 |
|
/\500|\x{1ff}/ matches \500 or \777 because the whole thing is treated as a |
| 112 |
|
Unicode string. |
| 113 |
|
|
| 114 |
|
I have not perpetrated such confusion in PCRE. Up till now, it took just |
| 115 |
|
the bottom 8 bits, as in old Perl. I have now made octal escapes with |
| 116 |
|
values greater than \377 illegal in non-UTF-8 mode. In UTF-8 mode they |
| 117 |
|
translate to the appropriate multibyte character. |
| 118 |
|
|
| 119 |
|
29. Applied some refactoring to reduce the number of warnings from Microsoft |
| 120 |
|
and Borland compilers. This has included removing the fudge introduced |
| 121 |
|
seven years ago for the OS/2 compiler (see 2.02/2 below) because it caused |
| 122 |
|
a warning about an unused variable. |
| 123 |
|
|
| 124 |
|
21. PCRE has not included VT (character 0x0b) in the set of whitespace |
| 125 |
|
characters since release 4.0, because Perl (from release 5.004) does not. |
| 126 |
|
[Or at least, is documented not to: some releases seem to be in conflict |
| 127 |
|
with the documentation.] However, when a pattern was studied with |
| 128 |
|
pcre_study() and all its branches started with \s, PCRE still included VT |
| 129 |
|
as a possible starting character. Of course, this did no harm; it just |
| 130 |
|
caused an unnecessary match attempt. |
| 131 |
|
|
| 132 |
|
22. Removed a now-redundant internal flag bit that recorded the fact that case |
| 133 |
|
dependency changed within the pattern. This was once needed for "required |
| 134 |
|
byte" processing, but is no longer used. This recovers a now-scarce options |
| 135 |
|
bit. Also moved the least significant internal flag bit to the most- |
| 136 |
|
significant bit of the word, which was not previously used (hangover from |
| 137 |
|
the days when it was an int rather than a uint) to free up another bit for |
| 138 |
|
the future. |
| 139 |
|
|
| 140 |
|
23. Added support for CRLF line endings as well as CR and LF. As well as the |
| 141 |
|
default being selectable at build time, it can now be changed at runtime |
| 142 |
|
via the PCRE_NEWLINE_xxx flags. There are now options for pcregrep to |
| 143 |
|
specify that it is scanning data with non-default line endings. |
| 144 |
|
|
| 145 |
|
24. Changed the definition of CXXLINK to make it agree with the definition of |
| 146 |
|
LINK in the Makefile, by replacing LDFLAGS to CXXFLAGS. |
| 147 |
|
|
| 148 |
|
25. Applied Ian Taylor's patches to avoid using another stack frame for tail |
| 149 |
|
recursions. This makes a big different to stack usage for some patterns. |
| 150 |
|
|
| 151 |
|
26. If a subpattern containing a named recursion or subroutine reference such |
| 152 |
|
as (?P>B) was quantified, for example (xxx(?P>B)){3}, the calculation of |
| 153 |
|
the space required for the compiled pattern went wrong and gave too small a |
| 154 |
|
value. Depending on the environment, this could lead to "Failed: internal |
| 155 |
|
error: code overflow at offset 49" or "glibc detected double free or |
| 156 |
|
corruption" errors. |
| 157 |
|
|
| 158 |
|
27. Applied patches from Google (a) to support the new newline modes and (b) to |
| 159 |
|
advance over multibyte UTF-8 characters in GlobalReplace. |
| 160 |
|
|
| 161 |
|
28. Change free() to pcre_free() in pcredemo.c. Apparently this makes a |
| 162 |
|
difference for some implementation of PCRE in some Windows version. |
| 163 |
|
|
| 164 |
|
29. Added some extra testing facilities to pcretest: |
| 165 |
|
|
| 166 |
|
\q<number> in a data line sets the "match limit" value |
| 167 |
|
\Q<number> in a data line sets the "match recursion limt" value |
| 168 |
|
-S <number> sets the stack size, where <number> is in megabytes |
| 169 |
|
|
| 170 |
|
The -S option isn't available for Windows. |
| 171 |
|
|
| 172 |
|
|
| 173 |
|
Version 6.6 06-Feb-06 |
| 174 |
|
--------------------- |
| 175 |
|
|
| 176 |
|
1. Change 16(a) for 6.5 broke things, because PCRE_DATA_SCOPE was not defined |
| 177 |
|
in pcreposix.h. I have copied the definition from pcre.h. |
| 178 |
|
|
| 179 |
|
2. Change 25 for 6.5 broke compilation in a build directory out-of-tree |
| 180 |
|
because pcre.h is no longer a built file. |
| 181 |
|
|
| 182 |
|
3. Added Jeff Friedl's additional debugging patches to pcregrep. These are |
| 183 |
|
not normally included in the compiled code. |
| 184 |
|
|
| 185 |
|
|
| 186 |
|
Version 6.5 01-Feb-06 |
| 187 |
|
--------------------- |
| 188 |
|
|
| 189 |
|
1. When using the partial match feature with pcre_dfa_exec(), it was not |
| 190 |
|
anchoring the second and subsequent partial matches at the new starting |
| 191 |
|
point. This could lead to incorrect results. For example, with the pattern |
| 192 |
|
/1234/, partially matching against "123" and then "a4" gave a match. |
| 193 |
|
|
| 194 |
|
2. Changes to pcregrep: |
| 195 |
|
|
| 196 |
|
(a) All non-match returns from pcre_exec() were being treated as failures |
| 197 |
|
to match the line. Now, unless the error is PCRE_ERROR_NOMATCH, an |
| 198 |
|
error message is output. Some extra information is given for the |
| 199 |
|
PCRE_ERROR_MATCHLIMIT and PCRE_ERROR_RECURSIONLIMIT errors, which are |
| 200 |
|
probably the only errors that are likely to be caused by users (by |
| 201 |
|
specifying a regex that has nested indefinite repeats, for instance). |
| 202 |
|
If there are more than 20 of these errors, pcregrep is abandoned. |
| 203 |
|
|
| 204 |
|
(b) A binary zero was treated as data while matching, but terminated the |
| 205 |
|
output line if it was written out. This has been fixed: binary zeroes |
| 206 |
|
are now no different to any other data bytes. |
| 207 |
|
|
| 208 |
|
(c) Whichever of the LC_ALL or LC_CTYPE environment variables is set is |
| 209 |
|
used to set a locale for matching. The --locale=xxxx long option has |
| 210 |
|
been added (no short equivalent) to specify a locale explicitly on the |
| 211 |
|
pcregrep command, overriding the environment variables. |
| 212 |
|
|
| 213 |
|
(d) When -B was used with -n, some line numbers in the output were one less |
| 214 |
|
than they should have been. |
| 215 |
|
|
| 216 |
|
(e) Added the -o (--only-matching) option. |
| 217 |
|
|
| 218 |
|
(f) If -A or -C was used with -c (count only), some lines of context were |
| 219 |
|
accidentally printed for the final match. |
| 220 |
|
|
| 221 |
|
(g) Added the -H (--with-filename) option. |
| 222 |
|
|
| 223 |
|
(h) The combination of options -rh failed to suppress file names for files |
| 224 |
|
that were found from directory arguments. |
| 225 |
|
|
| 226 |
|
(i) Added the -D (--devices) and -d (--directories) options. |
| 227 |
|
|
| 228 |
|
(j) Added the -F (--fixed-strings) option. |
| 229 |
|
|
| 230 |
|
(k) Allow "-" to be used as a file name for -f as well as for a data file. |
| 231 |
|
|
| 232 |
|
(l) Added the --colo(u)r option. |
| 233 |
|
|
| 234 |
|
(m) Added Jeffrey Friedl's -S testing option, but within #ifdefs so that it |
| 235 |
|
is not present by default. |
| 236 |
|
|
| 237 |
|
3. A nasty bug was discovered in the handling of recursive patterns, that is, |
| 238 |
|
items such as (?R) or (?1), when the recursion could match a number of |
| 239 |
|
alternatives. If it matched one of the alternatives, but subsequently, |
| 240 |
|
outside the recursion, there was a failure, the code tried to back up into |
| 241 |
|
the recursion. However, because of the way PCRE is implemented, this is not |
| 242 |
|
possible, and the result was an incorrect result from the match. |
| 243 |
|
|
| 244 |
|
In order to prevent this happening, the specification of recursion has |
| 245 |
|
been changed so that all such subpatterns are automatically treated as |
| 246 |
|
atomic groups. Thus, for example, (?R) is treated as if it were (?>(?R)). |
| 247 |
|
|
| 248 |
|
4. I had overlooked the fact that, in some locales, there are characters for |
| 249 |
|
which isalpha() is true but neither isupper() nor islower() are true. In |
| 250 |
|
the fr_FR locale, for instance, the \xAA and \xBA characters (ordmasculine |
| 251 |
|
and ordfeminine) are like this. This affected the treatment of \w and \W |
| 252 |
|
when they appeared in character classes, but not when they appeared outside |
| 253 |
|
a character class. The bit map for "word" characters is now created |
| 254 |
|
separately from the results of isalnum() instead of just taking it from the |
| 255 |
|
upper, lower, and digit maps. (Plus the underscore character, of course.) |
| 256 |
|
|
| 257 |
|
5. The above bug also affected the handling of POSIX character classes such as |
| 258 |
|
[[:alpha:]] and [[:alnum:]]. These do not have their own bit maps in PCRE's |
| 259 |
|
permanent tables. Instead, the bit maps for such a class were previously |
| 260 |
|
created as the appropriate unions of the upper, lower, and digit bitmaps. |
| 261 |
|
Now they are created by subtraction from the [[:word:]] class, which has |
| 262 |
|
its own bitmap. |
| 263 |
|
|
| 264 |
|
6. The [[:blank:]] character class matches horizontal, but not vertical space. |
| 265 |
|
It is created by subtracting the vertical space characters (\x09, \x0a, |
| 266 |
|
\x0b, \x0c) from the [[:space:]] bitmap. Previously, however, the |
| 267 |
|
subtraction was done in the overall bitmap for a character class, meaning |
| 268 |
|
that a class such as [\x0c[:blank:]] was incorrect because \x0c would not |
| 269 |
|
be recognized. This bug has been fixed. |
| 270 |
|
|
| 271 |
|
7. Patches from the folks at Google: |
| 272 |
|
|
| 273 |
|
(a) pcrecpp.cc: "to handle a corner case that may or may not happen in |
| 274 |
|
real life, but is still worth protecting against". |
| 275 |
|
|
| 276 |
|
(b) pcrecpp.cc: "corrects a bug when negative radixes are used with |
| 277 |
|
regular expressions". |
| 278 |
|
|
| 279 |
|
(c) pcre_scanner.cc: avoid use of std::count() because not all systems |
| 280 |
|
have it. |
| 281 |
|
|
| 282 |
|
(d) Split off pcrecpparg.h from pcrecpp.h and had the former built by |
| 283 |
|
"configure" and the latter not, in order to fix a problem somebody had |
| 284 |
|
with compiling the Arg class on HP-UX. |
| 285 |
|
|
| 286 |
|
(e) Improve the error-handling of the C++ wrapper a little bit. |
| 287 |
|
|
| 288 |
|
(f) New tests for checking recursion limiting. |
| 289 |
|
|
| 290 |
|
8. The pcre_memmove() function, which is used only if the environment does not |
| 291 |
|
have a standard memmove() function (and is therefore rarely compiled), |
| 292 |
|
contained two bugs: (a) use of int instead of size_t, and (b) it was not |
| 293 |
|
returning a result (though PCRE never actually uses the result). |
| 294 |
|
|
| 295 |
|
9. In the POSIX regexec() interface, if nmatch is specified as a ridiculously |
| 296 |
|
large number - greater than INT_MAX/(3*sizeof(int)) - REG_ESPACE is |
| 297 |
|
returned instead of calling malloc() with an overflowing number that would |
| 298 |
|
most likely cause subsequent chaos. |
| 299 |
|
|
| 300 |
|
10. The debugging option of pcretest was not showing the NO_AUTO_CAPTURE flag. |
| 301 |
|
|
| 302 |
|
11. The POSIX flag REG_NOSUB is now supported. When a pattern that was compiled |
| 303 |
|
with this option is matched, the nmatch and pmatch options of regexec() are |
| 304 |
|
ignored. |
| 305 |
|
|
| 306 |
|
12. Added REG_UTF8 to the POSIX interface. This is not defined by POSIX, but is |
| 307 |
|
provided in case anyone wants to the the POSIX interface with UTF-8 |
| 308 |
|
strings. |
| 309 |
|
|
| 310 |
|
13. Added CXXLDFLAGS to the Makefile parameters to provide settings only on the |
| 311 |
|
C++ linking (needed for some HP-UX environments). |
| 312 |
|
|
| 313 |
|
14. Avoid compiler warnings in get_ucpname() when compiled without UCP support |
| 314 |
|
(unused parameter) and in the pcre_printint() function (omitted "default" |
| 315 |
|
switch label when the default is to do nothing). |
| 316 |
|
|
| 317 |
|
15. Added some code to make it possible, when PCRE is compiled as a C++ |
| 318 |
|
library, to replace subject pointers for pcre_exec() with a smart pointer |
| 319 |
|
class, thus making it possible to process discontinuous strings. |
| 320 |
|
|
| 321 |
|
16. The two macros PCRE_EXPORT and PCRE_DATA_SCOPE are confusing, and perform |
| 322 |
|
much the same function. They were added by different people who were trying |
| 323 |
|
to make PCRE easy to compile on non-Unix systems. It has been suggested |
| 324 |
|
that PCRE_EXPORT be abolished now that there is more automatic apparatus |
| 325 |
|
for compiling on Windows systems. I have therefore replaced it with |
| 326 |
|
PCRE_DATA_SCOPE. This is set automatically for Windows; if not set it |
| 327 |
|
defaults to "extern" for C or "extern C" for C++, which works fine on |
| 328 |
|
Unix-like systems. It is now possible to override the value of PCRE_DATA_ |
| 329 |
|
SCOPE with something explicit in config.h. In addition: |
| 330 |
|
|
| 331 |
|
(a) pcreposix.h still had just "extern" instead of either of these macros; |
| 332 |
|
I have replaced it with PCRE_DATA_SCOPE. |
| 333 |
|
|
| 334 |
|
(b) Functions such as _pcre_xclass(), which are internal to the library, |
| 335 |
|
but external in the C sense, all had PCRE_EXPORT in their definitions. |
| 336 |
|
This is apparently wrong for the Windows case, so I have removed it. |
| 337 |
|
(It makes no difference on Unix-like systems.) |
| 338 |
|
|
| 339 |
|
17. Added a new limit, MATCH_LIMIT_RECURSION, which limits the depth of nesting |
| 340 |
|
of recursive calls to match(). This is different to MATCH_LIMIT because |
| 341 |
|
that limits the total number of calls to match(), not all of which increase |
| 342 |
|
the depth of recursion. Limiting the recursion depth limits the amount of |
| 343 |
|
stack (or heap if NO_RECURSE is set) that is used. The default can be set |
| 344 |
|
when PCRE is compiled, and changed at run time. A patch from Google adds |
| 345 |
|
this functionality to the C++ interface. |
| 346 |
|
|
| 347 |
|
18. Changes to the handling of Unicode character properties: |
| 348 |
|
|
| 349 |
|
(a) Updated the table to Unicode 4.1.0. |
| 350 |
|
|
| 351 |
|
(b) Recognize characters that are not in the table as "Cn" (undefined). |
| 352 |
|
|
| 353 |
|
(c) I revised the way the table is implemented to a much improved format |
| 354 |
|
which includes recognition of ranges. It now supports the ranges that |
| 355 |
|
are defined in UnicodeData.txt, and it also amalgamates other |
| 356 |
|
characters into ranges. This has reduced the number of entries in the |
| 357 |
|
table from around 16,000 to around 3,000, thus reducing its size |
| 358 |
|
considerably. I realized I did not need to use a tree structure after |
| 359 |
|
all - a binary chop search is just as efficient. Having reduced the |
| 360 |
|
number of entries, I extended their size from 6 bytes to 8 bytes to |
| 361 |
|
allow for more data. |
| 362 |
|
|
| 363 |
|
(d) Added support for Unicode script names via properties such as \p{Han}. |
| 364 |
|
|
| 365 |
|
19. In UTF-8 mode, a backslash followed by a non-Ascii character was not |
| 366 |
|
matching that character. |
| 367 |
|
|
| 368 |
|
20. When matching a repeated Unicode property with a minimum greater than zero, |
| 369 |
|
(for example \pL{2,}), PCRE could look past the end of the subject if it |
| 370 |
|
reached it while seeking the minimum number of characters. This could |
| 371 |
|
happen only if some of the characters were more than one byte long, because |
| 372 |
|
there is a check for at least the minimum number of bytes. |
| 373 |
|
|
| 374 |
|
21. Refactored the implementation of \p and \P so as to be more general, to |
| 375 |
|
allow for more different types of property in future. This has changed the |
| 376 |
|
compiled form incompatibly. Anybody with saved compiled patterns that use |
| 377 |
|
\p or \P will have to recompile them. |
| 378 |
|
|
| 379 |
|
22. Added "Any" and "L&" to the supported property types. |
| 380 |
|
|
| 381 |
|
23. Recognize \x{...} as a code point specifier, even when not in UTF-8 mode, |
| 382 |
|
but give a compile time error if the value is greater than 0xff. |
| 383 |
|
|
| 384 |
|
24. The man pages for pcrepartial, pcreprecompile, and pcre_compile2 were |
| 385 |
|
accidentally not being installed or uninstalled. |
| 386 |
|
|
| 387 |
|
25. The pcre.h file was built from pcre.h.in, but the only changes that were |
| 388 |
|
made were to insert the current release number. This seemed silly, because |
| 389 |
|
it made things harder for people building PCRE on systems that don't run |
| 390 |
|
"configure". I have turned pcre.h into a distributed file, no longer built |
| 391 |
|
by "configure", with the version identification directly included. There is |
| 392 |
|
no longer a pcre.h.in file. |
| 393 |
|
|
| 394 |
|
However, this change necessitated a change to the pcre-config script as |
| 395 |
|
well. It is built from pcre-config.in, and one of the substitutions was the |
| 396 |
|
release number. I have updated configure.ac so that ./configure now finds |
| 397 |
|
the release number by grepping pcre.h. |
| 398 |
|
|
| 399 |
|
26. Added the ability to run the tests under valgrind. |
| 400 |
|
|
| 401 |
|
|
| 402 |
|
Version 6.4 05-Sep-05 |
| 403 |
|
--------------------- |
| 404 |
|
|
| 405 |
|
1. Change 6.0/10/(l) to pcregrep introduced a bug that caused separator lines |
| 406 |
|
"--" to be printed when multiple files were scanned, even when none of the |
| 407 |
|
-A, -B, or -C options were used. This is not compatible with Gnu grep, so I |
| 408 |
|
consider it to be a bug, and have restored the previous behaviour. |
| 409 |
|
|
| 410 |
|
2. A couple of code tidies to get rid of compiler warnings. |
| 411 |
|
|
| 412 |
|
3. The pcretest program used to cheat by referring to symbols in the library |
| 413 |
|
whose names begin with _pcre_. These are internal symbols that are not |
| 414 |
|
really supposed to be visible externally, and in some environments it is |
| 415 |
|
possible to suppress them. The cheating is now confined to including |
| 416 |
|
certain files from the library's source, which is a bit cleaner. |
| 417 |
|
|
| 418 |
|
4. Renamed pcre.in as pcre.h.in to go with pcrecpp.h.in; it also makes the |
| 419 |
|
file's purpose clearer. |
| 420 |
|
|
| 421 |
|
5. Reorganized pcre_ucp_findchar(). |
| 422 |
|
|
| 423 |
|
|
| 424 |
|
Version 6.3 15-Aug-05 |
| 425 |
|
--------------------- |
| 426 |
|
|
| 427 |
|
1. The file libpcre.pc.in did not have general read permission in the tarball. |
| 428 |
|
|
| 429 |
|
2. There were some problems when building without C++ support: |
| 430 |
|
|
| 431 |
|
(a) If C++ support was not built, "make install" and "make test" still |
| 432 |
|
tried to test it. |
| 433 |
|
|
| 434 |
|
(b) There were problems when the value of CXX was explicitly set. Some |
| 435 |
|
changes have been made to try to fix these, and ... |
| 436 |
|
|
| 437 |
|
(c) --disable-cpp can now be used to explicitly disable C++ support. |
| 438 |
|
|
| 439 |
|
(d) The use of @CPP_OBJ@ directly caused a blank line preceded by a |
| 440 |
|
backslash in a target when C++ was disabled. This confuses some |
| 441 |
|
versions of "make", apparently. Using an intermediate variable solves |
| 442 |
|
this. (Same for CPP_LOBJ.) |
| 443 |
|
|
| 444 |
|
3. $(LINK_FOR_BUILD) now includes $(CFLAGS_FOR_BUILD) and $(LINK) |
| 445 |
|
(non-Windows) now includes $(CFLAGS) because these flags are sometimes |
| 446 |
|
necessary on certain architectures. |
| 447 |
|
|
| 448 |
|
4. Added a setting of -export-symbols-regex to the link command to remove |
| 449 |
|
those symbols that are exported in the C sense, but actually are local |
| 450 |
|
within the library, and not documented. Their names all begin with |
| 451 |
|
"_pcre_". This is not a perfect job, because (a) we have to except some |
| 452 |
|
symbols that pcretest ("illegally") uses, and (b) the facility isn't always |
| 453 |
|
available (and never for static libraries). I have made a note to try to |
| 454 |
|
find a way round (a) in the future. |
| 455 |
|
|
| 456 |
|
|
| 457 |
|
Version 6.2 01-Aug-05 |
| 458 |
|
--------------------- |
| 459 |
|
|
| 460 |
|
1. There was no test for integer overflow of quantifier values. A construction |
| 461 |
|
such as {1111111111111111} would give undefined results. What is worse, if |
| 462 |
|
a minimum quantifier for a parenthesized subpattern overflowed and became |
| 463 |
|
negative, the calculation of the memory size went wrong. This could have |
| 464 |
|
led to memory overwriting. |
| 465 |
|
|
| 466 |
|
2. Building PCRE using VPATH was broken. Hopefully it is now fixed. |
| 467 |
|
|
| 468 |
|
3. Added "b" to the 2nd argument of fopen() in dftables.c, for non-Unix-like |
| 469 |
|
operating environments where this matters. |
| 470 |
|
|
| 471 |
|
4. Applied Giuseppe Maxia's patch to add additional features for controlling |
| 472 |
|
PCRE options from within the C++ wrapper. |
| 473 |
|
|
| 474 |
|
5. Named capturing subpatterns were not being correctly counted when a pattern |
| 475 |
|
was compiled. This caused two problems: (a) If there were more than 100 |
| 476 |
|
such subpatterns, the calculation of the memory needed for the whole |
| 477 |
|
compiled pattern went wrong, leading to an overflow error. (b) Numerical |
| 478 |
|
back references of the form \12, where the number was greater than 9, were |
| 479 |
|
not recognized as back references, even though there were sufficient |
| 480 |
|
previous subpatterns. |
| 481 |
|
|
| 482 |
|
6. Two minor patches to pcrecpp.cc in order to allow it to compile on older |
| 483 |
|
versions of gcc, e.g. 2.95.4. |
| 484 |
|
|
| 485 |
|
|
| 486 |
|
Version 6.1 21-Jun-05 |
| 487 |
|
--------------------- |
| 488 |
|
|
| 489 |
|
1. There was one reference to the variable "posix" in pcretest.c that was not |
| 490 |
|
surrounded by "#if !defined NOPOSIX". |
| 491 |
|
|
| 492 |
|
2. Make it possible to compile pcretest without DFA support, UTF8 support, or |
| 493 |
|
the cross-check on the old pcre_info() function, for the benefit of the |
| 494 |
|
cut-down version of PCRE that is currently imported into Exim. |
| 495 |
|
|
| 496 |
|
3. A (silly) pattern starting with (?i)(?-i) caused an internal space |
| 497 |
|
allocation error. I've done the easy fix, which wastes 2 bytes for sensible |
| 498 |
|
patterns that start (?i) but I don't think that matters. The use of (?i) is |
| 499 |
|
just an example; this all applies to the other options as well. |
| 500 |
|
|
| 501 |
|
4. Since libtool seems to echo the compile commands it is issuing, the output |
| 502 |
|
from "make" can be reduced a bit by putting "@" in front of each libtool |
| 503 |
|
compile command. |
| 504 |
|
|
| 505 |
|
5. Patch from the folks at Google for configure.in to be a bit more thorough |
| 506 |
|
in checking for a suitable C++ installation before trying to compile the |
| 507 |
|
C++ stuff. This should fix a reported problem when a compiler was present, |
| 508 |
|
but no suitable headers. |
| 509 |
|
|
| 510 |
|
6. The man pages all had just "PCRE" as their title. I have changed them to |
| 511 |
|
be the relevant file name. I have also arranged that these names are |
| 512 |
|
retained in the file doc/pcre.txt, which is a concatenation in text format |
| 513 |
|
of all the man pages except the little individual ones for each function. |
| 514 |
|
|
| 515 |
|
7. The NON-UNIX-USE file had not been updated for the different set of source |
| 516 |
|
files that come with release 6. I also added a few comments about the C++ |
| 517 |
|
wrapper. |
| 518 |
|
|
| 519 |
|
|
| 520 |
|
Version 6.0 07-Jun-05 |
| 521 |
|
--------------------- |
| 522 |
|
|
| 523 |
|
1. Some minor internal re-organization to help with my DFA experiments. |
| 524 |
|
|
| 525 |
|
2. Some missing #ifdef SUPPORT_UCP conditionals in pcretest and printint that |
| 526 |
|
didn't matter for the library itself when fully configured, but did matter |
| 527 |
|
when compiling without UCP support, or within Exim, where the ucp files are |
| 528 |
|
not imported. |
| 529 |
|
|
| 530 |
|
3. Refactoring of the library code to split up the various functions into |
| 531 |
|
different source modules. The addition of the new DFA matching code (see |
| 532 |
|
below) to a single monolithic source would have made it really too |
| 533 |
|
unwieldy, quite apart from causing all the code to be include in a |
| 534 |
|
statically linked application, when only some functions are used. This is |
| 535 |
|
relevant even without the DFA addition now that patterns can be compiled in |
| 536 |
|
one application and matched in another. |
| 537 |
|
|
| 538 |
|
The downside of splitting up is that there have to be some external |
| 539 |
|
functions and data tables that are used internally in different modules of |
| 540 |
|
the library but which are not part of the API. These have all had their |
| 541 |
|
names changed to start with "_pcre_" so that they are unlikely to clash |
| 542 |
|
with other external names. |
| 543 |
|
|
| 544 |
|
4. Added an alternate matching function, pcre_dfa_exec(), which matches using |
| 545 |
|
a different (DFA) algorithm. Although it is slower than the original |
| 546 |
|
function, it does have some advantages for certain types of matching |
| 547 |
|
problem. |
| 548 |
|
|
| 549 |
|
5. Upgrades to pcretest in order to test the features of pcre_dfa_exec(), |
| 550 |
|
including restarting after a partial match. |
| 551 |
|
|
| 552 |
|
6. A patch for pcregrep that defines INVALID_FILE_ATTRIBUTES if it is not |
| 553 |
|
defined when compiling for Windows was sent to me. I have put it into the |
| 554 |
|
code, though I have no means of testing or verifying it. |
| 555 |
|
|
| 556 |
|
7. Added the pcre_refcount() auxiliary function. |
| 557 |
|
|
| 558 |
|
8. Added the PCRE_FIRSTLINE option. This constrains an unanchored pattern to |
| 559 |
|
match before or at the first newline in the subject string. In pcretest, |
| 560 |
|
the /f option on a pattern can be used to set this. |
| 561 |
|
|
| 562 |
|
9. A repeated \w when used in UTF-8 mode with characters greater than 256 |
| 563 |
|
would behave wrongly. This has been present in PCRE since release 4.0. |
| 564 |
|
|
| 565 |
|
10. A number of changes to the pcregrep command: |
| 566 |
|
|
| 567 |
|
(a) Refactored how -x works; insert ^(...)$ instead of setting |
| 568 |
|
PCRE_ANCHORED and checking the length, in preparation for adding |
| 569 |
|
something similar for -w. |
| 570 |
|
|
| 571 |
|
(b) Added the -w (match as a word) option. |
| 572 |
|
|
| 573 |
|
(c) Refactored the way lines are read and buffered so as to have more |
| 574 |
|
than one at a time available. |
| 575 |
|
|
| 576 |
|
(d) Implemented a pcregrep test script. |
| 577 |
|
|
| 578 |
|
(e) Added the -M (multiline match) option. This allows patterns to match |
| 579 |
|
over several lines of the subject. The buffering ensures that at least |
| 580 |
|
8K, or the rest of the document (whichever is the shorter) is available |
| 581 |
|
for matching (and similarly the previous 8K for lookbehind assertions). |
| 582 |
|
|
| 583 |
|
(f) Changed the --help output so that it now says |
| 584 |
|
|
| 585 |
|
-w, --word-regex(p) |
| 586 |
|
|
| 587 |
|
instead of two lines, one with "regex" and the other with "regexp" |
| 588 |
|
because that confused at least one person since the short forms are the |
| 589 |
|
same. (This required a bit of code, as the output is generated |
| 590 |
|
automatically from a table. It wasn't just a text change.) |
| 591 |
|
|
| 592 |
|
(g) -- can be used to terminate pcregrep options if the next thing isn't an |
| 593 |
|
option but starts with a hyphen. Could be a pattern or a path name |
| 594 |
|
starting with a hyphen, for instance. |
| 595 |
|
|
| 596 |
|
(h) "-" can be given as a file name to represent stdin. |
| 597 |
|
|
| 598 |
|
(i) When file names are being printed, "(standard input)" is used for |
| 599 |
|
the standard input, for compatibility with GNU grep. Previously |
| 600 |
|
"<stdin>" was used. |
| 601 |
|
|
| 602 |
|
(j) The option --label=xxx can be used to supply a name to be used for |
| 603 |
|
stdin when file names are being printed. There is no short form. |
| 604 |
|
|
| 605 |
|
(k) Re-factored the options decoding logic because we are going to add |
| 606 |
|
two more options that take data. Such options can now be given in four |
| 607 |
|
different ways, e.g. "-fname", "-f name", "--file=name", "--file name". |
| 608 |
|
|
| 609 |
|
(l) Added the -A, -B, and -C options for requesting that lines of context |
| 610 |
|
around matches be printed. |
| 611 |
|
|
| 612 |
|
(m) Added the -L option to print the names of files that do not contain |
| 613 |
|
any matching lines, that is, the complement of -l. |
| 614 |
|
|
| 615 |
|
(n) The return code is 2 if any file cannot be opened, but pcregrep does |
| 616 |
|
continue to scan other files. |
| 617 |
|
|
| 618 |
|
(o) The -s option was incorrectly implemented. For compatibility with other |
| 619 |
|
greps, it now suppresses the error message for a non-existent or non- |
| 620 |
|
accessible file (but not the return code). There is a new option called |
| 621 |
|
-q that suppresses the output of matching lines, which was what -s was |
| 622 |
|
previously doing. |
| 623 |
|
|
| 624 |
|
(p) Added --include and --exclude options to specify files for inclusion |
| 625 |
|
and exclusion when recursing. |
| 626 |
|
|
| 627 |
|
11. The Makefile was not using the Autoconf-supported LDFLAGS macro properly. |
| 628 |
|
Hopefully, it now does. |
| 629 |
|
|
| 630 |
|
12. Missing cast in pcre_study(). |
| 631 |
|
|
| 632 |
|
13. Added an "uninstall" target to the makefile. |
| 633 |
|
|
| 634 |
|
14. Replaced "extern" in the function prototypes in Makefile.in with |
| 635 |
|
"PCRE_DATA_SCOPE", which defaults to 'extern' or 'extern "C"' in the Unix |
| 636 |
|
world, but is set differently for Windows. |
| 637 |
|
|
| 638 |
|
15. Added a second compiling function called pcre_compile2(). The only |
| 639 |
|
difference is that it has an extra argument, which is a pointer to an |
| 640 |
|
integer error code. When there is a compile-time failure, this is set |
| 641 |
|
non-zero, in addition to the error test pointer being set to point to an |
| 642 |
|
error message. The new argument may be NULL if no error number is required |
| 643 |
|
(but then you may as well call pcre_compile(), which is now just a |
| 644 |
|
wrapper). This facility is provided because some applications need a |
| 645 |
|
numeric error indication, but it has also enabled me to tidy up the way |
| 646 |
|
compile-time errors are handled in the POSIX wrapper. |
| 647 |
|
|
| 648 |
|
16. Added VPATH=.libs to the makefile; this should help when building with one |
| 649 |
|
prefix path and installing with another. (Or so I'm told by someone who |
| 650 |
|
knows more about this stuff than I do.) |
| 651 |
|
|
| 652 |
|
17. Added a new option, REG_DOTALL, to the POSIX function regcomp(). This |
| 653 |
|
passes PCRE_DOTALL to the pcre_compile() function, making the "." character |
| 654 |
|
match everything, including newlines. This is not POSIX-compatible, but |
| 655 |
|
somebody wanted the feature. From pcretest it can be activated by using |
| 656 |
|
both the P and the s flags. |
| 657 |
|
|
| 658 |
|
18. AC_PROG_LIBTOOL appeared twice in Makefile.in. Removed one. |
| 659 |
|
|
| 660 |
|
19. libpcre.pc was being incorrectly installed as executable. |
| 661 |
|
|
| 662 |
|
20. A couple of places in pcretest check for end-of-line by looking for '\n'; |
| 663 |
|
it now also looks for '\r' so that it will work unmodified on Windows. |
| 664 |
|
|
| 665 |
|
21. Added Google's contributed C++ wrapper to the distribution. |
| 666 |
|
|
| 667 |
|
22. Added some untidy missing memory free() calls in pcretest, to keep |
| 668 |
|
Electric Fence happy when testing. |
| 669 |
|
|
| 670 |
|
|
| 671 |
|
|
| 672 |
Version 5.0 13-Sep-04 |
Version 5.0 13-Sep-04 |
| 673 |
--------------------- |
--------------------- |
| 674 |
|
|