| 1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
| 2 |
------------------ |
------------------ |
| 3 |
|
|
| 4 |
Version 8.01 11-Dec-09 |
Version 8.13 30-Apr-2011 |
| 5 |
---------------------- |
------------------------ |
| 6 |
|
|
| 7 |
|
1. The Unicode data tables have been updated to Unicode 6.0.0. |
| 8 |
|
|
| 9 |
|
2. Two minor typos in pcre_internal.h have been fixed. |
| 10 |
|
|
| 11 |
|
3. Added #include <string.h> to pcre_scanner_unittest.cc, pcrecpp.cc, and |
| 12 |
|
pcrecpp_unittest.cc. They are needed for strcmp(), memset(), and strchr() |
| 13 |
|
in some environments (e.g. Solaris 10/SPARC using Sun Studio 12U2). |
| 14 |
|
|
| 15 |
|
4. There were a number of related bugs in the code for matching backrefences |
| 16 |
|
caselessly in UTF-8 mode when codes for the characters concerned were |
| 17 |
|
different numbers of bytes. For example, U+023A and U+2C65 are an upper |
| 18 |
|
and lower case pair, using 2 and 3 bytes, respectively. The main bugs were: |
| 19 |
|
(a) A reference to 3 copies of a 2-byte code matched only 2 of a 3-byte |
| 20 |
|
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
| 21 |
|
2-byte code at the end of the subject (it thought there wasn't enough data |
| 22 |
|
left). |
| 23 |
|
|
| 24 |
|
5. Comprehensive information about what went wrong is now returned by |
| 25 |
|
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
| 26 |
|
as the output vector has at least 2 elements. The offset of the start of |
| 27 |
|
the failing character and a reason code are placed in the vector. |
| 28 |
|
|
| 29 |
|
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
| 30 |
|
now returned is for the first byte of the failing character, instead of the |
| 31 |
|
last byte inspected. This is an incompatible change, but I hope it is small |
| 32 |
|
enough not to be a problem. It makes the returned offset consistent with |
| 33 |
|
pcre_exec() and pcre_dfa_exec(). |
| 34 |
|
|
| 35 |
|
7. pcretest now gives a text phrase as well as the error number when |
| 36 |
|
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
| 37 |
|
failure, the offset and reason code are output. |
| 38 |
|
|
| 39 |
|
8. When \R was used with a maximizing quantifier it failed to skip backwards |
| 40 |
|
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
| 41 |
|
back over a single character (\n). This seems wrong (because it treated the |
| 42 |
|
two characters as a single entity when going forwards), conflicts with the |
| 43 |
|
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
| 44 |
|
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
| 45 |
|
has been changed. |
| 46 |
|
|
| 47 |
|
9. Some internal refactoring has changed the processing so that the handling |
| 48 |
|
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
| 49 |
|
time (the PCRE_DOTALL option was changed this way some time ago: version |
| 50 |
|
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
| 51 |
|
which was always a bit of a fudge. It also means that there is one less |
| 52 |
|
argument for the match() function, which reduces its stack requirements |
| 53 |
|
slightly. This change also fixes an incompatibility with Perl: the pattern |
| 54 |
|
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
| 55 |
|
|
| 56 |
|
10. More internal refactoring has drastically reduced the number of recursive |
| 57 |
|
calls to match() for possessively repeated groups such as (abc)++ when |
| 58 |
|
using pcre_exec(). |
| 59 |
|
|
| 60 |
|
11. While implementing 10, a number of bugs in the handling of groups were |
| 61 |
|
discovered and fixed: |
| 62 |
|
|
| 63 |
|
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
| 64 |
|
(a|)*(?1) gave a compile-time internal error. |
| 65 |
|
((a|)+)+ did not notice that the outer group could match an empty string. |
| 66 |
|
(^a|^)+ was not marked as anchored. |
| 67 |
|
(.*a|.*)+ was not marked as matching at start or after a newline. |
| 68 |
|
|
| 69 |
|
12. Yet more internal refactoring has removed another argument from the match() |
| 70 |
|
function. Special calls to this function are now indicated by setting a |
| 71 |
|
value in a variable in the "match data" data block. |
| 72 |
|
|
| 73 |
|
13. Be more explicit in pcre_study() instead of relying on "default" for |
| 74 |
|
opcodes that mean there is no starting character; this means that when new |
| 75 |
|
ones are added and accidentally left out of pcre_study(), testing should |
| 76 |
|
pick them up. |
| 77 |
|
|
| 78 |
|
14. The -s option of pcretest has been documented for ages as being an old |
| 79 |
|
synonym of -m (show memory usage). I have changed it to mean "force study |
| 80 |
|
for every regex", that is, assume /S for every regex. This is similar to -i |
| 81 |
|
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
| 82 |
|
using it. It makes it easier to run collections of tests with and without |
| 83 |
|
study enabled, and thereby test pcre_study() more easily. All the standard |
| 84 |
|
tests are now run with and without -s (but some patterns can be marked as |
| 85 |
|
"never study" - see 20 below). |
| 86 |
|
|
| 87 |
|
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
| 88 |
|
restoration of the capturing data to the outer values was not happening |
| 89 |
|
correctly. |
| 90 |
|
|
| 91 |
|
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
| 92 |
|
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
| 93 |
|
pattern had matched an empty string, and so incorrectly returned a no |
| 94 |
|
match. |
| 95 |
|
|
| 96 |
|
17. There was optimizing code for the last branch of non-capturing parentheses, |
| 97 |
|
and also for the obeyed branch of a conditional subexpression, which used |
| 98 |
|
tail recursion to cut down on stack usage. Unfortunately, not that there is |
| 99 |
|
the possibility of (*THEN) occurring in these branches, tail recursion is |
| 100 |
|
no longer possible because the return has to be checked for (*THEN). These |
| 101 |
|
two optimizations have therefore been removed. |
| 102 |
|
|
| 103 |
|
18. If a pattern containing \R was studied, it was assumed that \R always |
| 104 |
|
matched two bytes, thus causing the minimum subject length to be |
| 105 |
|
incorrectly computed because \R can also match just one byte. |
| 106 |
|
|
| 107 |
|
19. If a pattern containing (*ACCEPT) was studied, the minimum subject length |
| 108 |
|
was incorrectly computed. |
| 109 |
|
|
| 110 |
|
20. If /S is present twice on a test pattern in pcretest input, it *disables* |
| 111 |
|
studying, thereby overriding the use of -s on the command line. This is |
| 112 |
|
necessary for one or two tests to keep the output identical in both cases. |
| 113 |
|
|
| 114 |
|
21. When (*ACCEPT) was used in an assertion that matched an empty string and |
| 115 |
|
PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. |
| 116 |
|
|
| 117 |
|
22. When an atomic group that contained a capturing parenthesis was |
| 118 |
|
successfully matched, but the branch in which it appeared failed, the |
| 119 |
|
capturing was not being forgotten if a higher numbered group was later |
| 120 |
|
captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing |
| 121 |
|
group 1 to "a", when in fact it should be unset. This applied to multi- |
| 122 |
|
branched capturing and non-capturing groups, repeated or not, and also to |
| 123 |
|
positive assertions (capturing in negative assertions is not well defined |
| 124 |
|
in PCRE) and also to nested atomic groups. |
| 125 |
|
|
| 126 |
|
23. Add the ++ qualifier feature to pcretest, to show the remainder of the |
| 127 |
|
subject after a captured substring (to make it easier to tell which of a |
| 128 |
|
number of identical substrings has been captured). |
| 129 |
|
|
| 130 |
|
24. The way atomic groups are processed by pcre_exec() has been changed so that |
| 131 |
|
if they are repeated, backtracking one repetition now resets captured |
| 132 |
|
values correctly. For example, if ((?>(a+)b)+aabab) is matched against |
| 133 |
|
"aaaabaaabaabab" the value of captured group 2 is now correctly recorded as |
| 134 |
|
"aaa". Previously, it would have been "a". As part of this code |
| 135 |
|
refactoring, the way recursive calls are handled has also been changed. |
| 136 |
|
|
| 137 |
|
24. If an assertion condition captured any substrings, they were not passed |
| 138 |
|
back unless some other capturing happened later. For example, if |
| 139 |
|
(?(?=(a))a) was matched against "a", no capturing was returned. |
| 140 |
|
|
| 141 |
|
25. When studying a pattern that contained subroutine calls or assertions, |
| 142 |
|
the code for finding the minimum length of a possible match was handling |
| 143 |
|
direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where |
| 144 |
|
group 1 called group 2 while simultaneously a separate group 2 called group |
| 145 |
|
1). A stack overflow occurred in this case. I have fixed this by limiting |
| 146 |
|
the recursion depth to 10. |
| 147 |
|
|
| 148 |
|
26. Updated RunTest.bat in the distribution to the version supplied by Tom |
| 149 |
|
Fortmann. This supports explicit test numbers on the command line, and has |
| 150 |
|
argument validation and error reporting. |
| 151 |
|
|
| 152 |
|
27. An instance of \X with an unlimited repeat could fail if at any point the |
| 153 |
|
first character it looked at was a mark character. |
| 154 |
|
|
| 155 |
|
28. Some minor code refactoring concerning Unicode properties and scripts |
| 156 |
|
should reduce the stack requirement of match() slightly. |
| 157 |
|
|
| 158 |
|
29. Added the '=' option to pcretest to check the setting of unused capturing |
| 159 |
|
slots at the end of the pattern, which are documented as being -1, but are |
| 160 |
|
not included in the return count. |
| 161 |
|
|
| 162 |
|
30. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE |
| 163 |
|
compiled something random. Now it gives a compile-time error (as does |
| 164 |
|
Perl). |
| 165 |
|
|
| 166 |
|
31. A *MARK encountered during the processing of a positive assertion is now |
| 167 |
|
recorded and passed back (compatible with Perl). |
| 168 |
|
|
| 169 |
|
|
| 170 |
|
Version 8.12 15-Jan-2011 |
| 171 |
|
------------------------ |
| 172 |
|
|
| 173 |
|
1. Fixed some typos in the markup of the man pages, and wrote a script that |
| 174 |
|
checks for such things as part of the documentation building process. |
| 175 |
|
|
| 176 |
|
2. On a big-endian 64-bit system, pcregrep did not correctly process the |
| 177 |
|
--match-limit and --recursion-limit options (added for 8.11). In |
| 178 |
|
particular, this made one of the standard tests fail. (The integer value |
| 179 |
|
went into the wrong half of a long int.) |
| 180 |
|
|
| 181 |
|
3. If the --colour option was given to pcregrep with -v (invert match), it |
| 182 |
|
did strange things, either producing crazy output, or crashing. It should, |
| 183 |
|
of course, ignore a request for colour when reporting lines that do not |
| 184 |
|
match. |
| 185 |
|
|
| 186 |
|
4. Another pcregrep bug caused similar problems if --colour was specified with |
| 187 |
|
-M (multiline) and the pattern match finished with a line ending. |
| 188 |
|
|
| 189 |
|
5. In pcregrep, when a pattern that ended with a literal newline sequence was |
| 190 |
|
matched in multiline mode, the following line was shown as part of the |
| 191 |
|
match. This seems wrong, so I have changed it. |
| 192 |
|
|
| 193 |
|
6. Another pcregrep bug in multiline mode, when --colour was specified, caused |
| 194 |
|
the check for further matches in the same line (so they could be coloured) |
| 195 |
|
to overrun the end of the current line. If another match was found, it was |
| 196 |
|
incorrectly shown (and then shown again when found in the next line). |
| 197 |
|
|
| 198 |
|
7. If pcregrep was compiled under Windows, there was a reference to the |
| 199 |
|
function pcregrep_exit() before it was defined. I am assuming this was |
| 200 |
|
the cause of the "error C2371: 'pcregrep_exit' : redefinition;" that was |
| 201 |
|
reported by a user. I've moved the definition above the reference. |
| 202 |
|
|
| 203 |
|
|
| 204 |
|
Version 8.11 10-Dec-2010 |
| 205 |
|
------------------------ |
| 206 |
|
|
| 207 |
|
1. (*THEN) was not working properly if there were untried alternatives prior |
| 208 |
|
to it in the current branch. For example, in ((a|b)(*THEN)(*F)|c..) it |
| 209 |
|
backtracked to try for "b" instead of moving to the next alternative branch |
| 210 |
|
at the same level (in this case, to look for "c"). The Perl documentation |
| 211 |
|
is clear that when (*THEN) is backtracked onto, it goes to the "next |
| 212 |
|
alternative in the innermost enclosing group". |
| 213 |
|
|
| 214 |
|
2. (*COMMIT) was not overriding (*THEN), as it does in Perl. In a pattern |
| 215 |
|
such as (A(*COMMIT)B(*THEN)C|D) any failure after matching A should |
| 216 |
|
result in overall failure. Similarly, (*COMMIT) now overrides (*PRUNE) and |
| 217 |
|
(*SKIP), (*SKIP) overrides (*PRUNE) and (*THEN), and (*PRUNE) overrides |
| 218 |
|
(*THEN). |
| 219 |
|
|
| 220 |
|
3. If \s appeared in a character class, it removed the VT character from |
| 221 |
|
the class, even if it had been included by some previous item, for example |
| 222 |
|
in [\x00-\xff\s]. (This was a bug related to the fact that VT is not part |
| 223 |
|
of \s, but is part of the POSIX "space" class.) |
| 224 |
|
|
| 225 |
|
4. A partial match never returns an empty string (because you can always |
| 226 |
|
match an empty string at the end of the subject); however the checking for |
| 227 |
|
an empty string was starting at the "start of match" point. This has been |
| 228 |
|
changed to the "earliest inspected character" point, because the returned |
| 229 |
|
data for a partial match starts at this character. This means that, for |
| 230 |
|
example, /(?<=abc)def/ gives a partial match for the subject "abc" |
| 231 |
|
(previously it gave "no match"). |
| 232 |
|
|
| 233 |
|
5. Changes have been made to the way PCRE_PARTIAL_HARD affects the matching |
| 234 |
|
of $, \z, \Z, \b, and \B. If the match point is at the end of the string, |
| 235 |
|
previously a full match would be given. However, setting PCRE_PARTIAL_HARD |
| 236 |
|
has an implication that the given string is incomplete (because a partial |
| 237 |
|
match is preferred over a full match). For this reason, these items now |
| 238 |
|
give a partial match in this situation. [Aside: previously, the one case |
| 239 |
|
/t\b/ matched against "cat" with PCRE_PARTIAL_HARD set did return a partial |
| 240 |
|
match rather than a full match, which was wrong by the old rules, but is |
| 241 |
|
now correct.] |
| 242 |
|
|
| 243 |
|
6. There was a bug in the handling of #-introduced comments, recognized when |
| 244 |
|
PCRE_EXTENDED is set, when PCRE_NEWLINE_ANY and PCRE_UTF8 were also set. |
| 245 |
|
If a UTF-8 multi-byte character included the byte 0x85 (e.g. +U0445, whose |
| 246 |
|
UTF-8 encoding is 0xd1,0x85), this was misinterpreted as a newline when |
| 247 |
|
scanning for the end of the comment. (*Character* 0x85 is an "any" newline, |
| 248 |
|
but *byte* 0x85 is not, in UTF-8 mode). This bug was present in several |
| 249 |
|
places in pcre_compile(). |
| 250 |
|
|
| 251 |
|
7. Related to (6) above, when pcre_compile() was skipping #-introduced |
| 252 |
|
comments when looking ahead for named forward references to subpatterns, |
| 253 |
|
the only newline sequence it recognized was NL. It now handles newlines |
| 254 |
|
according to the set newline convention. |
| 255 |
|
|
| 256 |
|
8. SunOS4 doesn't have strerror() or strtoul(); pcregrep dealt with the |
| 257 |
|
former, but used strtoul(), whereas pcretest avoided strtoul() but did not |
| 258 |
|
cater for a lack of strerror(). These oversights have been fixed. |
| 259 |
|
|
| 260 |
|
9. Added --match-limit and --recursion-limit to pcregrep. |
| 261 |
|
|
| 262 |
|
10. Added two casts needed to build with Visual Studio when NO_RECURSE is set. |
| 263 |
|
|
| 264 |
|
11. When the -o option was used, pcregrep was setting a return code of 1, even |
| 265 |
|
when matches were found, and --line-buffered was not being honoured. |
| 266 |
|
|
| 267 |
|
12. Added an optional parentheses number to the -o and --only-matching options |
| 268 |
|
of pcregrep. |
| 269 |
|
|
| 270 |
|
13. Imitating Perl's /g action for multiple matches is tricky when the pattern |
| 271 |
|
can match an empty string. The code to do it in pcretest and pcredemo |
| 272 |
|
needed fixing: |
| 273 |
|
|
| 274 |
|
(a) When the newline convention was "crlf", pcretest got it wrong, skipping |
| 275 |
|
only one byte after an empty string match just before CRLF (this case |
| 276 |
|
just got forgotten; "any" and "anycrlf" were OK). |
| 277 |
|
|
| 278 |
|
(b) The pcretest code also had a bug, causing it to loop forever in UTF-8 |
| 279 |
|
mode when an empty string match preceded an ASCII character followed by |
| 280 |
|
a non-ASCII character. (The code for advancing by one character rather |
| 281 |
|
than one byte was nonsense.) |
| 282 |
|
|
| 283 |
|
(c) The pcredemo.c sample program did not have any code at all to handle |
| 284 |
|
the cases when CRLF is a valid newline sequence. |
| 285 |
|
|
| 286 |
|
14. Neither pcre_exec() nor pcre_dfa_exec() was checking that the value given |
| 287 |
|
as a starting offset was within the subject string. There is now a new |
| 288 |
|
error, PCRE_ERROR_BADOFFSET, which is returned if the starting offset is |
| 289 |
|
negative or greater than the length of the string. In order to test this, |
| 290 |
|
pcretest is extended to allow the setting of negative starting offsets. |
| 291 |
|
|
| 292 |
|
15. In both pcre_exec() and pcre_dfa_exec() the code for checking that the |
| 293 |
|
starting offset points to the beginning of a UTF-8 character was |
| 294 |
|
unnecessarily clumsy. I tidied it up. |
| 295 |
|
|
| 296 |
|
16. Added PCRE_ERROR_SHORTUTF8 to make it possible to distinguish between a |
| 297 |
|
bad UTF-8 sequence and one that is incomplete when using PCRE_PARTIAL_HARD. |
| 298 |
|
|
| 299 |
|
17. Nobody had reported that the --include_dir option, which was added in |
| 300 |
|
release 7.7 should have been called --include-dir (hyphen, not underscore) |
| 301 |
|
for compatibility with GNU grep. I have changed it to --include-dir, but |
| 302 |
|
left --include_dir as an undocumented synonym, and the same for |
| 303 |
|
--exclude-dir, though that is not available in GNU grep, at least as of |
| 304 |
|
release 2.5.4. |
| 305 |
|
|
| 306 |
|
18. At a user's suggestion, the macros GETCHAR and friends (which pick up UTF-8 |
| 307 |
|
characters from a string of bytes) have been redefined so as not to use |
| 308 |
|
loops, in order to improve performance in some environments. At the same |
| 309 |
|
time, I abstracted some of the common code into auxiliary macros to save |
| 310 |
|
repetition (this should not affect the compiled code). |
| 311 |
|
|
| 312 |
|
19. If \c was followed by a multibyte UTF-8 character, bad things happened. A |
| 313 |
|
compile-time error is now given if \c is not followed by an ASCII |
| 314 |
|
character, that is, a byte less than 128. (In EBCDIC mode, the code is |
| 315 |
|
different, and any byte value is allowed.) |
| 316 |
|
|
| 317 |
|
20. Recognize (*NO_START_OPT) at the start of a pattern to set the PCRE_NO_ |
| 318 |
|
START_OPTIMIZE option, which is now allowed at compile time - but just |
| 319 |
|
passed through to pcre_exec() or pcre_dfa_exec(). This makes it available |
| 320 |
|
to pcregrep and other applications that have no direct access to PCRE |
| 321 |
|
options. The new /Y option in pcretest sets this option when calling |
| 322 |
|
pcre_compile(). |
| 323 |
|
|
| 324 |
|
21. Change 18 of release 8.01 broke the use of named subpatterns for recursive |
| 325 |
|
back references. Groups containing recursive back references were forced to |
| 326 |
|
be atomic by that change, but in the case of named groups, the amount of |
| 327 |
|
memory required was incorrectly computed, leading to "Failed: internal |
| 328 |
|
error: code overflow". This has been fixed. |
| 329 |
|
|
| 330 |
|
22. Some patches to pcre_stringpiece.h, pcre_stringpiece_unittest.cc, and |
| 331 |
|
pcretest.c, to avoid build problems in some Borland environments. |
| 332 |
|
|
| 333 |
|
|
| 334 |
|
Version 8.10 25-Jun-2010 |
| 335 |
|
------------------------ |
| 336 |
|
|
| 337 |
|
1. Added support for (*MARK:ARG) and for ARG additions to PRUNE, SKIP, and |
| 338 |
|
THEN. |
| 339 |
|
|
| 340 |
|
2. (*ACCEPT) was not working when inside an atomic group. |
| 341 |
|
|
| 342 |
|
3. Inside a character class, \B is treated as a literal by default, but |
| 343 |
|
faulted if PCRE_EXTRA is set. This mimics Perl's behaviour (the -w option |
| 344 |
|
causes the error). The code is unchanged, but I tidied the documentation. |
| 345 |
|
|
| 346 |
|
4. Inside a character class, PCRE always treated \R and \X as literals, |
| 347 |
|
whereas Perl faults them if its -w option is set. I have changed PCRE so |
| 348 |
|
that it faults them when PCRE_EXTRA is set. |
| 349 |
|
|
| 350 |
|
5. Added support for \N, which always matches any character other than |
| 351 |
|
newline. (It is the same as "." when PCRE_DOTALL is not set.) |
| 352 |
|
|
| 353 |
|
6. When compiling pcregrep with newer versions of gcc which may have |
| 354 |
|
FORTIFY_SOURCE set, several warnings "ignoring return value of 'fwrite', |
| 355 |
|
declared with attribute warn_unused_result" were given. Just casting the |
| 356 |
|
result to (void) does not stop the warnings; a more elaborate fudge is |
| 357 |
|
needed. I've used a macro to implement this. |
| 358 |
|
|
| 359 |
|
7. Minor change to pcretest.c to avoid a compiler warning. |
| 360 |
|
|
| 361 |
|
8. Added four artifical Unicode properties to help with an option to make |
| 362 |
|
\s etc use properties (see next item). The new properties are: Xan |
| 363 |
|
(alphanumeric), Xsp (Perl space), Xps (POSIX space), and Xwd (word). |
| 364 |
|
|
| 365 |
|
9. Added PCRE_UCP to make \b, \d, \s, \w, and certain POSIX character classes |
| 366 |
|
use Unicode properties. (*UCP) at the start of a pattern can be used to set |
| 367 |
|
this option. Modified pcretest to add /W to test this facility. Added |
| 368 |
|
REG_UCP to make it available via the POSIX interface. |
| 369 |
|
|
| 370 |
|
10. Added --line-buffered to pcregrep. |
| 371 |
|
|
| 372 |
|
11. In UTF-8 mode, if a pattern that was compiled with PCRE_CASELESS was |
| 373 |
|
studied, and the match started with a letter with a code point greater than |
| 374 |
|
127 whose first byte was different to the first byte of the other case of |
| 375 |
|
the letter, the other case of this starting letter was not recognized |
| 376 |
|
(#976). |
| 377 |
|
|
| 378 |
|
12. If a pattern that was studied started with a repeated Unicode property |
| 379 |
|
test, for example, \p{Nd}+, there was the theoretical possibility of |
| 380 |
|
setting up an incorrect bitmap of starting bytes, but fortunately it could |
| 381 |
|
not have actually happened in practice until change 8 above was made (it |
| 382 |
|
added property types that matched character-matching opcodes). |
| 383 |
|
|
| 384 |
|
13. pcre_study() now recognizes \h, \v, and \R when constructing a bit map of |
| 385 |
|
possible starting bytes for non-anchored patterns. |
| 386 |
|
|
| 387 |
|
14. Extended the "auto-possessify" feature of pcre_compile(). It now recognizes |
| 388 |
|
\R, and also a number of cases that involve Unicode properties, both |
| 389 |
|
explicit and implicit when PCRE_UCP is set. |
| 390 |
|
|
| 391 |
|
15. If a repeated Unicode property match (e.g. \p{Lu}*) was used with non-UTF-8 |
| 392 |
|
input, it could crash or give wrong results if characters with values |
| 393 |
|
greater than 0xc0 were present in the subject string. (Detail: it assumed |
| 394 |
|
UTF-8 input when processing these items.) |
| 395 |
|
|
| 396 |
|
16. Added a lot of (int) casts to avoid compiler warnings in systems where |
| 397 |
|
size_t is 64-bit (#991). |
| 398 |
|
|
| 399 |
|
17. Added a check for running out of memory when PCRE is compiled with |
| 400 |
|
--disable-stack-for-recursion (#990). |
| 401 |
|
|
| 402 |
|
18. If the last data line in a file for pcretest does not have a newline on |
| 403 |
|
the end, a newline was missing in the output. |
| 404 |
|
|
| 405 |
|
19. The default pcre_chartables.c file recognizes only ASCII characters (values |
| 406 |
|
less than 128) in its various bitmaps. However, there is a facility for |
| 407 |
|
generating tables according to the current locale when PCRE is compiled. It |
| 408 |
|
turns out that in some environments, 0x85 and 0xa0, which are Unicode space |
| 409 |
|
characters, are recognized by isspace() and therefore were getting set in |
| 410 |
|
these tables, and indeed these tables seem to approximate to ISO 8859. This |
| 411 |
|
caused a problem in UTF-8 mode when pcre_study() was used to create a list |
| 412 |
|
of bytes that can start a match. For \s, it was including 0x85 and 0xa0, |
| 413 |
|
which of course cannot start UTF-8 characters. I have changed the code so |
| 414 |
|
that only real ASCII characters (less than 128) and the correct starting |
| 415 |
|
bytes for UTF-8 encodings are set for characters greater than 127 when in |
| 416 |
|
UTF-8 mode. (When PCRE_UCP is set - see 9 above - the code is different |
| 417 |
|
altogether.) |
| 418 |
|
|
| 419 |
|
20. Added the /T option to pcretest so as to be able to run tests with non- |
| 420 |
|
standard character tables, thus making it possible to include the tests |
| 421 |
|
used for 19 above in the standard set of tests. |
| 422 |
|
|
| 423 |
|
21. A pattern such as (?&t)(?#()(?(DEFINE)(?<t>a)) which has a forward |
| 424 |
|
reference to a subpattern the other side of a comment that contains an |
| 425 |
|
opening parenthesis caused either an internal compiling error, or a |
| 426 |
|
reference to the wrong subpattern. |
| 427 |
|
|
| 428 |
|
|
| 429 |
|
Version 8.02 19-Mar-2010 |
| 430 |
|
------------------------ |
| 431 |
|
|
| 432 |
|
1. The Unicode data tables have been updated to Unicode 5.2.0. |
| 433 |
|
|
| 434 |
|
2. Added the option --libs-cpp to pcre-config, but only when C++ support is |
| 435 |
|
configured. |
| 436 |
|
|
| 437 |
|
3. Updated the licensing terms in the pcregexp.pas file, as agreed with the |
| 438 |
|
original author of that file, following a query about its status. |
| 439 |
|
|
| 440 |
|
4. On systems that do not have stdint.h (e.g. Solaris), check for and include |
| 441 |
|
inttypes.h instead. This fixes a bug that was introduced by change 8.01/8. |
| 442 |
|
|
| 443 |
|
5. A pattern such as (?&t)*+(?(DEFINE)(?<t>.)) which has a possessive |
| 444 |
|
quantifier applied to a forward-referencing subroutine call, could compile |
| 445 |
|
incorrect code or give the error "internal error: previously-checked |
| 446 |
|
referenced subpattern not found". |
| 447 |
|
|
| 448 |
|
6. Both MS Visual Studio and Symbian OS have problems with initializing |
| 449 |
|
variables to point to external functions. For these systems, therefore, |
| 450 |
|
pcre_malloc etc. are now initialized to local functions that call the |
| 451 |
|
relevant global functions. |
| 452 |
|
|
| 453 |
|
7. There were two entries missing in the vectors called coptable and poptable |
| 454 |
|
in pcre_dfa_exec.c. This could lead to memory accesses outsize the vectors. |
| 455 |
|
I've fixed the data, and added a kludgy way of testing at compile time that |
| 456 |
|
the lengths are correct (equal to the number of opcodes). |
| 457 |
|
|
| 458 |
|
8. Following on from 7, I added a similar kludge to check the length of the |
| 459 |
|
eint vector in pcreposix.c. |
| 460 |
|
|
| 461 |
|
9. Error texts for pcre_compile() are held as one long string to avoid too |
| 462 |
|
much relocation at load time. To find a text, the string is searched, |
| 463 |
|
counting zeros. There was no check for running off the end of the string, |
| 464 |
|
which could happen if a new error number was added without updating the |
| 465 |
|
string. |
| 466 |
|
|
| 467 |
|
10. \K gave a compile-time error if it appeared in a lookbehind assersion. |
| 468 |
|
|
| 469 |
|
11. \K was not working if it appeared in an atomic group or in a group that |
| 470 |
|
was called as a "subroutine", or in an assertion. Perl 5.11 documents that |
| 471 |
|
\K is "not well defined" if used in an assertion. PCRE now accepts it if |
| 472 |
|
the assertion is positive, but not if it is negative. |
| 473 |
|
|
| 474 |
|
12. Change 11 fortuitously reduced the size of the stack frame used in the |
| 475 |
|
"match()" function of pcre_exec.c by one pointer. Forthcoming |
| 476 |
|
implementation of support for (*MARK) will need an extra pointer on the |
| 477 |
|
stack; I have reserved it now, so that the stack frame size does not |
| 478 |
|
decrease. |
| 479 |
|
|
| 480 |
|
13. A pattern such as (?P<L1>(?P<L2>0)|(?P>L2)(?P>L1)) in which the only other |
| 481 |
|
item in branch that calls a recursion is a subroutine call - as in the |
| 482 |
|
second branch in the above example - was incorrectly given the compile- |
| 483 |
|
time error "recursive call could loop indefinitely" because pcre_compile() |
| 484 |
|
was not correctly checking the subroutine for matching a non-empty string. |
| 485 |
|
|
| 486 |
|
14. The checks for overrunning compiling workspace could trigger after an |
| 487 |
|
overrun had occurred. This is a "should never occur" error, but it can be |
| 488 |
|
triggered by pathological patterns such as hundreds of nested parentheses. |
| 489 |
|
The checks now trigger 100 bytes before the end of the workspace. |
| 490 |
|
|
| 491 |
|
15. Fix typo in configure.ac: "srtoq" should be "strtoq". |
| 492 |
|
|
| 493 |
|
|
| 494 |
|
Version 8.01 19-Jan-2010 |
| 495 |
|
------------------------ |
| 496 |
|
|
| 497 |
1. If a pattern contained a conditional subpattern with only one branch (in |
1. If a pattern contained a conditional subpattern with only one branch (in |
| 498 |
particular, this includes all (DEFINE) patterns), a call to pcre_study() |
particular, this includes all (*DEFINE) patterns), a call to pcre_study() |
| 499 |
computed the wrong minimum data length (which is of course zero for such |
computed the wrong minimum data length (which is of course zero for such |
| 500 |
subpatterns). |
subpatterns). This could cause incorrect "no match" results. |
| 501 |
|
|
| 502 |
2. For patterns such as (?i)a(?-i)b|c where an option setting at the start of |
2. For patterns such as (?i)a(?-i)b|c where an option setting at the start of |
| 503 |
the pattern is reset in the first branch, pcre_compile() failed with |
the pattern is reset in the first branch, pcre_compile() failed with |
| 551 |
custom one, because of the following reported problem in Windows: |
custom one, because of the following reported problem in Windows: |
| 552 |
|
|
| 553 |
- libbz2 uses the Pascal calling convention (WINAPI) for the functions |
- libbz2 uses the Pascal calling convention (WINAPI) for the functions |
| 554 |
under Win32. |
under Win32. |
| 555 |
- The standard autoconf AC_CHECK_LIB fails to include "bzlib.h", |
- The standard autoconf AC_CHECK_LIB fails to include "bzlib.h", |
| 556 |
therefore missing the function definition. |
therefore missing the function definition. |
| 557 |
- The compiler thus generates a "C" signature for the test function. |
- The compiler thus generates a "C" signature for the test function. |
| 558 |
- The linker fails to find the "C" function. |
- The linker fails to find the "C" function. |
| 559 |
- PCRE fails to configure if asked to do so against libbz2. |
- PCRE fails to configure if asked to do so against libbz2. |
| 560 |
|
|
| 561 |
|
11. When running libtoolize from libtool-2.2.6b as part of autogen.sh, these |
| 562 |
|
messages were output: |
| 563 |
|
|
| 564 |
|
Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and |
| 565 |
|
rerunning libtoolize, to keep the correct libtool macros in-tree. |
| 566 |
|
Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am. |
| 567 |
|
|
| 568 |
|
I have done both of these things. |
| 569 |
|
|
| 570 |
|
12. Although pcre_dfa_exec() does not use nearly as much stack as pcre_exec() |
| 571 |
|
most of the time, it *can* run out if it is given a pattern that contains a |
| 572 |
|
runaway infinite recursion. I updated the discussion in the pcrestack man |
| 573 |
|
page. |
| 574 |
|
|
| 575 |
|
13. Now that we have gone to the x.xx style of version numbers, the minor |
| 576 |
|
version may start with zero. Using 08 or 09 is a bad idea because users |
| 577 |
|
might check the value of PCRE_MINOR in their code, and 08 or 09 may be |
| 578 |
|
interpreted as invalid octal numbers. I've updated the previous comment in |
| 579 |
|
configure.ac, and also added a check that gives an error if 08 or 09 are |
| 580 |
|
used. |
| 581 |
|
|
| 582 |
|
14. Change 8.00/11 was not quite complete: code had been accidentally omitted, |
| 583 |
|
causing partial matching to fail when the end of the subject matched \W |
| 584 |
|
in a UTF-8 pattern where \W was quantified with a minimum of 3. |
| 585 |
|
|
| 586 |
|
15. There were some discrepancies between the declarations in pcre_internal.h |
| 587 |
|
of _pcre_is_newline(), _pcre_was_newline(), and _pcre_valid_utf8() and |
| 588 |
|
their definitions. The declarations used "const uschar *" and the |
| 589 |
|
definitions used USPTR. Even though USPTR is normally defined as "const |
| 590 |
|
unsigned char *" (and uschar is typedeffed as "unsigned char"), it was |
| 591 |
|
reported that: "This difference in casting confuses some C++ compilers, for |
| 592 |
|
example, SunCC recognizes above declarations as different functions and |
| 593 |
|
generates broken code for hbpcre." I have changed the declarations to use |
| 594 |
|
USPTR. |
| 595 |
|
|
| 596 |
|
16. GNU libtool is named differently on some systems. The autogen.sh script now |
| 597 |
|
tries several variants such as glibtoolize (MacOSX) and libtoolize1x |
| 598 |
|
(FreeBSD). |
| 599 |
|
|
| 600 |
|
17. Applied Craig's patch that fixes an HP aCC compile error in pcre 8.00 |
| 601 |
|
(strtoXX undefined when compiling pcrecpp.cc). The patch contains this |
| 602 |
|
comment: "Figure out how to create a longlong from a string: strtoll and |
| 603 |
|
equivalent. It's not enough to call AC_CHECK_FUNCS: hpux has a strtoll, for |
| 604 |
|
instance, but it only takes 2 args instead of 3!" |
| 605 |
|
|
| 606 |
|
18. A subtle bug concerned with back references has been fixed by a change of |
| 607 |
|
specification, with a corresponding code fix. A pattern such as |
| 608 |
|
^(xa|=?\1a)+$ which contains a back reference inside the group to which it |
| 609 |
|
refers, was giving matches when it shouldn't. For example, xa=xaaa would |
| 610 |
|
match that pattern. Interestingly, Perl (at least up to 5.11.3) has the |
| 611 |
|
same bug. Such groups have to be quantified to be useful, or contained |
| 612 |
|
inside another quantified group. (If there's no repetition, the reference |
| 613 |
|
can never match.) The problem arises because, having left the group and |
| 614 |
|
moved on to the rest of the pattern, a later failure that backtracks into |
| 615 |
|
the group uses the captured value from the final iteration of the group |
| 616 |
|
rather than the correct earlier one. I have fixed this in PCRE by forcing |
| 617 |
|
any group that contains a reference to itself to be an atomic group; that |
| 618 |
|
is, there cannot be any backtracking into it once it has completed. This is |
| 619 |
|
similar to recursive and subroutine calls. |
| 620 |
|
|
| 621 |
|
|
| 622 |
Version 8.00 19-Oct-09 |
Version 8.00 19-Oct-09 |
| 623 |
---------------------- |
---------------------- |
Parent Directory
| 
Revision Log
| 
Patch