Parent Directory
|
Revision Log
|
Patch
| revision 11 by nigel, Sat Feb 24 21:38:17 2007 UTC | revision 494 by ph10, Tue Mar 2 11:53:12 2010 UTC | |
|---|---|---|
| # | Line 1 | Line 1 |
| 1 | ChangeLog for PCRE | ChangeLog for PCRE |
| 2 | ------------------ | ------------------ |
| 3 | ||
| 4 | Version 8.02 01-Mar-2010 | |
| 5 | ------------------------ | |
| 6 | ||
| 7 | 1. The Unicode data tables have been updated to Unicode 5.2.0. | |
| 8 | ||
| 9 | 2. Added the option --libs-cpp to pcre-config, but only when C++ support is | |
| 10 | configured. | |
| 11 | ||
| 12 | 3. Updated the licensing terms in the pcregexp.pas file, as agreed with the | |
| 13 | original author of that file, following a query about its status. | |
| 14 | ||
| 15 | 4. On systems that do not have stdint.h (e.g. Solaris), check for and include | |
| 16 | inttypes.h instead. This fixes a bug that was introduced by change 8.01/8. | |
| 17 | ||
| 18 | ||
| 19 | Version 8.01 19-Jan-2010 | |
| 20 | ------------------------ | |
| 21 | ||
| 22 | 1. If a pattern contained a conditional subpattern with only one branch (in | |
| 23 | particular, this includes all (*DEFINE) patterns), a call to pcre_study() | |
| 24 | computed the wrong minimum data length (which is of course zero for such | |
| 25 | subpatterns). This could cause incorrect "no match" results. | |
| 26 | ||
| 27 | 2. For patterns such as (?i)a(?-i)b|c where an option setting at the start of | |
| 28 | the pattern is reset in the first branch, pcre_compile() failed with | |
| 29 | "internal error: code overflow at offset...". This happened only when | |
| 30 | the reset was to the original external option setting. (An optimization | |
| 31 | abstracts leading options settings into an external setting, which was the | |
| 32 | cause of this.) | |
| 33 | ||
| 34 | 3. A pattern such as ^(?!a(*SKIP)b) where a negative assertion contained one | |
| 35 | of the verbs SKIP, PRUNE, or COMMIT, did not work correctly. When the | |
| 36 | assertion pattern did not match (meaning that the assertion was true), it | |
| 37 | was incorrectly treated as false if the SKIP had been reached during the | |
| 38 | matching. This also applied to assertions used as conditions. | |
| 39 | ||
| 40 | 4. If an item that is not supported by pcre_dfa_exec() was encountered in an | |
| 41 | assertion subpattern, including such a pattern used as a condition, | |
| 42 | unpredictable results occurred, instead of the error return | |
| 43 | PCRE_ERROR_DFA_UITEM. | |
| 44 | ||
| 45 | 5. The C++ GlobalReplace function was not working like Perl for the special | |
| 46 | situation when an empty string is matched. It now does the fancy magic | |
| 47 | stuff that is necessary. | |
| 48 | ||
| 49 | 6. In pcre_internal.h, obsolete includes to setjmp.h and stdarg.h have been | |
| 50 | removed. (These were left over from very, very early versions of PCRE.) | |
| 51 | ||
| 52 | 7. Some cosmetic changes to the code to make life easier when compiling it | |
| 53 | as part of something else: | |
| 54 | ||
| 55 | (a) Change DEBUG to PCRE_DEBUG. | |
| 56 | ||
| 57 | (b) In pcre_compile(), rename the member of the "branch_chain" structure | |
| 58 | called "current" as "current_branch", to prevent a collision with the | |
| 59 | Linux macro when compiled as a kernel module. | |
| 60 | ||
| 61 | (c) In pcre_study(), rename the function set_bit() as set_table_bit(), to | |
| 62 | prevent a collision with the Linux macro when compiled as a kernel | |
| 63 | module. | |
| 64 | ||
| 65 | 8. In pcre_compile() there are some checks for integer overflows that used to | |
| 66 | cast potentially large values to (double). This has been changed to that | |
| 67 | when building, a check for int64_t is made, and if it is found, it is used | |
| 68 | instead, thus avoiding the use of floating point arithmetic. (There is no | |
| 69 | other use of FP in PCRE.) If int64_t is not found, the fallback is to | |
| 70 | double. | |
| 71 | ||
| 72 | 9. Added two casts to avoid signed/unsigned warnings from VS Studio Express | |
| 73 | 2005 (difference between two addresses compared to an unsigned value). | |
| 74 | ||
| 75 | 10. Change the standard AC_CHECK_LIB test for libbz2 in configure.ac to a | |
| 76 | custom one, because of the following reported problem in Windows: | |
| 77 | ||
| 78 | - libbz2 uses the Pascal calling convention (WINAPI) for the functions | |
| 79 | under Win32. | |
| 80 | - The standard autoconf AC_CHECK_LIB fails to include "bzlib.h", | |
| 81 | therefore missing the function definition. | |
| 82 | - The compiler thus generates a "C" signature for the test function. | |
| 83 | - The linker fails to find the "C" function. | |
| 84 | - PCRE fails to configure if asked to do so against libbz2. | |
| 85 | ||
| 86 | 11. When running libtoolize from libtool-2.2.6b as part of autogen.sh, these | |
| 87 | messages were output: | |
| 88 | ||
| 89 | Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and | |
| 90 | rerunning libtoolize, to keep the correct libtool macros in-tree. | |
| 91 | Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am. | |
| 92 | ||
| 93 | I have done both of these things. | |
| 94 | ||
| 95 | 12. Although pcre_dfa_exec() does not use nearly as much stack as pcre_exec() | |
| 96 | most of the time, it *can* run out if it is given a pattern that contains a | |
| 97 | runaway infinite recursion. I updated the discussion in the pcrestack man | |
| 98 | page. | |
| 99 | ||
| 100 | 13. Now that we have gone to the x.xx style of version numbers, the minor | |
| 101 | version may start with zero. Using 08 or 09 is a bad idea because users | |
| 102 | might check the value of PCRE_MINOR in their code, and 08 or 09 may be | |
| 103 | interpreted as invalid octal numbers. I've updated the previous comment in | |
| 104 | configure.ac, and also added a check that gives an error if 08 or 09 are | |
| 105 | used. | |
| 106 | ||
| 107 | 14. Change 8.00/11 was not quite complete: code had been accidentally omitted, | |
| 108 | causing partial matching to fail when the end of the subject matched \W | |
| 109 | in a UTF-8 pattern where \W was quantified with a minimum of 3. | |
| 110 | ||
| 111 | 15. There were some discrepancies between the declarations in pcre_internal.h | |
| 112 | of _pcre_is_newline(), _pcre_was_newline(), and _pcre_valid_utf8() and | |
| 113 | their definitions. The declarations used "const uschar *" and the | |
| 114 | definitions used USPTR. Even though USPTR is normally defined as "const | |
| 115 | unsigned char *" (and uschar is typedeffed as "unsigned char"), it was | |
| 116 | reported that: "This difference in casting confuses some C++ compilers, for | |
| 117 | example, SunCC recognizes above declarations as different functions and | |
| 118 | generates broken code for hbpcre." I have changed the declarations to use | |
| 119 | USPTR. | |
| 120 | ||
| 121 | 16. GNU libtool is named differently on some systems. The autogen.sh script now | |
| 122 | tries several variants such as glibtoolize (MacOSX) and libtoolize1x | |
| 123 | (FreeBSD). | |
| 124 | ||
| 125 | 17. Applied Craig's patch that fixes an HP aCC compile error in pcre 8.00 | |
| 126 | (strtoXX undefined when compiling pcrecpp.cc). The patch contains this | |
| 127 | comment: "Figure out how to create a longlong from a string: strtoll and | |
| 128 | equivalent. It's not enough to call AC_CHECK_FUNCS: hpux has a strtoll, for | |
| 129 | instance, but it only takes 2 args instead of 3!" | |
| 130 | ||
| 131 | 18. A subtle bug concerned with back references has been fixed by a change of | |
| 132 | specification, with a corresponding code fix. A pattern such as | |
| 133 | ^(xa|=?\1a)+$ which contains a back reference inside the group to which it | |
| 134 | refers, was giving matches when it shouldn't. For example, xa=xaaa would | |
| 135 | match that pattern. Interestingly, Perl (at least up to 5.11.3) has the | |
| 136 | same bug. Such groups have to be quantified to be useful, or contained | |
| 137 | inside another quantified group. (If there's no repetition, the reference | |
| 138 | can never match.) The problem arises because, having left the group and | |
| 139 | moved on to the rest of the pattern, a later failure that backtracks into | |
| 140 | the group uses the captured value from the final iteration of the group | |
| 141 | rather than the correct earlier one. I have fixed this in PCRE by forcing | |
| 142 | any group that contains a reference to itself to be an atomic group; that | |
| 143 | is, there cannot be any backtracking into it once it has completed. This is | |
| 144 | similar to recursive and subroutine calls. | |
| 145 | ||
| 146 | ||
| 147 | Version 8.00 19-Oct-09 | |
| 148 | ---------------------- | |
| 149 | ||
| 150 | 1. The table for translating pcre_compile() error codes into POSIX error codes | |
| 151 | was out-of-date, and there was no check on the pcre_compile() error code | |
| 152 | being within the table. This could lead to an OK return being given in | |
| 153 | error. | |
| 154 | ||
| 155 | 2. Changed the call to open a subject file in pcregrep from fopen(pathname, | |
| 156 | "r") to fopen(pathname, "rb"), which fixed a problem with some of the tests | |
| 157 | in a Windows environment. | |
| 158 | ||
| 159 | 3. The pcregrep --count option prints the count for each file even when it is | |
| 160 | zero, as does GNU grep. However, pcregrep was also printing all files when | |
| 161 | --files-with-matches was added. Now, when both options are given, it prints | |
| 162 | counts only for those files that have at least one match. (GNU grep just | |
| 163 | prints the file name in this circumstance, but including the count seems | |
| 164 | more useful - otherwise, why use --count?) Also ensured that the | |
| 165 | combination -clh just lists non-zero counts, with no names. | |
| 166 | ||
| 167 | 4. The long form of the pcregrep -F option was incorrectly implemented as | |
| 168 | --fixed_strings instead of --fixed-strings. This is an incompatible change, | |
| 169 | but it seems right to fix it, and I didn't think it was worth preserving | |
| 170 | the old behaviour. | |
| 171 | ||
| 172 | 5. The command line items --regex=pattern and --regexp=pattern were not | |
| 173 | recognized by pcregrep, which required --regex pattern or --regexp pattern | |
| 174 | (with a space rather than an '='). The man page documented the '=' forms, | |
| 175 | which are compatible with GNU grep; these now work. | |
| 176 | ||
| 177 | 6. No libpcreposix.pc file was created for pkg-config; there was just | |
| 178 | libpcre.pc and libpcrecpp.pc. The omission has been rectified. | |
| 179 | ||
| 180 | 7. Added #ifndef SUPPORT_UCP into the pcre_ucd.c module, to reduce its size | |
| 181 | when UCP support is not needed, by modifying the Python script that | |
| 182 | generates it from Unicode data files. This should not matter if the module | |
| 183 | is correctly used as a library, but I received one complaint about 50K of | |
| 184 | unwanted data. My guess is that the person linked everything into his | |
| 185 | program rather than using a library. Anyway, it does no harm. | |
| 186 | ||
| 187 | 8. A pattern such as /\x{123}{2,2}+/8 was incorrectly compiled; the trigger | |
| 188 | was a minimum greater than 1 for a wide character in a possessive | |
| 189 | repetition. The same bug could also affect patterns like /(\x{ff}{0,2})*/8 | |
| 190 | which had an unlimited repeat of a nested, fixed maximum repeat of a wide | |
| 191 | character. Chaos in the form of incorrect output or a compiling loop could | |
| 192 | result. | |
| 193 | ||
| 194 | 9. The restrictions on what a pattern can contain when partial matching is | |
| 195 | requested for pcre_exec() have been removed. All patterns can now be | |
| 196 | partially matched by this function. In addition, if there are at least two | |
| 197 | slots in the offset vector, the offset of the earliest inspected character | |
| 198 | for the match and the offset of the end of the subject are set in them when | |
| 199 | PCRE_ERROR_PARTIAL is returned. | |
| 200 | ||
| 201 | 10. Partial matching has been split into two forms: PCRE_PARTIAL_SOFT, which is | |
| 202 | synonymous with PCRE_PARTIAL, for backwards compatibility, and | |
| 203 | PCRE_PARTIAL_HARD, which causes a partial match to supersede a full match, | |
| 204 | and may be more useful for multi-segment matching. | |
| 205 | ||
| 206 | 11. Partial matching with pcre_exec() is now more intuitive. A partial match | |
| 207 | used to be given if ever the end of the subject was reached; now it is | |
| 208 | given only if matching could not proceed because another character was | |
| 209 | needed. This makes a difference in some odd cases such as Z(*FAIL) with the | |
| 210 | string "Z", which now yields "no match" instead of "partial match". In the | |
| 211 | case of pcre_dfa_exec(), "no match" is given if every matching path for the | |
| 212 | final character ended with (*FAIL). | |
| 213 | ||
| 214 | 12. Restarting a match using pcre_dfa_exec() after a partial match did not work | |
| 215 | if the pattern had a "must contain" character that was already found in the | |
| 216 | earlier partial match, unless partial matching was again requested. For | |
| 217 | example, with the pattern /dog.(body)?/, the "must contain" character is | |
| 218 | "g". If the first part-match was for the string "dog", restarting with | |
| 219 | "sbody" failed. This bug has been fixed. | |
| 220 | ||
| 221 | 13. The string returned by pcre_dfa_exec() after a partial match has been | |
| 222 | changed so that it starts at the first inspected character rather than the | |
| 223 | first character of the match. This makes a difference only if the pattern | |
| 224 | starts with a lookbehind assertion or \b or \B (\K is not supported by | |
| 225 | pcre_dfa_exec()). It's an incompatible change, but it makes the two | |
| 226 | matching functions compatible, and I think it's the right thing to do. | |
| 227 | ||
| 228 | 14. Added a pcredemo man page, created automatically from the pcredemo.c file, | |
| 229 | so that the demonstration program is easily available in environments where | |
| 230 | PCRE has not been installed from source. | |
| 231 | ||
| 232 | 15. Arranged to add -DPCRE_STATIC to cflags in libpcre.pc, libpcreposix.cp, | |
| 233 | libpcrecpp.pc and pcre-config when PCRE is not compiled as a shared | |
| 234 | library. | |
| 235 | ||
| 236 | 16. Added REG_UNGREEDY to the pcreposix interface, at the request of a user. | |
| 237 | It maps to PCRE_UNGREEDY. It is not, of course, POSIX-compatible, but it | |
| 238 | is not the first non-POSIX option to be added. Clearly some people find | |
| 239 | these options useful. | |
| 240 | ||
| 241 | 17. If a caller to the POSIX matching function regexec() passes a non-zero | |
| 242 | value for nmatch with a NULL value for pmatch, the value of | |
| 243 | nmatch is forced to zero. | |
| 244 | ||
| 245 | 18. RunGrepTest did not have a test for the availability of the -u option of | |
| 246 | the diff command, as RunTest does. It now checks in the same way as | |
| 247 | RunTest, and also checks for the -b option. | |
| 248 | ||
| 249 | 19. If an odd number of negated classes containing just a single character | |
| 250 | interposed, within parentheses, between a forward reference to a named | |
| 251 | subpattern and the definition of the subpattern, compilation crashed with | |
| 252 | an internal error, complaining that it could not find the referenced | |
| 253 | subpattern. An example of a crashing pattern is /(?&A)(([^m])(?<A>))/. | |
| 254 | [The bug was that it was starting one character too far in when skipping | |
| 255 | over the character class, thus treating the ] as data rather than | |
| 256 | terminating the class. This meant it could skip too much.] | |
| 257 | ||
| 258 | 20. Added PCRE_NOTEMPTY_ATSTART in order to be able to correctly implement the | |
| 259 | /g option in pcretest when the pattern contains \K, which makes it possible | |
| 260 | to have an empty string match not at the start, even when the pattern is | |
| 261 | anchored. Updated pcretest and pcredemo to use this option. | |
| 262 | ||
| 263 | 21. If the maximum number of capturing subpatterns in a recursion was greater | |
| 264 | than the maximum at the outer level, the higher number was returned, but | |
| 265 | with unset values at the outer level. The correct (outer level) value is | |
| 266 | now given. | |
| 267 | ||
| 268 | 22. If (*ACCEPT) appeared inside capturing parentheses, previous releases of | |
| 269 | PCRE did not set those parentheses (unlike Perl). I have now found a way to | |
| 270 | make it do so. The string so far is captured, making this feature | |
| 271 | compatible with Perl. | |
| 272 | ||
| 273 | 23. The tests have been re-organized, adding tests 11 and 12, to make it | |
| 274 | possible to check the Perl 5.10 features against Perl 5.10. | |
| 275 | ||
| 276 | 24. Perl 5.10 allows subroutine calls in lookbehinds, as long as the subroutine | |
| 277 | pattern matches a fixed length string. PCRE did not allow this; now it | |
| 278 | does. Neither allows recursion. | |
| 279 | ||
| 280 | 25. I finally figured out how to implement a request to provide the minimum | |
| 281 | length of subject string that was needed in order to match a given pattern. | |
| 282 | (It was back references and recursion that I had previously got hung up | |
| 283 | on.) This code has now been added to pcre_study(); it finds a lower bound | |
| 284 | to the length of subject needed. It is not necessarily the greatest lower | |
| 285 | bound, but using it to avoid searching strings that are too short does give | |
| 286 | some useful speed-ups. The value is available to calling programs via | |
| 287 | pcre_fullinfo(). | |
| 288 | ||
| 289 | 26. While implementing 25, I discovered to my embarrassment that pcretest had | |
| 290 | not been passing the result of pcre_study() to pcre_dfa_exec(), so the | |
| 291 | study optimizations had never been tested with that matching function. | |
| 292 | Oops. What is worse, even when it was passed study data, there was a bug in | |
| 293 | pcre_dfa_exec() that meant it never actually used it. Double oops. There | |
| 294 | were also very few tests of studied patterns with pcre_dfa_exec(). | |
| 295 | ||
| 296 | 27. If (?| is used to create subpatterns with duplicate numbers, they are now | |
| 297 | allowed to have the same name, even if PCRE_DUPNAMES is not set. However, | |
| 298 | on the other side of the coin, they are no longer allowed to have different | |
| 299 | names, because these cannot be distinguished in PCRE, and this has caused | |
| 300 | confusion. (This is a difference from Perl.) | |
| 301 | ||
| 302 | 28. When duplicate subpattern names are present (necessarily with different | |
| 303 | numbers, as required by 27 above), and a test is made by name in a | |
| 304 | conditional pattern, either for a subpattern having been matched, or for | |
| 305 | recursion in such a pattern, all the associated numbered subpatterns are | |
| 306 | tested, and the overall condition is true if the condition is true for any | |
| 307 | one of them. This is the way Perl works, and is also more like the way | |
| 308 | testing by number works. | |
| 309 | ||
| 310 | ||
| 311 | Version 7.9 11-Apr-09 | |
| 312 | --------------------- | |
| 313 | ||
| 314 | 1. When building with support for bzlib/zlib (pcregrep) and/or readline | |
| 315 | (pcretest), all targets were linked against these libraries. This included | |
| 316 | libpcre, libpcreposix, and libpcrecpp, even though they do not use these | |
| 317 | libraries. This caused unwanted dependencies to be created. This problem | |
| 318 | has been fixed, and now only pcregrep is linked with bzlib/zlib and only | |
| 319 | pcretest is linked with readline. | |
| 320 | ||
| 321 | 2. The "typedef int BOOL" in pcre_internal.h that was included inside the | |
| 322 | "#ifndef FALSE" condition by an earlier change (probably 7.8/18) has been | |
| 323 | moved outside it again, because FALSE and TRUE are already defined in AIX, | |
| 324 | but BOOL is not. | |
| 325 | ||
| 326 | 3. The pcre_config() function was treating the PCRE_MATCH_LIMIT and | |
| 327 | PCRE_MATCH_LIMIT_RECURSION values as ints, when they should be long ints. | |
| 328 | ||
| 329 | 4. The pcregrep documentation said spaces were inserted as well as colons (or | |
| 330 | hyphens) following file names and line numbers when outputting matching | |
| 331 | lines. This is not true; no spaces are inserted. I have also clarified the | |
| 332 | wording for the --colour (or --color) option. | |
| 333 | ||
| 334 | 5. In pcregrep, when --colour was used with -o, the list of matching strings | |
| 335 | was not coloured; this is different to GNU grep, so I have changed it to be | |
| 336 | the same. | |
| 337 | ||
| 338 | 6. When --colo(u)r was used in pcregrep, only the first matching substring in | |
| 339 | each matching line was coloured. Now it goes on to look for further matches | |
| 340 | of any of the test patterns, which is the same behaviour as GNU grep. | |
| 341 | ||
| 342 | 7. A pattern that could match an empty string could cause pcregrep to loop; it | |
| 343 | doesn't make sense to accept an empty string match in pcregrep, so I have | |
| 344 | locked it out (using PCRE's PCRE_NOTEMPTY option). By experiment, this | |
| 345 | seems to be how GNU grep behaves. | |
| 346 | ||
| 347 | 8. The pattern (?(?=.*b)b|^) was incorrectly compiled as "match must be at | |
| 348 | start or after a newline", because the conditional assertion was not being | |
| 349 | correctly handled. The rule now is that both the assertion and what follows | |
| 350 | in the first alternative must satisfy the test. | |
| 351 | ||
| 352 | 9. If auto-callout was enabled in a pattern with a conditional group whose | |
| 353 | condition was an assertion, PCRE could crash during matching, both with | |
| 354 | pcre_exec() and pcre_dfa_exec(). | |
| 355 | ||
| 356 | 10. The PCRE_DOLLAR_ENDONLY option was not working when pcre_dfa_exec() was | |
| 357 | used for matching. | |
| 358 | ||
| 359 | 11. Unicode property support in character classes was not working for | |
| 360 | characters (bytes) greater than 127 when not in UTF-8 mode. | |
| 361 | ||
| 362 | 12. Added the -M command line option to pcretest. | |
| 363 | ||
| 364 | 14. Added the non-standard REG_NOTEMPTY option to the POSIX interface. | |
| 365 | ||
| 366 | 15. Added the PCRE_NO_START_OPTIMIZE match-time option. | |
| 367 | ||
| 368 | 16. Added comments and documentation about mis-use of no_arg in the C++ | |
| 369 | wrapper. | |
| 370 | ||
| 371 | 17. Implemented support for UTF-8 encoding in EBCDIC environments, a patch | |
| 372 | from Martin Jerabek that uses macro names for all relevant character and | |
| 373 | string constants. | |
| 374 | ||
| 375 | 18. Added to pcre_internal.h two configuration checks: (a) If both EBCDIC and | |
| 376 | SUPPORT_UTF8 are set, give an error; (b) If SUPPORT_UCP is set without | |
| 377 | SUPPORT_UTF8, define SUPPORT_UTF8. The "configure" script handles both of | |
| 378 | these, but not everybody uses configure. | |
| 379 | ||
| 380 | 19. A conditional group that had only one branch was not being correctly | |
| 381 | recognized as an item that could match an empty string. This meant that an | |
| 382 | enclosing group might also not be so recognized, causing infinite looping | |
| 383 | (and probably a segfault) for patterns such as ^"((?(?=[a])[^"])|b)*"$ | |
| 384 | with the subject "ab", where knowledge that the repeated group can match | |
| 385 | nothing is needed in order to break the loop. | |
| 386 | ||
| 387 | 20. If a pattern that was compiled with callouts was matched using pcre_dfa_ | |
| 388 | exec(), but without supplying a callout function, matching went wrong. | |
| 389 | ||
| 390 | 21. If PCRE_ERROR_MATCHLIMIT occurred during a recursion, there was a memory | |
| 391 | leak if the size of the offset vector was greater than 30. When the vector | |
| 392 | is smaller, the saved offsets during recursion go onto a local stack | |
| 393 | vector, but for larger vectors malloc() is used. It was failing to free | |
| 394 | when the recursion yielded PCRE_ERROR_MATCH_LIMIT (or any other "abnormal" | |
| 395 | error, in fact). | |
| 396 | ||
| 397 | 22. There was a missing #ifdef SUPPORT_UTF8 round one of the variables in the | |
| 398 | heapframe that is used only when UTF-8 support is enabled. This caused no | |
| 399 | problem, but was untidy. | |
| 400 | ||
| 401 | 23. Steven Van Ingelgem's patch to CMakeLists.txt to change the name | |
| 402 | CMAKE_BINARY_DIR to PROJECT_BINARY_DIR so that it works when PCRE is | |
| 403 | included within another project. | |
| 404 | ||
| 405 | 24. Steven Van Ingelgem's patches to add more options to the CMake support, | |
| 406 | slightly modified by me: | |
| 407 | ||
| 408 | (a) PCRE_BUILD_TESTS can be set OFF not to build the tests, including | |
| 409 | not building pcregrep. | |
| 410 | ||
| 411 | (b) PCRE_BUILD_PCREGREP can be see OFF not to build pcregrep, but only | |
| 412 | if PCRE_BUILD_TESTS is also set OFF, because the tests use pcregrep. | |
| 413 | ||
| 414 | 25. Forward references, both numeric and by name, in patterns that made use of | |
| 415 | duplicate group numbers, could behave incorrectly or give incorrect errors, | |
| 416 | because when scanning forward to find the reference group, PCRE was not | |
| 417 | taking into account the duplicate group numbers. A pattern such as | |
| 418 | ^X(?3)(a)(?|(b)|(q))(Y) is an example. | |
| 419 | ||
| 420 | 26. Changed a few more instances of "const unsigned char *" to USPTR, making | |
| 421 | the feature of a custom pointer more persuasive (as requested by a user). | |
| 422 | ||
| 423 | 27. Wrapped the definitions of fileno and isatty for Windows, which appear in | |
| 424 | pcretest.c, inside #ifndefs, because it seems they are sometimes already | |
| 425 | pre-defined. | |
| 426 | ||
| 427 | 28. Added support for (*UTF8) at the start of a pattern. | |
| 428 | ||
| 429 | 29. Arrange for flags added by the "release type" setting in CMake to be shown | |
| 430 | in the configuration summary. | |
| 431 | ||
| 432 | ||
| 433 | Version 7.8 05-Sep-08 | |
| 434 | --------------------- | |
| 435 | ||
| 436 | 1. Replaced UCP searching code with optimized version as implemented for Ad | |
| 437 | Muncher (http://www.admuncher.com/) by Peter Kankowski. This uses a two- | |
| 438 | stage table and inline lookup instead of a function, giving speed ups of 2 | |
| 439 | to 5 times on some simple patterns that I tested. Permission was given to | |
| 440 | distribute the MultiStage2.py script that generates the tables (it's not in | |
| 441 | the tarball, but is in the Subversion repository). | |
| 442 | ||
| 443 | 2. Updated the Unicode datatables to Unicode 5.1.0. This adds yet more | |
| 444 | scripts. | |
| 445 | ||
| 446 | 3. Change 12 for 7.7 introduced a bug in pcre_study() when a pattern contained | |
| 447 | a group with a zero qualifier. The result of the study could be incorrect, | |
| 448 | or the function might crash, depending on the pattern. | |
| 449 | ||
| 450 | 4. Caseless matching was not working for non-ASCII characters in back | |
| 451 | references. For example, /(\x{de})\1/8i was not matching \x{de}\x{fe}. | |
| 452 | It now works when Unicode Property Support is available. | |
| 453 | ||
| 454 | 5. In pcretest, an escape such as \x{de} in the data was always generating | |
| 455 | a UTF-8 string, even in non-UTF-8 mode. Now it generates a single byte in | |
| 456 | non-UTF-8 mode. If the value is greater than 255, it gives a warning about | |
| 457 | truncation. | |
| 458 | ||
| 459 | 6. Minor bugfix in pcrecpp.cc (change "" == ... to NULL == ...). | |
| 460 | ||
| 461 | 7. Added two (int) casts to pcregrep when printing the difference of two | |
| 462 | pointers, in case they are 64-bit values. | |
| 463 | ||
| 464 | 8. Added comments about Mac OS X stack usage to the pcrestack man page and to | |
| 465 | test 2 if it fails. | |
| 466 | ||
| 467 | 9. Added PCRE_CALL_CONVENTION just before the names of all exported functions, | |
| 468 | and a #define of that name to empty if it is not externally set. This is to | |
| 469 | allow users of MSVC to set it if necessary. | |
| 470 | ||
| 471 | 10. The PCRE_EXP_DEFN macro which precedes exported functions was missing from | |
| 472 | the convenience functions in the pcre_get.c source file. | |
| 473 | ||
| 474 | 11. An option change at the start of a pattern that had top-level alternatives | |
| 475 | could cause overwriting and/or a crash. This command provoked a crash in | |
| 476 | some environments: | |
| 477 | ||
| 478 | printf "/(?i)[\xc3\xa9\xc3\xbd]|[\xc3\xa9\xc3\xbdA]/8\n" | pcretest | |
| 479 | ||
| 480 | This potential security problem was recorded as CVE-2008-2371. | |
| 481 | ||
| 482 | 12. For a pattern where the match had to start at the beginning or immediately | |
| 483 | after a newline (e.g /.*anything/ without the DOTALL flag), pcre_exec() and | |
| 484 | pcre_dfa_exec() could read past the end of the passed subject if there was | |
| 485 | no match. To help with detecting such bugs (e.g. with valgrind), I modified | |
| 486 | pcretest so that it places the subject at the end of its malloc-ed buffer. | |
| 487 | ||
| 488 | 13. The change to pcretest in 12 above threw up a couple more cases when pcre_ | |
| 489 | exec() might read past the end of the data buffer in UTF-8 mode. | |
| 490 | ||
| 491 | 14. A similar bug to 7.3/2 existed when the PCRE_FIRSTLINE option was set and | |
| 492 | the data contained the byte 0x85 as part of a UTF-8 character within its | |
| 493 | first line. This applied both to normal and DFA matching. | |
| 494 | ||
| 495 | 15. Lazy qualifiers were not working in some cases in UTF-8 mode. For example, | |
| 496 | /^[^d]*?$/8 failed to match "abc". | |
| 497 | ||
| 498 | 16. Added a missing copyright notice to pcrecpp_internal.h. | |
| 499 | ||
| 500 | 17. Make it more clear in the documentation that values returned from | |
| 501 | pcre_exec() in ovector are byte offsets, not character counts. | |
| 502 | ||
| 503 | 18. Tidied a few places to stop certain compilers from issuing warnings. | |
| 504 | ||
| 505 | 19. Updated the Virtual Pascal + BCC files to compile the latest v7.7, as | |
| 506 | supplied by Stefan Weber. I made a further small update for 7.8 because | |
| 507 | there is a change of source arrangements: the pcre_searchfuncs.c module is | |
| 508 | replaced by pcre_ucd.c. | |
| 509 | ||
| 510 | ||
| 511 | Version 7.7 07-May-08 | |
| 512 | --------------------- | |
| 513 | ||
| 514 | 1. Applied Craig's patch to sort out a long long problem: "If we can't convert | |
| 515 | a string to a long long, pretend we don't even have a long long." This is | |
| 516 | done by checking for the strtoq, strtoll, and _strtoi64 functions. | |
| 517 | ||
| 518 | 2. Applied Craig's patch to pcrecpp.cc to restore ABI compatibility with | |
| 519 | pre-7.6 versions, which defined a global no_arg variable instead of putting | |
| 520 | it in the RE class. (See also #8 below.) | |
| 521 | ||
| 522 | 3. Remove a line of dead code, identified by coverity and reported by Nuno | |
| 523 | Lopes. | |
| 524 | ||
| 525 | 4. Fixed two related pcregrep bugs involving -r with --include or --exclude: | |
| 526 | ||
| 527 | (1) The include/exclude patterns were being applied to the whole pathnames | |
| 528 | of files, instead of just to the final components. | |
| 529 | ||
| 530 | (2) If there was more than one level of directory, the subdirectories were | |
| 531 | skipped unless they satisfied the include/exclude conditions. This is | |
| 532 | inconsistent with GNU grep (and could even be seen as contrary to the | |
| 533 | pcregrep specification - which I improved to make it absolutely clear). | |
| 534 | The action now is always to scan all levels of directory, and just | |
| 535 | apply the include/exclude patterns to regular files. | |
| 536 | ||
| 537 | 5. Added the --include_dir and --exclude_dir patterns to pcregrep, and used | |
| 538 | --exclude_dir in the tests to avoid scanning .svn directories. | |
| 539 | ||
| 540 | 6. Applied Craig's patch to the QuoteMeta function so that it escapes the | |
| 541 | NUL character as backslash + 0 rather than backslash + NUL, because PCRE | |
| 542 | doesn't support NULs in patterns. | |
| 543 | ||
| 544 | 7. Added some missing "const"s to declarations of static tables in | |
| 545 | pcre_compile.c and pcre_dfa_exec.c. | |
| 546 | ||
| 547 | 8. Applied Craig's patch to pcrecpp.cc to fix a problem in OS X that was | |
| 548 | caused by fix #2 above. (Subsequently also a second patch to fix the | |
| 549 | first patch. And a third patch - this was a messy problem.) | |
| 550 | ||
| 551 | 9. Applied Craig's patch to remove the use of push_back(). | |
| 552 | ||
| 553 | 10. Applied Alan Lehotsky's patch to add REG_STARTEND support to the POSIX | |
| 554 | matching function regexec(). | |
| 555 | ||
| 556 | 11. Added support for the Oniguruma syntax \g<name>, \g<n>, \g'name', \g'n', | |
| 557 | which, however, unlike Perl's \g{...}, are subroutine calls, not back | |
| 558 | references. PCRE supports relative numbers with this syntax (I don't think | |
| 559 | Oniguruma does). | |
| 560 | ||
| 561 | 12. Previously, a group with a zero repeat such as (...){0} was completely | |
| 562 | omitted from the compiled regex. However, this means that if the group | |
| 563 | was called as a subroutine from elsewhere in the pattern, things went wrong | |
| 564 | (an internal error was given). Such groups are now left in the compiled | |
| 565 | pattern, with a new opcode that causes them to be skipped at execution | |
| 566 | time. | |
| 567 | ||
| 568 | 13. Added the PCRE_JAVASCRIPT_COMPAT option. This makes the following changes | |
| 569 | to the way PCRE behaves: | |
| 570 | ||
| 571 | (a) A lone ] character is dis-allowed (Perl treats it as data). | |
| 572 | ||
| 573 | (b) A back reference to an unmatched subpattern matches an empty string | |
| 574 | (Perl fails the current match path). | |
| 575 | ||
| 576 | (c) A data ] in a character class must be notated as \] because if the | |
| 577 | first data character in a class is ], it defines an empty class. (In | |
| 578 | Perl it is not possible to have an empty class.) The empty class [] | |
| 579 | never matches; it forces failure and is equivalent to (*FAIL) or (?!). | |
| 580 | The negative empty class [^] matches any one character, independently | |
| 581 | of the DOTALL setting. | |
| 582 | ||
| 583 | 14. A pattern such as /(?2)[]a()b](abc)/ which had a forward reference to a | |
| 584 | non-existent subpattern following a character class starting with ']' and | |
| 585 | containing () gave an internal compiling error instead of "reference to | |
| 586 | non-existent subpattern". Fortunately, when the pattern did exist, the | |
| 587 | compiled code was correct. (When scanning forwards to check for the | |
| 588 | existencd of the subpattern, it was treating the data ']' as terminating | |
| 589 | the class, so got the count wrong. When actually compiling, the reference | |
| 590 | was subsequently set up correctly.) | |
| 591 | ||
| 592 | 15. The "always fail" assertion (?!) is optimzed to (*FAIL) by pcre_compile; | |
| 593 | it was being rejected as not supported by pcre_dfa_exec(), even though | |
| 594 | other assertions are supported. I have made pcre_dfa_exec() support | |
| 595 | (*FAIL). | |
| 596 | ||
| 597 | 16. The implementation of 13c above involved the invention of a new opcode, | |
| 598 | OP_ALLANY, which is like OP_ANY but doesn't check the /s flag. Since /s | |
| 599 | cannot be changed at match time, I realized I could make a small | |
| 600 | improvement to matching performance by compiling OP_ALLANY instead of | |
| 601 | OP_ANY for "." when DOTALL was set, and then removing the runtime tests | |
| 602 | on the OP_ANY path. | |
| 603 | ||
| 604 | 17. Compiling pcretest on Windows with readline support failed without the | |
| 605 | following two fixes: (1) Make the unistd.h include conditional on | |
| 606 | HAVE_UNISTD_H; (2) #define isatty and fileno as _isatty and _fileno. | |
| 607 | ||
| 608 | 18. Changed CMakeLists.txt and cmake/FindReadline.cmake to arrange for the | |
| 609 | ncurses library to be included for pcretest when ReadLine support is | |
| 610 | requested, but also to allow for it to be overridden. This patch came from | |
| 611 | Daniel Bergström. | |
| 612 | ||
| 613 | 19. There was a typo in the file ucpinternal.h where f0_rangeflag was defined | |
| 614 | as 0x00f00000 instead of 0x00800000. Luckily, this would not have caused | |
| 615 | any errors with the current Unicode tables. Thanks to Peter Kankowski for | |
| 616 | spotting this. | |
| 617 | ||
| 618 | ||
| 619 | Version 7.6 28-Jan-08 | |
| 620 | --------------------- | |
| 621 | ||
| 622 | 1. A character class containing a very large number of characters with | |
| 623 | codepoints greater than 255 (in UTF-8 mode, of course) caused a buffer | |
| 624 | overflow. | |
| 625 | ||
| 626 | 2. Patch to cut out the "long long" test in pcrecpp_unittest when | |
| 627 | HAVE_LONG_LONG is not defined. | |
| 628 | ||
| 629 | 3. Applied Christian Ehrlicher's patch to update the CMake build files to | |
| 630 | bring them up to date and include new features. This patch includes: | |
| 631 | ||
| 632 | - Fixed PH's badly added libz and libbz2 support. | |
| 633 | - Fixed a problem with static linking. | |
| 634 | - Added pcredemo. [But later removed - see 7 below.] | |
| 635 | - Fixed dftables problem and added an option. | |
| 636 | - Added a number of HAVE_XXX tests, including HAVE_WINDOWS_H and | |
| 637 | HAVE_LONG_LONG. | |
| 638 | - Added readline support for pcretest. | |
| 639 | - Added an listing of the option settings after cmake has run. | |
| 640 | ||
| 641 | 4. A user submitted a patch to Makefile that makes it easy to create | |
| 642 | "pcre.dll" under mingw when using Configure/Make. I added stuff to | |
| 643 | Makefile.am that cause it to include this special target, without | |
| 644 | affecting anything else. Note that the same mingw target plus all | |
| 645 | the other distribution libraries and programs are now supported | |
| 646 | when configuring with CMake (see 6 below) instead of with | |
| 647 | Configure/Make. | |
| 648 | ||
| 649 | 5. Applied Craig's patch that moves no_arg into the RE class in the C++ code. | |
| 650 | This is an attempt to solve the reported problem "pcrecpp::no_arg is not | |
| 651 | exported in the Windows port". It has not yet been confirmed that the patch | |
| 652 | solves the problem, but it does no harm. | |
| 653 | ||
| 654 | 6. Applied Sheri's patch to CMakeLists.txt to add NON_STANDARD_LIB_PREFIX and | |
| 655 | NON_STANDARD_LIB_SUFFIX for dll names built with mingw when configured | |
| 656 | with CMake, and also correct the comment about stack recursion. | |
| 657 | ||
| 658 | 7. Remove the automatic building of pcredemo from the ./configure system and | |
| 659 | from CMakeLists.txt. The whole idea of pcredemo.c is that it is an example | |
| 660 | of a program that users should build themselves after PCRE is installed, so | |
| 661 | building it automatically is not really right. What is more, it gave | |
| 662 | trouble in some build environments. | |
| 663 | ||
| 664 | 8. Further tidies to CMakeLists.txt from Sheri and Christian. | |
| 665 | ||
| 666 | ||
| 667 | Version 7.5 10-Jan-08 | |
| 668 | --------------------- | |
| 669 | ||
| 670 | 1. Applied a patch from Craig: "This patch makes it possible to 'ignore' | |
| 671 | values in parens when parsing an RE using the C++ wrapper." | |
| 672 | ||
| 673 | 2. Negative specials like \S did not work in character classes in UTF-8 mode. | |
| 674 | Characters greater than 255 were excluded from the class instead of being | |
| 675 | included. | |
| 676 | ||
| 677 | 3. The same bug as (2) above applied to negated POSIX classes such as | |
| 678 | [:^space:]. | |
| 679 | ||
| 680 | 4. PCRECPP_STATIC was referenced in pcrecpp_internal.h, but nowhere was it | |
| 681 | defined or documented. It seems to have been a typo for PCRE_STATIC, so | |
| 682 | I have changed it. | |
| 683 | ||
| 684 | 5. The construct (?&) was not diagnosed as a syntax error (it referenced the | |
| 685 | first named subpattern) and a construct such as (?&a) would reference the | |
| 686 | first named subpattern whose name started with "a" (in other words, the | |
| 687 | length check was missing). Both these problems are fixed. "Subpattern name | |
| 688 | expected" is now given for (?&) (a zero-length name), and this patch also | |
| 689 | makes it give the same error for \k'' (previously it complained that that | |
| 690 | was a reference to a non-existent subpattern). | |
| 691 | ||
| 692 | 6. The erroneous patterns (?+-a) and (?-+a) give different error messages; | |
| 693 | this is right because (?- can be followed by option settings as well as by | |
| 694 | digits. I have, however, made the messages clearer. | |
| 695 | ||
| 696 | 7. Patterns such as (?(1)a|b) (a pattern that contains fewer subpatterns | |
| 697 | than the number used in the conditional) now cause a compile-time error. | |
| 698 | This is actually not compatible with Perl, which accepts such patterns, but | |
| 699 | treats the conditional as always being FALSE (as PCRE used to), but it | |
| 700 | seems to me that giving a diagnostic is better. | |
| 701 | ||
| 702 | 8. Change "alphameric" to the more common word "alphanumeric" in comments | |
| 703 | and messages. | |
| 704 | ||
| 705 | 9. Fix two occurrences of "backslash" in comments that should have been | |
| 706 | "backspace". | |
| 707 | ||
| 708 | 10. Remove two redundant lines of code that can never be obeyed (their function | |
| 709 | was moved elsewhere). | |
| 710 | ||
| 711 | 11. The program that makes PCRE's Unicode character property table had a bug | |
| 712 | which caused it to generate incorrect table entries for sequences of | |
| 713 | characters that have the same character type, but are in different scripts. | |
| 714 | It amalgamated them into a single range, with the script of the first of | |
| 715 | them. In other words, some characters were in the wrong script. There were | |
| 716 | thirteen such cases, affecting characters in the following ranges: | |
| 717 | ||
| 718 | U+002b0 - U+002c1 | |
| 719 | U+0060c - U+0060d | |
| 720 | U+0061e - U+00612 | |
| 721 | U+0064b - U+0065e | |
| 722 | U+0074d - U+0076d | |
| 723 | U+01800 - U+01805 | |
| 724 | U+01d00 - U+01d77 | |
| 725 | U+01d9b - U+01dbf | |
| 726 | U+0200b - U+0200f | |
| 727 | U+030fc - U+030fe | |
| 728 | U+03260 - U+0327f | |
| 729 | U+0fb46 - U+0fbb1 | |
| 730 | U+10450 - U+1049d | |
| 731 | ||
| 732 | 12. The -o option (show only the matching part of a line) for pcregrep was not | |
| 733 | compatible with GNU grep in that, if there was more than one match in a | |
| 734 | line, it showed only the first of them. It now behaves in the same way as | |
| 735 | GNU grep. | |
| 736 | ||
| 737 | 13. If the -o and -v options were combined for pcregrep, it printed a blank | |
| 738 | line for every non-matching line. GNU grep prints nothing, and pcregrep now | |
| 739 | does the same. The return code can be used to tell if there were any | |
| 740 | non-matching lines. | |
| 741 | ||
| 742 | 14. Added --file-offsets and --line-offsets to pcregrep. | |
| 743 | ||
| 744 | 15. The pattern (?=something)(?R) was not being diagnosed as a potentially | |
| 745 | infinitely looping recursion. The bug was that positive lookaheads were not | |
| 746 | being skipped when checking for a possible empty match (negative lookaheads | |
| 747 | and both kinds of lookbehind were skipped). | |
| 748 | ||
| 749 | 16. Fixed two typos in the Windows-only code in pcregrep.c, and moved the | |
| 750 | inclusion of <windows.h> to before rather than after the definition of | |
| 751 | INVALID_FILE_ATTRIBUTES (patch from David Byron). | |
| 752 | ||
| 753 | 17. Specifying a possessive quantifier with a specific limit for a Unicode | |
| 754 | character property caused pcre_compile() to compile bad code, which led at | |
| 755 | runtime to PCRE_ERROR_INTERNAL (-14). Examples of patterns that caused this | |
| 756 | are: /\p{Zl}{2,3}+/8 and /\p{Cc}{2}+/8. It was the possessive "+" that | |
| 757 | caused the error; without that there was no problem. | |
| 758 | ||
| 759 | 18. Added --enable-pcregrep-libz and --enable-pcregrep-libbz2. | |
| 760 | ||
| 761 | 19. Added --enable-pcretest-libreadline. | |
| 762 | ||
| 763 | 20. In pcrecpp.cc, the variable 'count' was incremented twice in | |
| 764 | RE::GlobalReplace(). As a result, the number of replacements returned was | |
| 765 | double what it should be. I removed one of the increments, but Craig sent a | |
| 766 | later patch that removed the other one (the right fix) and added unit tests | |
| 767 | that check the return values (which was not done before). | |
| 768 | ||
| 769 | 21. Several CMake things: | |
| 770 | ||
| 771 | (1) Arranged that, when cmake is used on Unix, the libraries end up with | |
| 772 | the names libpcre and libpcreposix, not just pcre and pcreposix. | |
| 773 | ||
| 774 | (2) The above change means that pcretest and pcregrep are now correctly | |
| 775 | linked with the newly-built libraries, not previously installed ones. | |
| 776 | ||
| 777 | (3) Added PCRE_SUPPORT_LIBREADLINE, PCRE_SUPPORT_LIBZ, PCRE_SUPPORT_LIBBZ2. | |
| 778 | ||
| 779 | 22. In UTF-8 mode, with newline set to "any", a pattern such as .*a.*=.b.* | |
| 780 | crashed when matching a string such as a\x{2029}b (note that \x{2029} is a | |
| 781 | UTF-8 newline character). The key issue is that the pattern starts .*; | |
| 782 | this means that the match must be either at the beginning, or after a | |
| 783 | newline. The bug was in the code for advancing after a failed match and | |
| 784 | checking that the new position followed a newline. It was not taking | |
| 785 | account of UTF-8 characters correctly. | |
| 786 | ||
| 787 | 23. PCRE was behaving differently from Perl in the way it recognized POSIX | |
| 788 | character classes. PCRE was not treating the sequence [:...:] as a | |
| 789 | character class unless the ... were all letters. Perl, however, seems to | |
| 790 | allow any characters between [: and :], though of course it rejects as | |
| 791 | unknown any "names" that contain non-letters, because all the known class | |
| 792 | names consist only of letters. Thus, Perl gives an error for [[:1234:]], | |
| 793 | for example, whereas PCRE did not - it did not recognize a POSIX character | |
| 794 | class. This seemed a bit dangerous, so the code has been changed to be | |
| 795 | closer to Perl. The behaviour is not identical to Perl, because PCRE will | |
| 796 | diagnose an unknown class for, for example, [[:l\ower:]] where Perl will | |
| 797 | treat it as [[:lower:]]. However, PCRE does now give "unknown" errors where | |
| 798 | Perl does, and where it didn't before. | |
| 799 | ||
| 800 | 24. Rewrite so as to remove the single use of %n from pcregrep because in some | |
| 801 | Windows environments %n is disabled by default. | |
| 802 | ||
| 803 | ||
| 804 | Version 7.4 21-Sep-07 | |
| 805 | --------------------- | |
| 806 | ||
| 807 | 1. Change 7.3/28 was implemented for classes by looking at the bitmap. This | |
| 808 | means that a class such as [\s] counted as "explicit reference to CR or | |
| 809 | LF". That isn't really right - the whole point of the change was to try to | |
| 810 | help when there was an actual mention of one of the two characters. So now | |
| 811 | the change happens only if \r or \n (or a literal CR or LF) character is | |
| 812 | encountered. | |
| 813 | ||
| 814 | 2. The 32-bit options word was also used for 6 internal flags, but the numbers | |
| 815 | of both had grown to the point where there were only 3 bits left. | |
| 816 | Fortunately, there was spare space in the data structure, and so I have | |
| 817 | moved the internal flags into a new 16-bit field to free up more option | |
| 818 | bits. | |
| 819 | ||
| 820 | 3. The appearance of (?J) at the start of a pattern set the DUPNAMES option, | |
| 821 | but did not set the internal JCHANGED flag - either of these is enough to | |
| 822 | control the way the "get" function works - but the PCRE_INFO_JCHANGED | |
| 823 | facility is supposed to tell if (?J) was ever used, so now (?J) at the | |
| 824 | start sets both bits. | |
| 825 | ||
| 826 | 4. Added options (at build time, compile time, exec time) to change \R from | |
| 827 | matching any Unicode line ending sequence to just matching CR, LF, or CRLF. | |
| 828 | ||
| 829 | 5. doc/pcresyntax.html was missing from the distribution. | |
| 830 | ||
| 831 | 6. Put back the definition of PCRE_ERROR_NULLWSLIMIT, for backward | |
| 832 | compatibility, even though it is no longer used. | |
| 833 | ||
| 834 | 7. Added macro for snprintf to pcrecpp_unittest.cc and also for strtoll and | |
| 835 | strtoull to pcrecpp.cc to select the available functions in WIN32 when the | |
| 836 | windows.h file is present (where different names are used). [This was | |
| 837 | reversed later after testing - see 16 below.] | |
| 838 | ||
| 839 | 8. Changed all #include <config.h> to #include "config.h". There were also | |
| 840 | some further <pcre.h> cases that I changed to "pcre.h". | |
| 841 | ||
| 842 | 9. When pcregrep was used with the --colour option, it missed the line ending | |
| 843 | sequence off the lines that it output. | |
| 844 | ||
| 845 | 10. It was pointed out to me that arrays of string pointers cause lots of | |
| 846 | relocations when a shared library is dynamically loaded. A technique of | |
| 847 | using a single long string with a table of offsets can drastically reduce | |
| 848 | these. I have refactored PCRE in four places to do this. The result is | |
| 849 | dramatic: | |
| 850 | ||
| 851 | Originally: 290 | |
| 852 | After changing UCP table: 187 | |
| 853 | After changing error message table: 43 | |
| 854 | After changing table of "verbs" 36 | |
| 855 | After changing table of Posix names 22 | |
| 856 | ||
| 857 | Thanks to the folks working on Gregex for glib for this insight. | |
| 858 | ||
| 859 | 11. --disable-stack-for-recursion caused compiling to fail unless -enable- | |
| 860 | unicode-properties was also set. | |
| 861 | ||
| 862 | 12. Updated the tests so that they work when \R is defaulted to ANYCRLF. | |
| 863 | ||
| 864 | 13. Added checks for ANY and ANYCRLF to pcrecpp.cc where it previously | |
| 865 | checked only for CRLF. | |
| 866 | ||
| 867 | 14. Added casts to pcretest.c to avoid compiler warnings. | |
| 868 | ||
| 869 | 15. Added Craig's patch to various pcrecpp modules to avoid compiler warnings. | |
| 870 | ||
| 871 | 16. Added Craig's patch to remove the WINDOWS_H tests, that were not working, | |
| 872 | and instead check for _strtoi64 explicitly, and avoid the use of snprintf() | |
| 873 | entirely. This removes changes made in 7 above. | |
| 874 | ||
| 875 | 17. The CMake files have been updated, and there is now more information about | |
| 876 | building with CMake in the NON-UNIX-USE document. | |
| 877 | ||
| 878 | ||
| 879 | Version 7.3 28-Aug-07 | |
| 880 | --------------------- | |
| 881 | ||
| 882 | 1. In the rejigging of the build system that eventually resulted in 7.1, the | |
| 883 | line "#include <pcre.h>" was included in pcre_internal.h. The use of angle | |
| 884 | brackets there is not right, since it causes compilers to look for an | |
| 885 | installed pcre.h, not the version that is in the source that is being | |
| 886 | compiled (which of course may be different). I have changed it back to: | |
| 887 | ||
| 888 | #include "pcre.h" | |
| 889 | ||
| 890 | I have a vague recollection that the change was concerned with compiling in | |
| 891 | different directories, but in the new build system, that is taken care of | |
| 892 | by the VPATH setting the Makefile. | |
| 893 | ||
| 894 | 2. The pattern .*$ when run in not-DOTALL UTF-8 mode with newline=any failed | |
| 895 | when the subject happened to end in the byte 0x85 (e.g. if the last | |
| 896 | character was \x{1ec5}). *Character* 0x85 is one of the "any" newline | |
| 897 | characters but of course it shouldn't be taken as a newline when it is part | |
| 898 | of another character. The bug was that, for an unlimited repeat of . in | |
| 899 | not-DOTALL UTF-8 mode, PCRE was advancing by bytes rather than by | |
| 900 | characters when looking for a newline. | |
| 901 | ||
| 902 | 3. A small performance improvement in the DOTALL UTF-8 mode .* case. | |
| 903 | ||
| 904 | 4. Debugging: adjusted the names of opcodes for different kinds of parentheses | |
| 905 | in debug output. | |
| 906 | ||
| 907 | 5. Arrange to use "%I64d" instead of "%lld" and "%I64u" instead of "%llu" for | |
| 908 | long printing in the pcrecpp unittest when running under MinGW. | |
| 909 | ||
| 910 | 6. ESC_K was left out of the EBCDIC table. | |
| 911 | ||
| 912 | 7. Change 7.0/38 introduced a new limit on the number of nested non-capturing | |
| 913 | parentheses; I made it 1000, which seemed large enough. Unfortunately, the | |
| 914 | limit also applies to "virtual nesting" when a pattern is recursive, and in | |
| 915 | this case 1000 isn't so big. I have been able to remove this limit at the | |
| 916 | expense of backing off one optimization in certain circumstances. Normally, | |
| 917 | when pcre_exec() would call its internal match() function recursively and | |
| 918 | immediately return the result unconditionally, it uses a "tail recursion" | |
| 919 | feature to save stack. However, when a subpattern that can match an empty | |
| 920 | string has an unlimited repetition quantifier, it no longer makes this | |
| 921 | optimization. That gives it a stack frame in which to save the data for | |
| 922 | checking that an empty string has been matched. Previously this was taken | |
| 923 | from the 1000-entry workspace that had been reserved. So now there is no | |
| 924 | explicit limit, but more stack is used. | |
| 925 | ||
| 926 | 8. Applied Daniel's patches to solve problems with the import/export magic | |
| 927 | syntax that is required for Windows, and which was going wrong for the | |
| 928 | pcreposix and pcrecpp parts of the library. These were overlooked when this | |
| 929 | problem was solved for the main library. | |
| 930 | ||
| 931 | 9. There were some crude static tests to avoid integer overflow when computing | |
| 932 | the size of patterns that contain repeated groups with explicit upper | |
| 933 | limits. As the maximum quantifier is 65535, the maximum group length was | |
| 934 | set at 30,000 so that the product of these two numbers did not overflow a | |
| 935 | 32-bit integer. However, it turns out that people want to use groups that | |
| 936 | are longer than 30,000 bytes (though not repeat them that many times). | |
| 937 | Change 7.0/17 (the refactoring of the way the pattern size is computed) has | |
| 938 | made it possible to implement the integer overflow checks in a much more | |
| 939 | dynamic way, which I have now done. The artificial limitation on group | |
| 940 | length has been removed - we now have only the limit on the total length of | |
| 941 | the compiled pattern, which depends on the LINK_SIZE setting. | |
| 942 | ||
| 943 | 10. Fixed a bug in the documentation for get/copy named substring when | |
| 944 | duplicate names are permitted. If none of the named substrings are set, the | |
| 945 | functions return PCRE_ERROR_NOSUBSTRING (7); the doc said they returned an | |
| 946 | empty string. | |
| 947 | ||
| 948 | 11. Because Perl interprets \Q...\E at a high level, and ignores orphan \E | |
| 949 | instances, patterns such as [\Q\E] or [\E] or even [^\E] cause an error, | |
| 950 | because the ] is interpreted as the first data character and the | |
| 951 | terminating ] is not found. PCRE has been made compatible with Perl in this | |
| 952 | regard. Previously, it interpreted [\Q\E] as an empty class, and [\E] could | |
| 953 | cause memory overwriting. | |
| 954 | ||
| 955 | 10. Like Perl, PCRE automatically breaks an unlimited repeat after an empty | |
| 956 | string has been matched (to stop an infinite loop). It was not recognizing | |
| 957 | a conditional subpattern that could match an empty string if that | |
| 958 | subpattern was within another subpattern. For example, it looped when | |
| 959 | trying to match (((?(1)X|))*) but it was OK with ((?(1)X|)*) where the | |
| 960 | condition was not nested. This bug has been fixed. | |
| 961 | ||
| 962 | 12. A pattern like \X?\d or \P{L}?\d in non-UTF-8 mode could cause a backtrack | |
| 963 | past the start of the subject in the presence of bytes with the top bit | |
| 964 | set, for example "\x8aBCD". | |
| 965 | ||
| 966 | 13. Added Perl 5.10 experimental backtracking controls (*FAIL), (*F), (*PRUNE), | |
| 967 | (*SKIP), (*THEN), (*COMMIT), and (*ACCEPT). | |
| 968 | ||
| 969 | 14. Optimized (?!) to (*FAIL). | |
| 970 | ||
| 971 | 15. Updated the test for a valid UTF-8 string to conform to the later RFC 3629. | |
| 972 | This restricts code points to be within the range 0 to 0x10FFFF, excluding | |
| 973 | the "low surrogate" sequence 0xD800 to 0xDFFF. Previously, PCRE allowed the | |
| 974 | full range 0 to 0x7FFFFFFF, as defined by RFC 2279. Internally, it still | |
| 975 | does: it's just the validity check that is more restrictive. | |
| 976 | ||
| 977 | 16. Inserted checks for integer overflows during escape sequence (backslash) | |
| 978 | processing, and also fixed erroneous offset values for syntax errors during | |
| 979 | backslash processing. | |
| 980 | ||
| 981 | 17. Fixed another case of looking too far back in non-UTF-8 mode (cf 12 above) | |
| 982 | for patterns like [\PPP\x8a]{1,}\x80 with the subject "A\x80". | |
| 983 | ||
| 984 | 18. An unterminated class in a pattern like (?1)\c[ with a "forward reference" | |
| 985 | caused an overrun. | |
| 986 | ||
| 987 | 19. A pattern like (?:[\PPa*]*){8,} which had an "extended class" (one with | |
| 988 | something other than just ASCII characters) inside a group that had an | |
| 989 | unlimited repeat caused a loop at compile time (while checking to see | |
| 990 | whether the group could match an empty string). | |
| 991 | ||
| 992 | 20. Debugging a pattern containing \p or \P could cause a crash. For example, | |
| 993 | [\P{Any}] did so. (Error in the code for printing property names.) | |
| 994 | ||
| 995 | 21. An orphan \E inside a character class could cause a crash. | |
| 996 | ||
| 997 | 22. A repeated capturing bracket such as (A)? could cause a wild memory | |
| 998 | reference during compilation. | |
| 999 | ||
| 1000 | 23. There are several functions in pcre_compile() that scan along a compiled | |
| 1001 | expression for various reasons (e.g. to see if it's fixed length for look | |
| 1002 | behind). There were bugs in these functions when a repeated \p or \P was | |
| 1003 | present in the pattern. These operators have additional parameters compared | |
| 1004 | with \d, etc, and these were not being taken into account when moving along | |
| 1005 | the compiled data. Specifically: | |
| 1006 | ||
| 1007 | (a) A item such as \p{Yi}{3} in a lookbehind was not treated as fixed | |
| 1008 | length. | |
| 1009 | ||
| 1010 | (b) An item such as \pL+ within a repeated group could cause crashes or | |
| 1011 | loops. | |
| 1012 | ||
| 1013 | (c) A pattern such as \p{Yi}+(\P{Yi}+)(?1) could give an incorrect | |
| 1014 | "reference to non-existent subpattern" error. | |
| 1015 | ||
| 1016 | (d) A pattern like (\P{Yi}{2}\277)? could loop at compile time. | |
| 1017 | ||
| 1018 | 24. A repeated \S or \W in UTF-8 mode could give wrong answers when multibyte | |
| 1019 | characters were involved (for example /\S{2}/8g with "A\x{a3}BC"). | |
| 1020 | ||
| 1021 | 25. Using pcregrep in multiline, inverted mode (-Mv) caused it to loop. | |
| 1022 | ||
| 1023 | 26. Patterns such as [\P{Yi}A] which include \p or \P and just one other | |
| 1024 | character were causing crashes (broken optimization). | |
| 1025 | ||
| 1026 | 27. Patterns such as (\P{Yi}*\277)* (group with possible zero repeat containing | |
| 1027 | \p or \P) caused a compile-time loop. | |
| 1028 | ||
| 1029 | 28. More problems have arisen in unanchored patterns when CRLF is a valid line | |
| 1030 | break. For example, the unstudied pattern [\r\n]A does not match the string | |
| 1031 | "\r\nA" because change 7.0/46 below moves the current point on by two | |
| 1032 | characters after failing to match at the start. However, the pattern \nA | |
| 1033 | *does* match, because it doesn't start till \n, and if [\r\n]A is studied, | |
| 1034 | the same is true. There doesn't seem any very clean way out of this, but | |
| 1035 | what I have chosen to do makes the common cases work: PCRE now takes note | |
| 1036 | of whether there can be an explicit match for \r or \n anywhere in the | |
| 1037 | pattern, and if so, 7.0/46 no longer applies. As part of this change, | |
| 1038 | there's a new PCRE_INFO_HASCRORLF option for finding out whether a compiled | |
| 1039 | pattern has explicit CR or LF references. | |
| 1040 | ||
| 1041 | 29. Added (*CR) etc for changing newline setting at start of pattern. | |
| 1042 | ||
| 1043 | ||
| 1044 | Version 7.2 19-Jun-07 | |
| 1045 | --------------------- | |
| 1046 | ||
| 1047 | 1. If the fr_FR locale cannot be found for test 3, try the "french" locale, | |
| 1048 | which is apparently normally available under Windows. | |
| 1049 | ||
| 1050 | 2. Re-jig the pcregrep tests with different newline settings in an attempt | |
| 1051 | to make them independent of the local environment's newline setting. | |
| 1052 | ||
| 1053 | 3. Add code to configure.ac to remove -g from the CFLAGS default settings. | |
| 1054 | ||
| 1055 | 4. Some of the "internals" tests were previously cut out when the link size | |
| 1056 | was not 2, because the output contained actual offsets. The recent new | |
| 1057 | "Z" feature of pcretest means that these can be cut out, making the tests | |
| 1058 | usable with all link sizes. | |
| 1059 | ||
| 1060 | 5. Implemented Stan Switzer's goto replacement for longjmp() when not using | |
| 1061 | stack recursion. This gives a massive performance boost under BSD, but just | |
| 1062 | a small improvement under Linux. However, it saves one field in the frame | |
| 1063 | in all cases. | |
| 1064 | ||
| 1065 | 6. Added more features from the forthcoming Perl 5.10: | |
| 1066 | ||
| 1067 | (a) (?-n) (where n is a string of digits) is a relative subroutine or | |
| 1068 | recursion call. It refers to the nth most recently opened parentheses. | |
| 1069 | ||
| 1070 | (b) (?+n) is also a relative subroutine call; it refers to the nth next | |
| 1071 | to be opened parentheses. | |
| 1072 | ||
| 1073 | (c) Conditions that refer to capturing parentheses can be specified | |
| 1074 | relatively, for example, (?(-2)... or (?(+3)... | |
| 1075 | ||
| 1076 | (d) \K resets the start of the current match so that everything before | |
| 1077 | is not part of it. | |
| 1078 | ||
| 1079 | (e) \k{name} is synonymous with \k<name> and \k'name' (.NET compatible). | |
| 1080 | ||
| 1081 | (f) \g{name} is another synonym - part of Perl 5.10's unification of | |
| 1082 | reference syntax. | |
| 1083 | ||
| 1084 | (g) (?| introduces a group in which the numbering of parentheses in each | |
| 1085 | alternative starts with the same number. | |
| 1086 | ||
| 1087 | (h) \h, \H, \v, and \V match horizontal and vertical whitespace. | |
| 1088 | ||
| 1089 | 7. Added two new calls to pcre_fullinfo(): PCRE_INFO_OKPARTIAL and | |
| 1090 | PCRE_INFO_JCHANGED. | |
| 1091 | ||
| 1092 | 8. A pattern such as (.*(.)?)* caused pcre_exec() to fail by either not | |
| 1093 | terminating or by crashing. Diagnosed by Viktor Griph; it was in the code | |
| 1094 | for detecting groups that can match an empty string. | |
| 1095 | ||
| 1096 | 9. A pattern with a very large number of alternatives (more than several | |
| 1097 | hundred) was running out of internal workspace during the pre-compile | |
| 1098 | phase, where pcre_compile() figures out how much memory will be needed. A | |
| 1099 | bit of new cunning has reduced the workspace needed for groups with | |
| 1100 | alternatives. The 1000-alternative test pattern now uses 12 bytes of | |
| 1101 | workspace instead of running out of the 4096 that are available. | |
| 1102 | ||
| 1103 | 10. Inserted some missing (unsigned int) casts to get rid of compiler warnings. | |
| 1104 | ||
| 1105 | 11. Applied patch from Google to remove an optimization that didn't quite work. | |
| 1106 | The report of the bug said: | |
| 1107 | ||
| 1108 | pcrecpp::RE("a*").FullMatch("aaa") matches, while | |
| 1109 | pcrecpp::RE("a*?").FullMatch("aaa") does not, and | |
| 1110 | pcrecpp::RE("a*?\\z").FullMatch("aaa") does again. | |
| 1111 | ||
| 1112 | 12. If \p or \P was used in non-UTF-8 mode on a character greater than 127 | |
| 1113 | it matched the wrong number of bytes. | |
| 1114 | ||
| 1115 | ||
| 1116 | Version 7.1 24-Apr-07 | |
| 1117 | --------------------- | |
| 1118 | ||
| 1119 | 1. Applied Bob Rossi and Daniel G's patches to convert the build system to one | |
| 1120 | that is more "standard", making use of automake and other Autotools. There | |
| 1121 | is some re-arrangement of the files and adjustment of comments consequent | |
| 1122 | on this. | |
| 1123 | ||
| 1124 | 2. Part of the patch fixed a problem with the pcregrep tests. The test of -r | |
| 1125 | for recursive directory scanning broke on some systems because the files | |
| 1126 | are not scanned in any specific order and on different systems the order | |
| 1127 | was different. A call to "sort" has been inserted into RunGrepTest for the | |
| 1128 | approprate test as a short-term fix. In the longer term there may be an | |
| 1129 | alternative. | |
| 1130 | ||
| 1131 | 3. I had an email from Eric Raymond about problems translating some of PCRE's | |
| 1132 | man pages to HTML (despite the fact that I distribute HTML pages, some | |
| 1133 | people do their own conversions for various reasons). The problems | |
| 1134 | concerned the use of low-level troff macros .br and .in. I have therefore | |
| 1135 | removed all such uses from the man pages (some were redundant, some could | |
| 1136 | be replaced by .nf/.fi pairs). The 132html script that I use to generate | |
| 1137 | HTML has been updated to handle .nf/.fi and to complain if it encounters | |
| 1138 | .br or .in. | |
| 1139 | ||
| 1140 | 4. Updated comments in configure.ac that get placed in config.h.in and also | |
| 1141 | arranged for config.h to be included in the distribution, with the name | |
| 1142 | config.h.generic, for the benefit of those who have to compile without | |
| 1143 | Autotools (compare pcre.h, which is now distributed as pcre.h.generic). | |
| 1144 | ||
| 1145 | 5. Updated the support (such as it is) for Virtual Pascal, thanks to Stefan | |
| 1146 | Weber: (1) pcre_internal.h was missing some function renames; (2) updated | |
| 1147 | makevp.bat for the current PCRE, using the additional files | |
| 1148 | makevp_c.txt, makevp_l.txt, and pcregexp.pas. | |
| 1149 | ||
| 1150 | 6. A Windows user reported a minor discrepancy with test 2, which turned out | |
| 1151 | to be caused by a trailing space on an input line that had got lost in his | |
| 1152 | copy. The trailing space was an accident, so I've just removed it. | |
| 1153 | ||
| 1154 | 7. Add -Wl,-R... flags in pcre-config.in for *BSD* systems, as I'm told | |
| 1155 | that is needed. | |
| 1156 | ||
| 1157 | 8. Mark ucp_table (in ucptable.h) and ucp_gentype (in pcre_ucp_searchfuncs.c) | |
| 1158 | as "const" (a) because they are and (b) because it helps the PHP | |
| 1159 | maintainers who have recently made a script to detect big data structures | |
| 1160 | in the php code that should be moved to the .rodata section. I remembered | |
| 1161 | to update Builducptable as well, so it won't revert if ucptable.h is ever | |
| 1162 | re-created. | |
| 1163 | ||
| 1164 | 9. Added some extra #ifdef SUPPORT_UTF8 conditionals into pcretest.c, | |
| 1165 | pcre_printint.src, pcre_compile.c, pcre_study.c, and pcre_tables.c, in | |
| 1166 | order to be able to cut out the UTF-8 tables in the latter when UTF-8 | |
| 1167 | support is not required. This saves 1.5-2K of code, which is important in | |
| 1168 | some applications. | |
| 1169 | ||
| 1170 | Later: more #ifdefs are needed in pcre_ord2utf8.c and pcre_valid_utf8.c | |
| 1171 | so as not to refer to the tables, even though these functions will never be | |
| 1172 | called when UTF-8 support is disabled. Otherwise there are problems with a | |
| 1173 | shared library. | |
| 1174 | ||
| 1175 | 10. Fixed two bugs in the emulated memmove() function in pcre_internal.h: | |
| 1176 | ||
| 1177 | (a) It was defining its arguments as char * instead of void *. | |
| 1178 | ||
| 1179 | (b) It was assuming that all moves were upwards in memory; this was true | |
| 1180 | a long time ago when I wrote it, but is no longer the case. | |
| 1181 | ||
| 1182 | The emulated memove() is provided for those environments that have neither | |
| 1183 | memmove() nor bcopy(). I didn't think anyone used it these days, but that | |
| 1184 | is clearly not the case, as these two bugs were recently reported. | |
| 1185 | ||
| 1186 | 11. The script PrepareRelease is now distributed: it calls 132html, CleanTxt, | |
| 1187 | and Detrail to create the HTML documentation, the .txt form of the man | |
| 1188 | pages, and it removes trailing spaces from listed files. It also creates | |
| 1189 | pcre.h.generic and config.h.generic from pcre.h and config.h. In the latter | |
| 1190 | case, it wraps all the #defines with #ifndefs. This script should be run | |
| 1191 | before "make dist". | |
| 1192 | ||
| 1193 | 12. Fixed two fairly obscure bugs concerned with quantified caseless matching | |
| 1194 | with Unicode property support. | |
| 1195 | ||
| 1196 | (a) For a maximizing quantifier, if the two different cases of the | |
| 1197 | character were of different lengths in their UTF-8 codings (there are | |
| 1198 | some cases like this - I found 11), and the matching function had to | |
| 1199 | back up over a mixture of the two cases, it incorrectly assumed they | |
| 1200 | were both the same length. | |
| 1201 | ||
| 1202 | (b) When PCRE was configured to use the heap rather than the stack for | |
| 1203 | recursion during matching, it was not correctly preserving the data for | |
| 1204 | the other case of a UTF-8 character when checking ahead for a match | |
| 1205 | while processing a minimizing repeat. If the check also involved | |
| 1206 | matching a wide character, but failed, corruption could cause an | |
| 1207 | erroneous result when trying to check for a repeat of the original | |
| 1208 | character. | |
| 1209 | ||
| 1210 | 13. Some tidying changes to the testing mechanism: | |
| 1211 | ||
| 1212 | (a) The RunTest script now detects the internal link size and whether there | |
| 1213 | is UTF-8 and UCP support by running ./pcretest -C instead of relying on | |
| 1214 | values substituted by "configure". (The RunGrepTest script already did | |
| 1215 | this for UTF-8.) The configure.ac script no longer substitutes the | |
| 1216 | relevant variables. | |
| 1217 | ||
| 1218 | (b) The debugging options /B and /D in pcretest show the compiled bytecode | |
| 1219 | with length and offset values. This means that the output is different | |
| 1220 | for different internal link sizes. Test 2 is skipped for link sizes | |
| 1221 | other than 2 because of this, bypassing the problem. Unfortunately, | |
| 1222 | there was also a test in test 3 (the locale tests) that used /B and | |
| 1223 | failed for link sizes other than 2. Rather than cut the whole test out, | |
| 1224 | I have added a new /Z option to pcretest that replaces the length and | |
| 1225 | offset values with spaces. This is now used to make test 3 independent | |
| 1226 | of link size. (Test 2 will be tidied up later.) | |
| 1227 | ||
| 1228 | 14. If erroroffset was passed as NULL to pcre_compile, it provoked a | |
| 1229 | segmentation fault instead of returning the appropriate error message. | |
| 1230 | ||
| 1231 | 15. In multiline mode when the newline sequence was set to "any", the pattern | |
| 1232 | ^$ would give a match between the \r and \n of a subject such as "A\r\nB". | |
| 1233 | This doesn't seem right; it now treats the CRLF combination as the line | |
| 1234 | ending, and so does not match in that case. It's only a pattern such as ^$ | |
| 1235 | that would hit this one: something like ^ABC$ would have failed after \r | |
| 1236 | and then tried again after \r\n. | |
| 1237 | ||
| 1238 | 16. Changed the comparison command for RunGrepTest from "diff -u" to "diff -ub" | |
| 1239 | in an attempt to make files that differ only in their line terminators | |
| 1240 | compare equal. This works on Linux. | |
| 1241 | ||
| 1242 | 17. Under certain error circumstances pcregrep might try to free random memory | |
| 1243 | as it exited. This is now fixed, thanks to valgrind. | |
| 1244 | ||
| 1245 | 19. In pcretest, if the pattern /(?m)^$/g<any> was matched against the string | |
| 1246 | "abc\r\n\r\n", it found an unwanted second match after the second \r. This | |
| 1247 | was because its rules for how to advance for /g after matching an empty | |
| 1248 | string at the end of a line did not allow for this case. They now check for | |
| 1249 | it specially. | |
| 1250 | ||
| 1251 | 20. pcretest is supposed to handle patterns and data of any length, by | |
| 1252 | extending its buffers when necessary. It was getting this wrong when the | |
| 1253 | buffer for a data line had to be extended. | |
| 1254 | ||
| 1255 | 21. Added PCRE_NEWLINE_ANYCRLF which is like ANY, but matches only CR, LF, or | |
| 1256 | CRLF as a newline sequence. | |
| 1257 | ||
| 1258 | 22. Code for handling Unicode properties in pcre_dfa_exec() wasn't being cut | |
| 1259 | out by #ifdef SUPPORT_UCP. This did no harm, as it could never be used, but | |
| 1260 | I have nevertheless tidied it up. | |
| 1261 | ||
| 1262 | 23. Added some casts to kill warnings from HP-UX ia64 compiler. | |
| 1263 | ||
| 1264 | 24. Added a man page for pcre-config. | |
| 1265 | ||
| 1266 | ||
| 1267 | Version 7.0 19-Dec-06 | |
| 1268 | --------------------- | |
| 1269 | ||
| 1270 | 1. Fixed a signed/unsigned compiler warning in pcre_compile.c, shown up by | |
| 1271 | moving to gcc 4.1.1. | |
| 1272 | ||
| 1273 | 2. The -S option for pcretest uses setrlimit(); I had omitted to #include | |
| 1274 | sys/time.h, which is documented as needed for this function. It doesn't | |
| 1275 | seem to matter on Linux, but it showed up on some releases of OS X. | |
| 1276 | ||
| 1277 | 3. It seems that there are systems where bytes whose values are greater than | |
| 1278 | 127 match isprint() in the "C" locale. The "C" locale should be the | |
| 1279 | default when a C program starts up. In most systems, only ASCII printing | |
| 1280 | characters match isprint(). This difference caused the output from pcretest | |
| 1281 | to vary, making some of the tests fail. I have changed pcretest so that: | |
| 1282 | ||
| 1283 | (a) When it is outputting text in the compiled version of a pattern, bytes | |
| 1284 | other than 32-126 are always shown as hex escapes. | |
| 1285 | ||
| 1286 | (b) When it is outputting text that is a matched part of a subject string, | |
| 1287 | it does the same, unless a different locale has been set for the match | |
| 1288 | (using the /L modifier). In this case, it uses isprint() to decide. | |
| 1289 | ||
| 1290 | 4. Fixed a major bug that caused incorrect computation of the amount of memory | |
| 1291 | required for a compiled pattern when options that changed within the | |
| 1292 | pattern affected the logic of the preliminary scan that determines the | |
| 1293 | length. The relevant options are -x, and -i in UTF-8 mode. The result was | |
| 1294 | that the computed length was too small. The symptoms of this bug were | |
| 1295 | either the PCRE error "internal error: code overflow" from pcre_compile(), | |
| 1296 | or a glibc crash with a message such as "pcretest: free(): invalid next | |
| 1297 | size (fast)". Examples of patterns that provoked this bug (shown in | |
| 1298 | pcretest format) are: | |
| 1299 | ||
| 1300 | /(?-x: )/x | |
| 1301 | /(?x)(?-x: \s*#\s*)/ | |
| 1302 | /((?i)[\x{c0}])/8 | |
| 1303 | /(?i:[\x{c0}])/8 | |
| 1304 | ||
| 1305 | HOWEVER: Change 17 below makes this fix obsolete as the memory computation | |
| 1306 | is now done differently. | |
| 1307 | ||
| 1308 | 5. Applied patches from Google to: (a) add a QuoteMeta function to the C++ | |
| 1309 | wrapper classes; (b) implement a new function in the C++ scanner that is | |
| 1310 | more efficient than the old way of doing things because it avoids levels of | |
| 1311 | recursion in the regex matching; (c) add a paragraph to the documentation | |
| 1312 | for the FullMatch() function. | |
| 1313 | ||
| 1314 | 6. The escape sequence \n was being treated as whatever was defined as | |
| 1315 | "newline". Not only was this contrary to the documentation, which states | |
| 1316 | that \n is character 10 (hex 0A), but it also went horribly wrong when | |
| 1317 | "newline" was defined as CRLF. This has been fixed. | |
| 1318 | ||
| 1319 | 7. In pcre_dfa_exec.c the value of an unsigned integer (the variable called c) | |
| 1320 | was being set to -1 for the "end of line" case (supposedly a value that no | |
| 1321 | character can have). Though this value is never used (the check for end of | |
| 1322 | line is "zero bytes in current character"), it caused compiler complaints. | |
| 1323 | I've changed it to 0xffffffff. | |
| 1324 | ||
| 1325 | 8. In pcre_version.c, the version string was being built by a sequence of | |
| 1326 | C macros that, in the event of PCRE_PRERELEASE being defined as an empty | |
| 1327 | string (as it is for production releases) called a macro with an empty | |
| 1328 | argument. The C standard says the result of this is undefined. The gcc | |
| 1329 | compiler treats it as an empty string (which was what was wanted) but it is | |
| 1330 | reported that Visual C gives an error. The source has been hacked around to | |
| 1331 | avoid this problem. | |
| 1332 | ||
| 1333 | 9. On the advice of a Windows user, included <io.h> and <fcntl.h> in Windows | |
| 1334 | builds of pcretest, and changed the call to _setmode() to use _O_BINARY | |
| 1335 | instead of 0x8000. Made all the #ifdefs test both _WIN32 and WIN32 (not all | |
| 1336 | of them did). | |
| 1337 | ||
| 1338 | 10. Originally, pcretest opened its input and output without "b"; then I was | |
| 1339 | told that "b" was needed in some environments, so it was added for release | |
| 1340 | 5.0 to both the input and output. (It makes no difference on Unix-like | |
| 1341 | systems.) Later I was told that it is wrong for the input on Windows. I've | |
| 1342 | now abstracted the modes into two macros, to make it easier to fiddle with | |
| 1343 | them, and removed "b" from the input mode under Windows. | |
| 1344 | ||
| 1345 | 11. Added pkgconfig support for the C++ wrapper library, libpcrecpp. | |
| 1346 | ||
| 1347 | 12. Added -help and --help to pcretest as an official way of being reminded | |
| 1348 | of the options. | |
| 1349 | ||
| 1350 | 13. Removed some redundant semicolons after macro calls in pcrecpparg.h.in | |
| 1351 | and pcrecpp.cc because they annoy compilers at high warning levels. | |
| 1352 | ||
| 1353 | 14. A bit of tidying/refactoring in pcre_exec.c in the main bumpalong loop. | |
| 1354 | ||
| 1355 | 15. Fixed an occurrence of == in configure.ac that should have been = (shell | |
| 1356 | scripts are not C programs :-) and which was not noticed because it works | |
| 1357 | on Linux. | |
| 1358 | ||
| 1359 | 16. pcretest is supposed to handle any length of pattern and data line (as one | |
| 1360 | line or as a continued sequence of lines) by extending its input buffer if | |
| 1361 | necessary. This feature was broken for very long pattern lines, leading to | |
| 1362 | a string of junk being passed to pcre_compile() if the pattern was longer | |
| 1363 | than about 50K. | |
| 1364 | ||
| 1365 | 17. I have done a major re-factoring of the way pcre_compile() computes the | |
| 1366 | amount of memory needed for a compiled pattern. Previously, there was code | |
| 1367 | that made a preliminary scan of the pattern in order to do this. That was | |
| 1368 | OK when PCRE was new, but as the facilities have expanded, it has become | |
| 1369 | harder and harder to keep it in step with the real compile phase, and there | |
| 1370 | have been a number of bugs (see for example, 4 above). I have now found a | |
| 1371 | cunning way of running the real compile function in a "fake" mode that | |
| 1372 | enables it to compute how much memory it would need, while actually only | |
| 1373 | ever using a few hundred bytes of working memory and without too many | |
| 1374 | tests of the mode. This should make future maintenance and development | |
| 1375 | easier. A side effect of this work is that the limit of 200 on the nesting | |
| 1376 | depth of parentheses has been removed (though this was never a serious | |
| 1377 | limitation, I suspect). However, there is a downside: pcre_compile() now | |
| 1378 | runs more slowly than before (30% or more, depending on the pattern). I | |
| 1379 | hope this isn't a big issue. There is no effect on runtime performance. | |
| 1380 | ||
| 1381 | 18. Fixed a minor bug in pcretest: if a pattern line was not terminated by a | |
| 1382 | newline (only possible for the last line of a file) and it was a | |
| 1383 | pattern that set a locale (followed by /Lsomething), pcretest crashed. | |
| 1384 | ||
| 1385 | 19. Added additional timing features to pcretest. (1) The -tm option now times | |
| 1386 | matching only, not compiling. (2) Both -t and -tm can be followed, as a | |
| 1387 | separate command line item, by a number that specifies the number of | |
| 1388 | repeats to use when timing. The default is 50000; this gives better | |
| 1389 | precision, but takes uncomfortably long for very large patterns. | |
| 1390 | ||
| 1391 | 20. Extended pcre_study() to be more clever in cases where a branch of a | |
| 1392 | subpattern has no definite first character. For example, (a*|b*)[cd] would | |
| 1393 | previously give no result from pcre_study(). Now it recognizes that the | |
| 1394 | first character must be a, b, c, or d. | |
| 1395 | ||
| 1396 | 21. There was an incorrect error "recursive call could loop indefinitely" if | |
| 1397 | a subpattern (or the entire pattern) that was being tested for matching an | |
| 1398 | empty string contained only one non-empty item after a nested subpattern. | |
| 1399 | For example, the pattern (?>\x{100}*)\d(?R) provoked this error | |
| 1400 | incorrectly, because the \d was being skipped in the check. | |
| 1401 | ||
| 1402 | 22. The pcretest program now has a new pattern option /B and a command line | |
| 1403 | option -b, which is equivalent to adding /B to every pattern. This causes | |
| 1404 | it to show the compiled bytecode, without the additional information that | |
| 1405 | -d shows. The effect of -d is now the same as -b with -i (and similarly, /D | |
| 1406 | is the same as /B/I). | |
| 1407 | ||
| 1408 | 23. A new optimization is now able automatically to treat some sequences such | |
| 1409 | as a*b as a*+b. More specifically, if something simple (such as a character | |
| 1410 | or a simple class like \d) has an unlimited quantifier, and is followed by | |
| 1411 | something that cannot possibly match the quantified thing, the quantifier | |
| 1412 | is automatically "possessified". | |
| 1413 | ||
| 1414 | 24. A recursive reference to a subpattern whose number was greater than 39 | |
| 1415 | went wrong under certain circumstances in UTF-8 mode. This bug could also | |
| 1416 | have affected the operation of pcre_study(). | |
| 1417 | ||
| 1418 | 25. Realized that a little bit of performance could be had by replacing | |
| 1419 | (c & 0xc0) == 0xc0 with c >= 0xc0 when processing UTF-8 characters. | |
| 1420 | ||
| 1421 | 26. Timing data from pcretest is now shown to 4 decimal places instead of 3. | |
| 1422 | ||
| 1423 | 27. Possessive quantifiers such as a++ were previously implemented by turning | |
| 1424 | them into atomic groups such as ($>a+). Now they have their own opcodes, | |
| 1425 | which improves performance. This includes the automatically created ones | |
| 1426 | from 23 above. | |
| 1427 | ||
| 1428 | 28. A pattern such as (?=(\w+))\1: which simulates an atomic group using a | |
| 1429 | lookahead was broken if it was not anchored. PCRE was mistakenly expecting | |
| 1430 | the first matched character to be a colon. This applied both to named and | |
| 1431 | numbered groups. | |
| 1432 | ||
| 1433 | 29. The ucpinternal.h header file was missing its idempotency #ifdef. | |
| 1434 | ||
| 1435 | 30. I was sent a "project" file called libpcre.a.dev which I understand makes | |
| 1436 | building PCRE on Windows easier, so I have included it in the distribution. | |
| 1437 | ||
| 1438 | 31. There is now a check in pcretest against a ridiculously large number being | |
| 1439 | returned by pcre_exec() or pcre_dfa_exec(). If this happens in a /g or /G | |
| 1440 | loop, the loop is abandoned. | |
| 1441 | ||
| 1442 | 32. Forward references to subpatterns in conditions such as (?(2)...) where | |
| 1443 | subpattern 2 is defined later cause pcre_compile() to search forwards in | |
| 1444 | the pattern for the relevant set of parentheses. This search went wrong | |
| 1445 | when there were unescaped parentheses in a character class, parentheses | |
| 1446 | escaped with \Q...\E, or parentheses in a #-comment in /x mode. | |
| 1447 | ||
| 1448 | 33. "Subroutine" calls and backreferences were previously restricted to | |
| 1449 | referencing subpatterns earlier in the regex. This restriction has now | |
| 1450 | been removed. | |
| 1451 | ||
| 1452 | 34. Added a number of extra features that are going to be in Perl 5.10. On the | |
| 1453 | whole, these are just syntactic alternatives for features that PCRE had | |
| 1454 | previously implemented using the Python syntax or my own invention. The | |
| 1455 | other formats are all retained for compatibility. | |
| 1456 | ||
| 1457 | (a) Named groups can now be defined as (?<name>...) or (?'name'...) as well | |
| 1458 | as (?P<name>...). The new forms, as well as being in Perl 5.10, are | |
| 1459 | also .NET compatible. | |
| 1460 | ||
| 1461 | (b) A recursion or subroutine call to a named group can now be defined as | |
| 1462 | (?&name) as well as (?P>name). | |
| 1463 | ||
| 1464 | (c) A backreference to a named group can now be defined as \k<name> or | |
| 1465 | \k'name' as well as (?P=name). The new forms, as well as being in Perl | |
| 1466 | 5.10, are also .NET compatible. | |
| 1467 | ||
| 1468 | (d) A conditional reference to a named group can now use the syntax | |
| 1469 | (?(<name>) or (?('name') as well as (?(name). | |
| 1470 | ||
| 1471 | (e) A "conditional group" of the form (?(DEFINE)...) can be used to define | |
| 1472 | groups (named and numbered) that are never evaluated inline, but can be | |
| 1473 | called as "subroutines" from elsewhere. In effect, the DEFINE condition | |
| 1474 | is always false. There may be only one alternative in such a group. | |
| 1475 | ||
| 1476 | (f) A test for recursion can be given as (?(R1).. or (?(R&name)... as well | |
| 1477 | as the simple (?(R). The condition is true only if the most recent | |
| 1478 | recursion is that of the given number or name. It does not search out | |
| 1479 | through the entire recursion stack. | |
| 1480 | ||
| 1481 | (g) The escape \gN or \g{N} has been added, where N is a positive or | |
| 1482 | negative number, specifying an absolute or relative reference. | |
| 1483 | ||
| 1484 | 35. Tidied to get rid of some further signed/unsigned compiler warnings and | |
| 1485 | some "unreachable code" warnings. | |
| 1486 | ||
| 1487 | 36. Updated the Unicode property tables to Unicode version 5.0.0. Amongst other | |
| 1488 | things, this adds five new scripts. | |
| 1489 | ||
| 1490 | 37. Perl ignores orphaned \E escapes completely. PCRE now does the same. | |
| 1491 | There were also incompatibilities regarding the handling of \Q..\E inside | |
| 1492 | character classes, for example with patterns like [\Qa\E-\Qz\E] where the | |
| 1493 | hyphen was adjacent to \Q or \E. I hope I've cleared all this up now. | |
| 1494 | ||
| 1495 | 38. Like Perl, PCRE detects when an indefinitely repeated parenthesized group | |
| 1496 | matches an empty string, and forcibly breaks the loop. There were bugs in | |
| 1497 | this code in non-simple cases. For a pattern such as ^(a()*)* matched | |
| 1498 | against aaaa the result was just "a" rather than "aaaa", for example. Two | |
| 1499 | separate and independent bugs (that affected different cases) have been | |
| 1500 | fixed. | |
| 1501 | ||
| 1502 | 39. Refactored the code to abolish the use of different opcodes for small | |
| 1503 | capturing bracket numbers. This is a tidy that I avoided doing when I | |
| 1504 | removed the limit on the number of capturing brackets for 3.5 back in 2001. | |
| 1505 | The new approach is not only tidier, it makes it possible to reduce the | |
| 1506 | memory needed to fix the previous bug (38). | |
| 1507 | ||
| 1508 | 40. Implemented PCRE_NEWLINE_ANY to recognize any of the Unicode newline | |
| 1509 | sequences (http://unicode.org/unicode/reports/tr18/) as "newline" when | |
| 1510 | processing dot, circumflex, or dollar metacharacters, or #-comments in /x | |
| 1511 | mode. | |
| 1512 | ||
| 1513 | 41. Add \R to match any Unicode newline sequence, as suggested in the Unicode | |
| 1514 | report. | |
| 1515 | ||
| 1516 | 42. Applied patch, originally from Ari Pollak, modified by Google, to allow | |
| 1517 | copy construction and assignment in the C++ wrapper. | |
| 1518 | ||
| 1519 | 43. Updated pcregrep to support "--newline=any". In the process, I fixed a | |
| 1520 | couple of bugs that could have given wrong results in the "--newline=crlf" | |
| 1521 | case. | |
| 1522 | ||
| 1523 | 44. Added a number of casts and did some reorganization of signed/unsigned int | |
| 1524 | variables following suggestions from Dair Grant. Also renamed the variable | |
| 1525 | "this" as "item" because it is a C++ keyword. | |
| 1526 | ||
| 1527 | 45. Arranged for dftables to add | |
| 1528 | ||
| 1529 | #include "pcre_internal.h" | |
| 1530 | ||
| 1531 | to pcre_chartables.c because without it, gcc 4.x may remove the array | |
| 1532 | definition from the final binary if PCRE is built into a static library and | |
| 1533 | dead code stripping is activated. | |
| 1534 | ||
| 1535 | 46. For an unanchored pattern, if a match attempt fails at the start of a | |
| 1536 | newline sequence, and the newline setting is CRLF or ANY, and the next two | |
| 1537 | characters are CRLF, advance by two characters instead of one. | |
| 1538 | ||
| 1539 | ||
| 1540 | Version 6.7 04-Jul-06 | |
| 1541 | --------------------- | |
| 1542 | ||
| 1543 | 1. In order to handle tests when input lines are enormously long, pcretest has | |
| 1544 | been re-factored so that it automatically extends its buffers when | |
| 1545 | necessary. The code is crude, but this _is_ just a test program. The | |
| 1546 | default size has been increased from 32K to 50K. | |
| 1547 | ||
| 1548 | 2. The code in pcre_study() was using the value of the re argument before | |
| 1549 | testing it for NULL. (Of course, in any sensible call of the function, it | |
| 1550 | won't be NULL.) | |
| 1551 | ||
| 1552 | 3. The memmove() emulation function in pcre_internal.h, which is used on | |
| 1553 | systems that lack both memmove() and bcopy() - that is, hardly ever - | |
| 1554 | was missing a "static" storage class specifier. | |
| 1555 | ||
| 1556 | 4. When UTF-8 mode was not set, PCRE looped when compiling certain patterns | |
| 1557 | containing an extended class (one that cannot be represented by a bitmap | |
| 1558 | because it contains high-valued characters or Unicode property items, e.g. | |
| 1559 | [\pZ]). Almost always one would set UTF-8 mode when processing such a | |
| 1560 | pattern, but PCRE should not loop if you do not (it no longer does). | |
| 1561 | [Detail: two cases were found: (a) a repeated subpattern containing an | |
| 1562 | extended class; (b) a recursive reference to a subpattern that followed a | |
| 1563 | previous extended class. It wasn't skipping over the extended class | |
| 1564 | correctly when UTF-8 mode was not set.] | |
| 1565 | ||
| 1566 | 5. A negated single-character class was not being recognized as fixed-length | |
| 1567 | in lookbehind assertions such as (?<=[^f]), leading to an incorrect | |
| 1568 | compile error "lookbehind assertion is not fixed length". | |
| 1569 | ||
| 1570 | 6. The RunPerlTest auxiliary script was showing an unexpected difference | |
| 1571 | between PCRE and Perl for UTF-8 tests. It turns out that it is hard to | |
| 1572 | write a Perl script that can interpret lines of an input file either as | |
| 1573 | byte characters or as UTF-8, which is what "perltest" was being required to | |
| 1574 | do for the non-UTF-8 and UTF-8 tests, respectively. Essentially what you | |
| 1575 | can't do is switch easily at run time between having the "use utf8;" pragma | |
| 1576 | or not. In the end, I fudged it by using the RunPerlTest script to insert | |
| 1577 | "use utf8;" explicitly for the UTF-8 tests. | |
| 1578 | ||
| 1579 | 7. In multiline (/m) mode, PCRE was matching ^ after a terminating newline at | |
| 1580 | the end of the subject string, contrary to the documentation and to what | |
| 1581 | Perl does. This was true of both matching functions. Now it matches only at | |
| 1582 | the start of the subject and immediately after *internal* newlines. | |
| 1583 | ||
| 1584 | 8. A call of pcre_fullinfo() from pcretest to get the option bits was passing | |
| 1585 | a pointer to an int instead of a pointer to an unsigned long int. This | |
| 1586 | caused problems on 64-bit systems. | |
| 1587 | ||
| 1588 | 9. Applied a patch from the folks at Google to pcrecpp.cc, to fix "another | |
| 1589 | instance of the 'standard' template library not being so standard". | |
| 1590 | ||
| 1591 | 10. There was no check on the number of named subpatterns nor the maximum | |
| 1592 | length of a subpattern name. The product of these values is used to compute | |
| 1593 | the size of the memory block for a compiled pattern. By supplying a very | |
| 1594 | long subpattern name and a large number of named subpatterns, the size | |
| 1595 | computation could be caused to overflow. This is now prevented by limiting | |
| 1596 | the length of names to 32 characters, and the number of named subpatterns | |
| 1597 | to 10,000. | |
| 1598 | ||
| 1599 | 11. Subpatterns that are repeated with specific counts have to be replicated in | |
| 1600 | the compiled pattern. The size of memory for this was computed from the | |
| 1601 | length of the subpattern and the repeat count. The latter is limited to | |
| 1602 | 65535, but there was no limit on the former, meaning that integer overflow | |
| 1603 | could in principle occur. The compiled length of a repeated subpattern is | |
| 1604 | now limited to 30,000 bytes in order to prevent this. | |
| 1605 | ||
| 1606 | 12. Added the optional facility to have named substrings with the same name. | |
| 1607 | ||
| 1608 | 13. Added the ability to use a named substring as a condition, using the | |
| 1609 | Python syntax: (?(name)yes|no). This overloads (?(R)... and names that | |
| 1610 | are numbers (not recommended). Forward references are permitted. | |
| 1611 | ||
| 1612 | 14. Added forward references in named backreferences (if you see what I mean). | |
| 1613 | ||
| 1614 | 15. In UTF-8 mode, with the PCRE_DOTALL option set, a quantified dot in the | |
| 1615 | pattern could run off the end of the subject. For example, the pattern | |
| 1616 | "(?s)(.{1,5})"8 did this with the subject "ab". | |
| 1617 | ||
| 1618 | 16. If PCRE_DOTALL or PCRE_MULTILINE were set, pcre_dfa_exec() behaved as if | |
| 1619 | PCRE_CASELESS was set when matching characters that were quantified with ? | |
| 1620 | or *. | |
| 1621 | ||
| 1622 | 17. A character class other than a single negated character that had a minimum | |
| 1623 | but no maximum quantifier - for example [ab]{6,} - was not handled | |
| 1624 | correctly by pce_dfa_exec(). It would match only one character. | |
| 1625 | ||
| 1626 | 18. A valid (though odd) pattern that looked like a POSIX character | |
| 1627 | class but used an invalid character after [ (for example [[,abc,]]) caused | |
| 1628 | pcre_compile() to give the error "Failed: internal error: code overflow" or | |
| 1629 | in some cases to crash with a glibc free() error. This could even happen if | |
| 1630 | the pattern terminated after [[ but there just happened to be a sequence of | |
| 1631 | letters, a binary zero, and a closing ] in the memory that followed. | |
| 1632 | ||
| 1633 | 19. Perl's treatment of octal escapes in the range \400 to \777 has changed | |
| 1634 | over the years. Originally (before any Unicode support), just the bottom 8 | |
| 1635 | bits were taken. Thus, for example, \500 really meant \100. Nowadays the | |
| 1636 | output from "man perlunicode" includes this: | |
| 1637 | ||
| 1638 | The regular expression compiler produces polymorphic opcodes. That | |
| 1639 | is, the pattern adapts to the data and automatically switches to | |
| 1640 | the Unicode character scheme when presented with Unicode data--or | |
| 1641 | instead uses a traditional byte scheme when presented with byte | |
| 1642 | data. | |
| 1643 | ||
| 1644 | Sadly, a wide octal escape does not cause a switch, and in a string with | |
| 1645 | no other multibyte characters, these octal escapes are treated as before. | |
| 1646 | Thus, in Perl, the pattern /\500/ actually matches \100 but the pattern | |
| 1647 | /\500|\x{1ff}/ matches \500 or \777 because the whole thing is treated as a | |
| 1648 | Unicode string. | |
| 1649 | ||
| 1650 | I have not perpetrated such confusion in PCRE. Up till now, it took just | |
| 1651 | the bottom 8 bits, as in old Perl. I have now made octal escapes with | |
| 1652 | values greater than \377 illegal in non-UTF-8 mode. In UTF-8 mode they | |
| 1653 | translate to the appropriate multibyte character. | |
| 1654 | ||
| 1655 | 29. Applied some refactoring to reduce the number of warnings from Microsoft | |
| 1656 | and Borland compilers. This has included removing the fudge introduced | |
| 1657 | seven years ago for the OS/2 compiler (see 2.02/2 below) because it caused | |
| 1658 | a warning about an unused variable. | |
| 1659 | ||
| 1660 | 21. PCRE has not included VT (character 0x0b) in the set of whitespace | |
| 1661 | characters since release 4.0, because Perl (from release 5.004) does not. | |
| 1662 | [Or at least, is documented not to: some releases seem to be in conflict | |
| 1663 | with the documentation.] However, when a pattern was studied with | |
| 1664 | pcre_study() and all its branches started with \s, PCRE still included VT | |
| 1665 | as a possible starting character. Of course, this did no harm; it just | |
| 1666 | caused an unnecessary match attempt. | |
| 1667 | ||
| 1668 | 22. Removed a now-redundant internal flag bit that recorded the fact that case | |
| 1669 | dependency changed within the pattern. This was once needed for "required | |
| 1670 | byte" processing, but is no longer used. This recovers a now-scarce options | |
| 1671 | bit. Also moved the least significant internal flag bit to the most- | |
| 1672 | significant bit of the word, which was not previously used (hangover from | |
| 1673 | the days when it was an int rather than a uint) to free up another bit for | |
| 1674 | the future. | |
| 1675 | ||
| 1676 | 23. Added support for CRLF line endings as well as CR and LF. As well as the | |
| 1677 | default being selectable at build time, it can now be changed at runtime | |
| 1678 | via the PCRE_NEWLINE_xxx flags. There are now options for pcregrep to | |
| 1679 | specify that it is scanning data with non-default line endings. | |
| 1680 | ||
| 1681 | 24. Changed the definition of CXXLINK to make it agree with the definition of | |
| 1682 | LINK in the Makefile, by replacing LDFLAGS to CXXFLAGS. | |
| 1683 | ||
| 1684 | 25. Applied Ian Taylor's patches to avoid using another stack frame for tail | |
| 1685 | recursions. This makes a big different to stack usage for some patterns. | |
| 1686 | ||
| 1687 | 26. If a subpattern containing a named recursion or subroutine reference such | |
| 1688 | as (?P>B) was quantified, for example (xxx(?P>B)){3}, the calculation of | |
| 1689 | the space required for the compiled pattern went wrong and gave too small a | |
| 1690 | value. Depending on the environment, this could lead to "Failed: internal | |
| 1691 | error: code overflow at offset 49" or "glibc detected double free or | |
| 1692 | corruption" errors. | |
| 1693 | ||
| 1694 | 27. Applied patches from Google (a) to support the new newline modes and (b) to | |
| 1695 | advance over multibyte UTF-8 characters in GlobalReplace. | |
| 1696 | ||
| 1697 | 28. Change free() to pcre_free() in pcredemo.c. Apparently this makes a | |
| 1698 | difference for some implementation of PCRE in some Windows version. | |
| 1699 | ||
| 1700 | 29. Added some extra testing facilities to pcretest: | |
| 1701 | ||
| 1702 | \q<number> in a data line sets the "match limit" value | |
| 1703 | \Q<number> in a data line sets the "match recursion limt" value | |
| 1704 | -S <number> sets the stack size, where <number> is in megabytes | |
| 1705 | ||
| 1706 | The -S option isn't available for Windows. | |
| 1707 | ||
| 1708 | ||
| 1709 | Version 6.6 06-Feb-06 | |
| 1710 | --------------------- | |
| 1711 | ||
| 1712 | 1. Change 16(a) for 6.5 broke things, because PCRE_DATA_SCOPE was not defined | |
| 1713 | in pcreposix.h. I have copied the definition from pcre.h. | |
| 1714 | ||
| 1715 | 2. Change 25 for 6.5 broke compilation in a build directory out-of-tree | |
| 1716 | because pcre.h is no longer a built file. | |
| 1717 | ||
| 1718 | 3. Added Jeff Friedl's additional debugging patches to pcregrep. These are | |
| 1719 | not normally included in the compiled code. | |
| 1720 | ||
| 1721 | ||
| 1722 | Version 6.5 01-Feb-06 | |
| 1723 | --------------------- | |
| 1724 | ||
| 1725 | 1. When using the partial match feature with pcre_dfa_exec(), it was not | |
| 1726 | anchoring the second and subsequent partial matches at the new starting | |
| 1727 | point. This could lead to incorrect results. For example, with the pattern | |
| 1728 | /1234/, partially matching against "123" and then "a4" gave a match. | |
| 1729 | ||
| 1730 | 2. Changes to pcregrep: | |
| 1731 | ||
| 1732 | (a) All non-match returns from pcre_exec() were being treated as failures | |
| 1733 | to match the line. Now, unless the error is PCRE_ERROR_NOMATCH, an | |
| 1734 | error message is output. Some extra information is given for the | |
| 1735 | PCRE_ERROR_MATCHLIMIT and PCRE_ERROR_RECURSIONLIMIT errors, which are | |
| 1736 | probably the only errors that are likely to be caused by users (by | |
| 1737 | specifying a regex that has nested indefinite repeats, for instance). | |
| 1738 | If there are more than 20 of these errors, pcregrep is abandoned. | |
| 1739 | ||
| 1740 | (b) A binary zero was treated as data while matching, but terminated the | |
| 1741 | output line if it was written out. This has been fixed: binary zeroes | |
| 1742 | are now no different to any other data bytes. | |
| 1743 | ||
| 1744 | (c) Whichever of the LC_ALL or LC_CTYPE environment variables is set is | |
| 1745 | used to set a locale for matching. The --locale=xxxx long option has | |
| 1746 | been added (no short equivalent) to specify a locale explicitly on the | |
| 1747 | pcregrep command, overriding the environment variables. | |
| 1748 | ||
| 1749 | (d) When -B was used with -n, some line numbers in the output were one less | |
| 1750 | than they should have been. | |
| 1751 | ||
| 1752 | (e) Added the -o (--only-matching) option. | |
| 1753 | ||
| 1754 | (f) If -A or -C was used with -c (count only), some lines of context were | |
| 1755 | accidentally printed for the final match. | |
| 1756 | ||
| 1757 | (g) Added the -H (--with-filename) option. | |
| 1758 | ||
| 1759 | (h) The combination of options -rh failed to suppress file names for files | |
| 1760 | that were found from directory arguments. | |
| 1761 | ||
| 1762 | (i) Added the -D (--devices) and -d (--directories) options. | |
| 1763 | ||
| 1764 | (j) Added the -F (--fixed-strings) option. | |
| 1765 | ||
| 1766 | (k) Allow "-" to be used as a file name for -f as well as for a data file. | |
| 1767 | ||
| 1768 | (l) Added the --colo(u)r option. | |
| 1769 | ||
| 1770 | (m) Added Jeffrey Friedl's -S testing option, but within #ifdefs so that it | |
| 1771 | is not present by default. | |
| 1772 | ||
| 1773 | 3. A nasty bug was discovered in the handling of recursive patterns, that is, | |
| 1774 | items such as (?R) or (?1), when the recursion could match a number of | |
| 1775 | alternatives. If it matched one of the alternatives, but subsequently, | |
| 1776 | outside the recursion, there was a failure, the code tried to back up into | |
| 1777 | the recursion. However, because of the way PCRE is implemented, this is not | |
| 1778 | possible, and the result was an incorrect result from the match. | |
| 1779 | ||
| 1780 | In order to prevent this happening, the specification of recursion has | |
| 1781 | been changed so that all such subpatterns are automatically treated as | |
| 1782 | atomic groups. Thus, for example, (?R) is treated as if it were (?>(?R)). | |
| 1783 | ||
| 1784 | 4. I had overlooked the fact that, in some locales, there are characters for | |
| 1785 | which isalpha() is true but neither isupper() nor islower() are true. In | |
| 1786 | the fr_FR locale, for instance, the \xAA and \xBA characters (ordmasculine | |
| 1787 | and ordfeminine) are like this. This affected the treatment of \w and \W | |
| 1788 | when they appeared in character classes, but not when they appeared outside | |
| 1789 | a character class. The bit map for "word" characters is now created | |
| 1790 | separately from the results of isalnum() instead of just taking it from the | |
| 1791 | upper, lower, and digit maps. (Plus the underscore character, of course.) | |
| 1792 | ||
| 1793 | 5. The above bug also affected the handling of POSIX character classes such as | |
| 1794 | [[:alpha:]] and [[:alnum:]]. These do not have their own bit maps in PCRE's | |
| 1795 | permanent tables. Instead, the bit maps for such a class were previously | |
| 1796 | created as the appropriate unions of the upper, lower, and digit bitmaps. | |
| 1797 | Now they are created by subtraction from the [[:word:]] class, which has | |
| 1798 | its own bitmap. | |
| 1799 | ||
| 1800 | 6. The [[:blank:]] character class matches horizontal, but not vertical space. | |
| 1801 | It is created by subtracting the vertical space characters (\x09, \x0a, | |
| 1802 | \x0b, \x0c) from the [[:space:]] bitmap. Previously, however, the | |
| 1803 | subtraction was done in the overall bitmap for a character class, meaning | |
| 1804 | that a class such as [\x0c[:blank:]] was incorrect because \x0c would not | |
| 1805 | be recognized. This bug has been fixed. | |
| 1806 | ||
| 1807 | 7. Patches from the folks at Google: | |
| 1808 | ||
| 1809 | (a) pcrecpp.cc: "to handle a corner case that may or may not happen in | |
| 1810 | real life, but is still worth protecting against". | |
| 1811 | ||
| 1812 | (b) pcrecpp.cc: "corrects a bug when negative radixes are used with | |
| 1813 | regular expressions". | |
| 1814 | ||
| 1815 | (c) pcre_scanner.cc: avoid use of std::count() because not all systems | |
| 1816 | have it. | |
| 1817 | ||
| 1818 | (d) Split off pcrecpparg.h from pcrecpp.h and had the former built by | |
| 1819 | "configure" and the latter not, in order to fix a problem somebody had | |
| 1820 | with compiling the Arg class on HP-UX. | |
| 1821 | ||
| 1822 | (e) Improve the error-handling of the C++ wrapper a little bit. | |
| 1823 | ||
| 1824 | (f) New tests for checking recursion limiting. | |
| 1825 | ||
| 1826 | 8. The pcre_memmove() function, which is used only if the environment does not | |
| 1827 | have a standard memmove() function (and is therefore rarely compiled), | |
| 1828 | contained two bugs: (a) use of int instead of size_t, and (b) it was not | |
| 1829 | returning a result (though PCRE never actually uses the result). | |
| 1830 | ||
| 1831 | 9. In the POSIX regexec() interface, if nmatch is specified as a ridiculously | |
| 1832 | large number - greater than INT_MAX/(3*sizeof(int)) - REG_ESPACE is | |
| 1833 | returned instead of calling malloc() with an overflowing number that would | |
| 1834 | most likely cause subsequent chaos. | |
| 1835 | ||
| 1836 | 10. The debugging option of pcretest was not showing the NO_AUTO_CAPTURE flag. | |
| 1837 | ||
| 1838 | 11. The POSIX flag REG_NOSUB is now supported. When a pattern that was compiled | |
| 1839 | with this option is matched, the nmatch and pmatch options of regexec() are | |
| 1840 | ignored. | |
| 1841 | ||
| 1842 | 12. Added REG_UTF8 to the POSIX interface. This is not defined by POSIX, but is | |
| 1843 | provided in case anyone wants to the the POSIX interface with UTF-8 | |
| 1844 | strings. | |
| 1845 | ||
| 1846 | 13. Added CXXLDFLAGS to the Makefile parameters to provide settings only on the | |
| 1847 | C++ linking (needed for some HP-UX environments). | |
| 1848 | ||
| 1849 | 14. Avoid compiler warnings in get_ucpname() when compiled without UCP support | |
| 1850 | (unused parameter) and in the pcre_printint() function (omitted "default" | |
| 1851 | switch label when the default is to do nothing). | |
| 1852 | ||
| 1853 | 15. Added some code to make it possible, when PCRE is compiled as a C++ | |
| 1854 | library, to replace subject pointers for pcre_exec() with a smart pointer | |
| 1855 | class, thus making it possible to process discontinuous strings. | |
| 1856 | ||
| 1857 | 16. The two macros PCRE_EXPORT and PCRE_DATA_SCOPE are confusing, and perform | |
| 1858 | much the same function. They were added by different people who were trying | |
| 1859 | to make PCRE easy to compile on non-Unix systems. It has been suggested | |
| 1860 | that PCRE_EXPORT be abolished now that there is more automatic apparatus | |
| 1861 | for compiling on Windows systems. I have therefore replaced it with | |
| 1862 | PCRE_DATA_SCOPE. This is set automatically for Windows; if not set it | |
| 1863 | defaults to "extern" for C or "extern C" for C++, which works fine on | |
| 1864 | Unix-like systems. It is now possible to override the value of PCRE_DATA_ | |
| 1865 | SCOPE with something explicit in config.h. In addition: | |
| 1866 | ||
| 1867 | (a) pcreposix.h still had just "extern" instead of either of these macros; | |
| 1868 | I have replaced it with PCRE_DATA_SCOPE. | |
| 1869 | ||
| 1870 | (b) Functions such as _pcre_xclass(), which are internal to the library, | |
| 1871 | but external in the C sense, all had PCRE_EXPORT in their definitions. | |
| 1872 | This is apparently wrong for the Windows case, so I have removed it. | |
| 1873 | (It makes no difference on Unix-like systems.) | |
| 1874 | ||
| 1875 | 17. Added a new limit, MATCH_LIMIT_RECURSION, which limits the depth of nesting | |
| 1876 | of recursive calls to match(). This is different to MATCH_LIMIT because | |
| 1877 | that limits the total number of calls to match(), not all of which increase | |
| 1878 | the depth of recursion. Limiting the recursion depth limits the amount of | |
| 1879 | stack (or heap if NO_RECURSE is set) that is used. The default can be set | |
| 1880 | when PCRE is compiled, and changed at run time. A patch from Google adds | |
| 1881 | this functionality to the C++ interface. | |
| 1882 | ||
| 1883 | 18. Changes to the handling of Unicode character properties: | |
| 1884 | ||
| 1885 | (a) Updated the table to Unicode 4.1.0. | |
| 1886 | ||
| 1887 | (b) Recognize characters that are not in the table as "Cn" (undefined). | |
| 1888 | ||
| 1889 | (c) I revised the way the table is implemented to a much improved format | |
| 1890 | which includes recognition of ranges. It now supports the ranges that | |
| 1891 | are defined in UnicodeData.txt, and it also amalgamates other | |
| 1892 | characters into ranges. This has reduced the number of entries in the | |
| 1893 | table from around 16,000 to around 3,000, thus reducing its size | |
| 1894 | considerably. I realized I did not need to use a tree structure after | |
| 1895 | all - a binary chop search is just as efficient. Having reduced the | |
| 1896 | number of entries, I extended their size from 6 bytes to 8 bytes to | |
| 1897 | allow for more data. | |
| 1898 | ||
| 1899 | (d) Added support for Unicode script names via properties such as \p{Han}. | |
| 1900 | ||
| 1901 | 19. In UTF-8 mode, a backslash followed by a non-Ascii character was not | |
| 1902 | matching that character. | |
| 1903 | ||
| 1904 | 20. When matching a repeated Unicode property with a minimum greater than zero, | |
| 1905 | (for example \pL{2,}), PCRE could look past the end of the subject if it | |
| 1906 | reached it while seeking the minimum number of characters. This could | |
| 1907 | happen only if some of the characters were more than one byte long, because | |
| 1908 | there is a check for at least the minimum number of bytes. | |
| 1909 | ||
| 1910 | 21. Refactored the implementation of \p and \P so as to be more general, to | |
| 1911 | allow for more different types of property in future. This has changed the | |
| 1912 | compiled form incompatibly. Anybody with saved compiled patterns that use | |
| 1913 | \p or \P will have to recompile them. | |
| 1914 | ||
| 1915 | 22. Added "Any" and "L&" to the supported property types. | |
| 1916 | ||
| 1917 | 23. Recognize \x{...} as a code point specifier, even when not in UTF-8 mode, | |
| 1918 | but give a compile time error if the value is greater than 0xff. | |
| 1919 | ||
| 1920 | 24. The man pages for pcrepartial, pcreprecompile, and pcre_compile2 were | |
| 1921 | accidentally not being installed or uninstalled. | |
| 1922 | ||
| 1923 | 25. The pcre.h file was built from pcre.h.in, but the only changes that were | |
| 1924 | made were to insert the current release number. This seemed silly, because | |
| 1925 | it made things harder for people building PCRE on systems that don't run | |
| 1926 | "configure". I have turned pcre.h into a distributed file, no longer built | |
| 1927 | by "configure", with the version identification directly included. There is | |
| 1928 | no longer a pcre.h.in file. | |
| 1929 | ||
| 1930 | However, this change necessitated a change to the pcre-config script as | |
| 1931 | well. It is built from pcre-config.in, and one of the substitutions was the | |
| 1932 | release number. I have updated configure.ac so that ./configure now finds | |
| 1933 | the release number by grepping pcre.h. | |
| 1934 | ||
| 1935 | 26. Added the ability to run the tests under valgrind. | |
| 1936 | ||
| 1937 | ||
| 1938 | Version 6.4 05-Sep-05 | |
| 1939 | --------------------- | |
| 1940 | ||
| 1941 | 1. Change 6.0/10/(l) to pcregrep introduced a bug that caused separator lines | |
| 1942 | "--" to be printed when multiple files were scanned, even when none of the | |
| 1943 | -A, -B, or -C options were used. This is not compatible with Gnu grep, so I | |
| 1944 | consider it to be a bug, and have restored the previous behaviour. | |
| 1945 | ||
| 1946 | 2. A couple of code tidies to get rid of compiler warnings. | |
| 1947 | ||
| 1948 | 3. The pcretest program used to cheat by referring to symbols in the library | |
| 1949 | whose names begin with _pcre_. These are internal symbols that are not | |
| 1950 | really supposed to be visible externally, and in some environments it is | |
| 1951 | possible to suppress them. The cheating is now confined to including | |
| 1952 | certain files from the library's source, which is a bit cleaner. | |
| 1953 | ||
| 1954 | 4. Renamed pcre.in as pcre.h.in to go with pcrecpp.h.in; it also makes the | |
| 1955 | file's purpose clearer. | |
| 1956 | ||
| 1957 | 5. Reorganized pcre_ucp_findchar(). | |
| 1958 | ||
| 1959 | ||
| 1960 | Version 6.3 15-Aug-05 | |
| 1961 | --------------------- | |
| 1962 | ||
| 1963 | 1. The file libpcre.pc.in did not have general read permission in the tarball. | |
| 1964 | ||
| 1965 | 2. There were some problems when building without C++ support: | |
| 1966 | ||
| 1967 | (a) If C++ support was not built, "make install" and "make test" still | |
| 1968 | tried to test it. | |
| 1969 | ||
| 1970 | (b) There were problems when the value of CXX was explicitly set. Some | |
| 1971 | changes have been made to try to fix these, and ... | |
| 1972 | ||
| 1973 | (c) --disable-cpp can now be used to explicitly disable C++ support. | |
| 1974 | ||
| 1975 | (d) The use of @CPP_OBJ@ directly caused a blank line preceded by a | |
| 1976 | backslash in a target when C++ was disabled. This confuses some | |
| 1977 | versions of "make", apparently. Using an intermediate variable solves | |
| 1978 | this. (Same for CPP_LOBJ.) | |
| 1979 | ||
| 1980 | 3. $(LINK_FOR_BUILD) now includes $(CFLAGS_FOR_BUILD) and $(LINK) | |
| 1981 | (non-Windows) now includes $(CFLAGS) because these flags are sometimes | |
| 1982 | necessary on certain architectures. | |
| 1983 | ||
| 1984 | 4. Added a setting of -export-symbols-regex to the link command to remove | |
| 1985 | those symbols that are exported in the C sense, but actually are local | |
| 1986 | within the library, and not documented. Their names all begin with | |
| 1987 | "_pcre_". This is not a perfect job, because (a) we have to except some | |
| 1988 | symbols that pcretest ("illegally") uses, and (b) the facility isn't always | |
| 1989 | available (and never for static libraries). I have made a note to try to | |
| 1990 | find a way round (a) in the future. | |
| 1991 | ||
| 1992 | ||
| 1993 | Version 6.2 01-Aug-05 | |
| 1994 | --------------------- | |
| 1995 | ||
| 1996 | 1. There was no test for integer overflow of quantifier values. A construction | |
| 1997 | such as {1111111111111111} would give undefined results. What is worse, if | |
| 1998 | a minimum quantifier for a parenthesized subpattern overflowed and became | |
| 1999 | negative, the calculation of the memory size went wrong. This could have | |
| 2000 | led to memory overwriting. | |
| 2001 | ||
| 2002 | 2. Building PCRE using VPATH was broken. Hopefully it is now fixed. | |
| 2003 | ||
| 2004 | 3. Added "b" to the 2nd argument of fopen() in dftables.c, for non-Unix-like | |
| 2005 | operating environments where this matters. | |
| 2006 | ||
| 2007 | 4. Applied Giuseppe Maxia's patch to add additional features for controlling | |
| 2008 | PCRE options from within the C++ wrapper. | |
| 2009 | ||
| 2010 | 5. Named capturing subpatterns were not being correctly counted when a pattern | |
| 2011 | was compiled. This caused two problems: (a) If there were more than 100 | |
| 2012 | such subpatterns, the calculation of the memory needed for the whole | |
| 2013 | compiled pattern went wrong, leading to an overflow error. (b) Numerical | |
| 2014 | back references of the form \12, where the number was greater than 9, were | |
| 2015 | not recognized as back references, even though there were sufficient | |
| 2016 | previous subpatterns. | |
| 2017 | ||
| 2018 | 6. Two minor patches to pcrecpp.cc in order to allow it to compile on older | |
| 2019 | versions of gcc, e.g. 2.95.4. | |
| 2020 | ||
| 2021 | ||
| 2022 | Version 6.1 21-Jun-05 | |
| 2023 | --------------------- | |
| 2024 | ||
| 2025 | 1. There was one reference to the variable "posix" in pcretest.c that was not | |
| 2026 | surrounded by "#if !defined NOPOSIX". | |
| 2027 | ||
| 2028 | 2. Make it possible to compile pcretest without DFA support, UTF8 support, or | |
| 2029 | the cross-check on the old pcre_info() function, for the benefit of the | |
| 2030 | cut-down version of PCRE that is currently imported into Exim. | |
| 2031 | ||
| 2032 | 3. A (silly) pattern starting with (?i)(?-i) caused an internal space | |
| 2033 | allocation error. I've done the easy fix, which wastes 2 bytes for sensible | |
| 2034 | patterns that start (?i) but I don't think that matters. The use of (?i) is | |
| 2035 | just an example; this all applies to the other options as well. | |
| 2036 | ||
| 2037 | 4. Since libtool seems to echo the compile commands it is issuing, the output | |
| 2038 | from "make" can be reduced a bit by putting "@" in front of each libtool | |
| 2039 | compile command. | |
| 2040 | ||
| 2041 | 5. Patch from the folks at Google for configure.in to be a bit more thorough | |
| 2042 | in checking for a suitable C++ installation before trying to compile the | |
| 2043 | C++ stuff. This should fix a reported problem when a compiler was present, | |
| 2044 | but no suitable headers. | |
| 2045 | ||
| 2046 | 6. The man pages all had just "PCRE" as their title. I have changed them to | |
| 2047 | be the relevant file name. I have also arranged that these names are | |
| 2048 | retained in the file doc/pcre.txt, which is a concatenation in text format | |
| 2049 | of all the man pages except the little individual ones for each function. | |
| 2050 | ||
| 2051 | 7. The NON-UNIX-USE file had not been updated for the different set of source | |
| 2052 | files that come with release 6. I also added a few comments about the C++ | |
| 2053 | wrapper. | |
| 2054 | ||
| 2055 | ||
| 2056 | Version 6.0 07-Jun-05 | |
| 2057 | --------------------- | |
| 2058 | ||
| 2059 | 1. Some minor internal re-organization to help with my DFA experiments. | |
| 2060 | ||
| 2061 | 2. Some missing #ifdef SUPPORT_UCP conditionals in pcretest and printint that | |
| 2062 | didn't matter for the library itself when fully configured, but did matter | |
| 2063 | when compiling without UCP support, or within Exim, where the ucp files are | |
| 2064 | not imported. | |
| 2065 | ||
| 2066 | 3. Refactoring of the library code to split up the various functions into | |
| 2067 | different source modules. The addition of the new DFA matching code (see | |
| 2068 | below) to a single monolithic source would have made it really too | |
| 2069 | unwieldy, quite apart from causing all the code to be include in a | |
| 2070 | statically linked application, when only some functions are used. This is | |
| 2071 | relevant even without the DFA addition now that patterns can be compiled in | |
| 2072 | one application and matched in another. | |
| 2073 | ||
| 2074 | The downside of splitting up is that there have to be some external | |
| 2075 | functions and data tables that are used internally in different modules of | |
| 2076 | the library but which are not part of the API. These have all had their | |
| 2077 | names changed to start with "_pcre_" so that they are unlikely to clash | |
| 2078 | with other external names. | |
| 2079 | ||
| 2080 | 4. Added an alternate matching function, pcre_dfa_exec(), which matches using | |
| 2081 | a different (DFA) algorithm. Although it is slower than the original | |
| 2082 | function, it does have some advantages for certain types of matching | |
| 2083 | problem. | |
| 2084 | ||
| 2085 | 5. Upgrades to pcretest in order to test the features of pcre_dfa_exec(), | |
| 2086 | including restarting after a partial match. | |
| 2087 | ||
| 2088 | 6. A patch for pcregrep that defines INVALID_FILE_ATTRIBUTES if it is not | |
| 2089 | defined when compiling for Windows was sent to me. I have put it into the | |
| 2090 | code, though I have no means of testing or verifying it. | |
| 2091 | ||
| 2092 | 7. Added the pcre_refcount() auxiliary function. | |
| 2093 | ||
| 2094 | 8. Added the PCRE_FIRSTLINE option. This constrains an unanchored pattern to | |
| 2095 | match before or at the first newline in the subject string. In pcretest, | |
| 2096 | the /f option on a pattern can be used to set this. | |
| 2097 | ||
| 2098 | 9. A repeated \w when used in UTF-8 mode with characters greater than 256 | |
| 2099 | would behave wrongly. This has been present in PCRE since release 4.0. | |
| 2100 | ||
| 2101 | 10. A number of changes to the pcregrep command: | |
| 2102 | ||
| 2103 | (a) Refactored how -x works; insert ^(...)$ instead of setting | |
| 2104 | PCRE_ANCHORED and checking the length, in preparation for adding | |
| 2105 | something similar for -w. | |
| 2106 | ||
| 2107 | (b) Added the -w (match as a word) option. | |
| 2108 | ||
| 2109 | (c) Refactored the way lines are read and buffered so as to have more | |
| 2110 | than one at a time available. | |
| 2111 | ||
| 2112 | (d) Implemented a pcregrep test script. | |
| 2113 | ||
| 2114 | (e) Added the -M (multiline match) option. This allows patterns to match | |
| 2115 | over several lines of the subject. The buffering ensures that at least | |
| 2116 | 8K, or the rest of the document (whichever is the shorter) is available | |
| 2117 | for matching (and similarly the previous 8K for lookbehind assertions). | |
| 2118 | ||
| 2119 | (f) Changed the --help output so that it now says | |
| 2120 | ||
| 2121 | -w, --word-regex(p) | |
| 2122 | ||
| 2123 | instead of two lines, one with "regex" and the other with "regexp" | |
| 2124 | because that confused at least one person since the short forms are the | |
| 2125 | same. (This required a bit of code, as the output is generated | |
| 2126 | automatically from a table. It wasn't just a text change.) | |
| 2127 | ||
| 2128 | (g) -- can be used to terminate pcregrep options if the next thing isn't an | |
| 2129 | option but starts with a hyphen. Could be a pattern or a path name | |
| 2130 | starting with a hyphen, for instance. | |
| 2131 | ||
| 2132 | (h) "-" can be given as a file name to represent stdin. | |
| 2133 | ||
| 2134 | (i) When file names are being printed, "(standard input)" is used for | |
| 2135 | the standard input, for compatibility with GNU grep. Previously | |
| 2136 | "<stdin>" was used. | |
| 2137 | ||
| 2138 | (j) The option --label=xxx can be used to supply a name to be used for | |
| 2139 | stdin when file names are being printed. There is no short form. | |
| 2140 | ||
| 2141 | (k) Re-factored the options decoding logic because we are going to add | |
| 2142 | two more options that take data. Such options can now be given in four | |
| 2143 | different ways, e.g. "-fname", "-f name", "--file=name", "--file name". | |
| 2144 | ||
| 2145 | (l) Added the -A, -B, and -C options for requesting that lines of context | |
| 2146 | around matches be printed. | |
| 2147 | ||
| 2148 | (m) Added the -L option to print the names of files that do not contain | |
| 2149 | any matching lines, that is, the complement of -l. | |
| 2150 | ||
| 2151 | (n) The return code is 2 if any file cannot be opened, but pcregrep does | |
| 2152 | continue to scan other files. | |
| 2153 | ||
| 2154 | (o) The -s option was incorrectly implemented. For compatibility with other | |
| 2155 | greps, it now suppresses the error message for a non-existent or non- | |
| 2156 | accessible file (but not the return code). There is a new option called | |
| 2157 | -q that suppresses the output of matching lines, which was what -s was | |
| 2158 | previously doing. | |
| 2159 | ||
| 2160 | (p) Added --include and --exclude options to specify files for inclusion | |
| 2161 | and exclusion when recursing. | |
| 2162 | ||
| 2163 | 11. The Makefile was not using the Autoconf-supported LDFLAGS macro properly. | |
| 2164 | Hopefully, it now does. | |
| 2165 | ||
| 2166 | 12. Missing cast in pcre_study(). | |
| 2167 | ||
| 2168 | 13. Added an "uninstall" target to the makefile. | |
| 2169 | ||
| 2170 | 14. Replaced "extern" in the function prototypes in Makefile.in with | |
| 2171 | "PCRE_DATA_SCOPE", which defaults to 'extern' or 'extern "C"' in the Unix | |
| 2172 | world, but is set differently for Windows. | |
| 2173 | ||
| 2174 | 15. Added a second compiling function called pcre_compile2(). The only | |
| 2175 | difference is that it has an extra argument, which is a pointer to an | |
| 2176 | integer error code. When there is a compile-time failure, this is set | |
| 2177 | non-zero, in addition to the error test pointer being set to point to an | |
| 2178 | error message. The new argument may be NULL if no error number is required | |
| 2179 | (but then you may as well call pcre_compile(), which is now just a | |
| 2180 | wrapper). This facility is provided because some applications need a | |
| 2181 | numeric error indication, but it has also enabled me to tidy up the way | |
| 2182 | compile-time errors are handled in the POSIX wrapper. | |
| 2183 | ||
| 2184 | 16. Added VPATH=.libs to the makefile; this should help when building with one | |
| 2185 | prefix path and installing with another. (Or so I'm told by someone who | |
| 2186 | knows more about this stuff than I do.) | |
| 2187 | ||
| 2188 | 17. Added a new option, REG_DOTALL, to the POSIX function regcomp(). This | |
| 2189 | passes PCRE_DOTALL to the pcre_compile() function, making the "." character | |
| 2190 | match everything, including newlines. This is not POSIX-compatible, but | |
| 2191 | somebody wanted the feature. From pcretest it can be activated by using | |
| 2192 | both the P and the s flags. | |
| 2193 | ||
| 2194 | 18. AC_PROG_LIBTOOL appeared twice in Makefile.in. Removed one. | |
| 2195 | ||
| 2196 | 19. libpcre.pc was being incorrectly installed as executable. | |
| 2197 | ||
| 2198 | 20. A couple of places in pcretest check for end-of-line by looking for '\n'; | |
| 2199 | it now also looks for '\r' so that it will work unmodified on Windows. | |
| 2200 | ||
| 2201 | 21. Added Google's contributed C++ wrapper to the distribution. | |
| 2202 | ||
| 2203 | 22. Added some untidy missing memory free() calls in pcretest, to keep | |
| 2204 | Electric Fence happy when testing. | |
| 2205 | ||
| 2206 | ||
| 2207 | ||
| 2208 | Version 5.0 13-Sep-04 | |
| 2209 | --------------------- | |
| 2210 | ||
| 2211 | 1. Internal change: literal characters are no longer packed up into items | |
| 2212 | containing multiple characters in a single byte-string. Each character | |
| 2213 | is now matched using a separate opcode. However, there may be more than one | |
| 2214 | byte in the character in UTF-8 mode. | |
| 2215 | ||
| 2216 | 2. The pcre_callout_block structure has two new fields: pattern_position and | |
| 2217 | next_item_length. These contain the offset in the pattern to the next match | |
| 2218 | item, and its length, respectively. | |
| 2219 | ||
| 2220 | 3. The PCRE_AUTO_CALLOUT option for pcre_compile() requests the automatic | |
| 2221 | insertion of callouts before each pattern item. Added the /C option to | |
| 2222 | pcretest to make use of this. | |
| 2223 | ||
| 2224 | 4. On the advice of a Windows user, the lines | |
| 2225 | ||
| 2226 | #if defined(_WIN32) || defined(WIN32) | |
| 2227 | _setmode( _fileno( stdout ), 0x8000 ); | |
| 2228 | #endif /* defined(_WIN32) || defined(WIN32) */ | |
| 2229 | ||
| 2230 | have been added to the source of pcretest. This apparently does useful | |
| 2231 | magic in relation to line terminators. | |
| 2232 | ||
| 2233 | 5. Changed "r" and "w" in the calls to fopen() in pcretest to "rb" and "wb" | |
| 2234 | for the benefit of those environments where the "b" makes a difference. | |
| 2235 | ||
| 2236 | 6. The icc compiler has the same options as gcc, but "configure" doesn't seem | |
| 2237 | to know about it. I have put a hack into configure.in that adds in code | |
| 2238 | to set GCC=yes if CC=icc. This seems to end up at a point in the | |
| 2239 | generated configure script that is early enough to affect the setting of | |
| 2240 | compiler options, which is what is needed, but I have no means of testing | |
| 2241 | whether it really works. (The user who reported this had patched the | |
| 2242 | generated configure script, which of course I cannot do.) | |
| 2243 | ||
| 2244 | LATER: After change 22 below (new libtool files), the configure script | |
| 2245 | seems to know about icc (and also ecc). Therefore, I have commented out | |
| 2246 | this hack in configure.in. | |
| 2247 | ||
| 2248 | 7. Added support for pkg-config (2 patches were sent in). | |
| 2249 | ||
| 2250 | 8. Negated POSIX character classes that used a combination of internal tables | |
| 2251 | were completely broken. These were [[:^alpha:]], [[:^alnum:]], and | |
| 2252 | [[:^ascii]]. Typically, they would match almost any characters. The other | |
| 2253 | POSIX classes were not broken in this way. | |
| 2254 | ||
| 2255 | 9. Matching the pattern "\b.*?" against "ab cd", starting at offset 1, failed | |
| 2256 | to find the match, as PCRE was deluded into thinking that the match had to | |
| 2257 | start at the start point or following a newline. The same bug applied to | |
| 2258 | patterns with negative forward assertions or any backward assertions | |
| 2259 | preceding ".*" at the start, unless the pattern required a fixed first | |
| 2260 | character. This was a failing pattern: "(?!.bcd).*". The bug is now fixed. | |
| 2261 | ||
| 2262 | 10. In UTF-8 mode, when moving forwards in the subject after a failed match | |
| 2263 | starting at the last subject character, bytes beyond the end of the subject | |
| 2264 | string were read. | |
| 2265 | ||
| 2266 | 11. Renamed the variable "class" as "classbits" to make life easier for C++ | |
| 2267 | users. (Previously there was a macro definition, but it apparently wasn't | |
| 2268 | enough.) | |
| 2269 | ||
| 2270 | 12. Added the new field "tables" to the extra data so that tables can be passed | |
| 2271 | in at exec time, or the internal tables can be re-selected. This allows | |
| 2272 | a compiled regex to be saved and re-used at a later time by a different | |
| 2273 | program that might have everything at different addresses. | |
| 2274 | ||
| 2275 | 13. Modified the pcre-config script so that, when run on Solaris, it shows a | |
| 2276 | -R library as well as a -L library. | |
| 2277 | ||
| 2278 | 14. The debugging options of pcretest (-d on the command line or D on a | |
| 2279 | pattern) showed incorrect output for anything following an extended class | |
| 2280 | that contained multibyte characters and which was followed by a quantifier. | |
| 2281 | ||
| 2282 | 15. Added optional support for general category Unicode character properties | |
| 2283 | via the \p, \P, and \X escapes. Unicode property support implies UTF-8 | |
| 2284 | support. It adds about 90K to the size of the library. The meanings of the | |
| 2285 | inbuilt class escapes such as \d and \s have NOT been changed. | |
| 2286 | ||
| 2287 | 16. Updated pcredemo.c to include calls to free() to release the memory for the | |
| 2288 | compiled pattern. | |
| 2289 | ||
| 2290 | 17. The generated file chartables.c was being created in the source directory | |
| 2291 | instead of in the building directory. This caused the build to fail if the | |
| 2292 | source directory was different from the building directory, and was | |
| 2293 | read-only. | |
| 2294 | ||
| 2295 | 18. Added some sample Win commands from Mark Tetrode into the NON-UNIX-USE | |
| 2296 | file. No doubt somebody will tell me if they don't make sense... Also added | |
| 2297 | Dan Mooney's comments about building on OpenVMS. | |
| 2298 | ||
| 2299 | 19. Added support for partial matching via the PCRE_PARTIAL option for | |
| 2300 | pcre_exec() and the \P data escape in pcretest. | |
| 2301 | ||
| 2302 | 20. Extended pcretest with 3 new pattern features: | |
| 2303 | ||
| 2304 | (i) A pattern option of the form ">rest-of-line" causes pcretest to | |
| 2305 | write the compiled pattern to the file whose name is "rest-of-line". | |
| 2306 | This is a straight binary dump of the data, with the saved pointer to | |
| 2307 | the character tables forced to be NULL. The study data, if any, is | |
| 2308 | written too. After writing, pcretest reads a new pattern. | |
| 2309 | ||
| 2310 | (ii) If, instead of a pattern, "<rest-of-line" is given, pcretest reads a | |
| 2311 | compiled pattern from the given file. There must not be any | |
| 2312 | occurrences of "<" in the file name (pretty unlikely); if there are, | |
| 2313 | pcretest will instead treat the initial "<" as a pattern delimiter. | |
| 2314 | After reading in the pattern, pcretest goes on to read data lines as | |
| 2315 | usual. | |
| 2316 | ||
| 2317 | (iii) The F pattern option causes pcretest to flip the bytes in the 32-bit | |
| 2318 | and 16-bit fields in a compiled pattern, to simulate a pattern that | |
| 2319 | was compiled on a host of opposite endianness. | |
| 2320 | ||
| 2321 | 21. The pcre-exec() function can now cope with patterns that were compiled on | |
| 2322 | hosts of opposite endianness, with this restriction: | |
| 2323 | ||
| 2324 | As for any compiled expression that is saved and used later, the tables | |
| 2325 | pointer field cannot be preserved; the extra_data field in the arguments | |
| 2326 | to pcre_exec() should be used to pass in a tables address if a value | |
| 2327 | other than the default internal tables were used at compile time. | |
| 2328 | ||
| 2329 | 22. Calling pcre_exec() with a negative value of the "ovecsize" parameter is | |
| 2330 | now diagnosed as an error. Previously, most of the time, a negative number | |
| 2331 | would have been treated as zero, but if in addition "ovector" was passed as | |
| 2332 | NULL, a crash could occur. | |
| 2333 | ||
| 2334 | 23. Updated the files ltmain.sh, config.sub, config.guess, and aclocal.m4 with | |
| 2335 | new versions from the libtool 1.5 distribution (the last one is a copy of | |
| 2336 | a file called libtool.m4). This seems to have fixed the need to patch | |
| 2337 | "configure" to support Darwin 1.3 (which I used to do). However, I still | |
| 2338 | had to patch ltmain.sh to ensure that ${SED} is set (it isn't on my | |
| 2339 | workstation). | |
| 2340 | ||
| 2341 | 24. Changed the PCRE licence to be the more standard "BSD" licence. | |
| 2342 | ||
| 2343 | ||
| 2344 | Version 4.5 01-Dec-03 | |
| 2345 | --------------------- | |
| 2346 | ||
| 2347 | 1. There has been some re-arrangement of the code for the match() function so | |
| 2348 | that it can be compiled in a version that does not call itself recursively. | |
| 2349 | Instead, it keeps those local variables that need separate instances for | |
| 2350 | each "recursion" in a frame on the heap, and gets/frees frames whenever it | |
| 2351 | needs to "recurse". Keeping track of where control must go is done by means | |
| 2352 | of setjmp/longjmp. The whole thing is implemented by a set of macros that | |
| 2353 | hide most of the details from the main code, and operates only if | |
| 2354 | NO_RECURSE is defined while compiling pcre.c. If PCRE is built using the | |
| 2355 | "configure" mechanism, "--disable-stack-for-recursion" turns on this way of | |
| 2356 | operating. | |
| 2357 | ||
| 2358 | To make it easier for callers to provide specially tailored get/free | |
| 2359 | functions for this usage, two new functions, pcre_stack_malloc, and | |
| 2360 | pcre_stack_free, are used. They are always called in strict stacking order, | |
| 2361 | and the size of block requested is always the same. | |
| 2362 | ||
| 2363 | The PCRE_CONFIG_STACKRECURSE info parameter can be used to find out whether | |
| 2364 | PCRE has been compiled to use the stack or the heap for recursion. The | |
| 2365 | -C option of pcretest uses this to show which version is compiled. | |
| 2366 | ||
| 2367 | A new data escape \S, is added to pcretest; it causes the amounts of store | |
| 2368 | obtained and freed by both kinds of malloc/free at match time to be added | |
| 2369 | to the output. | |
| 2370 | ||
| 2371 | 2. Changed the locale test to use "fr_FR" instead of "fr" because that's | |
| 2372 | what's available on my current Linux desktop machine. | |
| 2373 | ||
| 2374 | 3. When matching a UTF-8 string, the test for a valid string at the start has | |
| 2375 | been extended. If start_offset is not zero, PCRE now checks that it points | |
| 2376 | to a byte that is the start of a UTF-8 character. If not, it returns | |
| 2377 | PCRE_ERROR_BADUTF8_OFFSET (-11). Note: the whole string is still checked; | |
| 2378 | this is necessary because there may be backward assertions in the pattern. | |
| 2379 | When matching the same subject several times, it may save resources to use | |
| 2380 | PCRE_NO_UTF8_CHECK on all but the first call if the string is long. | |
| 2381 | ||
| 2382 | 4. The code for checking the validity of UTF-8 strings has been tightened so | |
| 2383 | that it rejects (a) strings containing 0xfe or 0xff bytes and (b) strings | |
| 2384 | containing "overlong sequences". | |
| 2385 | ||
| 2386 | 5. Fixed a bug (appearing twice) that I could not find any way of exploiting! | |
| 2387 | I had written "if ((digitab[*p++] && chtab_digit) == 0)" where the "&&" | |
| 2388 | should have been "&", but it just so happened that all the cases this let | |
| 2389 | through by mistake were picked up later in the function. | |
| 2390 | ||
| 2391 | 6. I had used a variable called "isblank" - this is a C99 function, causing | |
| 2392 | some compilers to warn. To avoid this, I renamed it (as "blankclass"). | |
| 2393 | ||
| 2394 | 7. Cosmetic: (a) only output another newline at the end of pcretest if it is | |
| 2395 | prompting; (b) run "./pcretest /dev/null" at the start of the test script | |
| 2396 | so the version is shown; (c) stop "make test" echoing "./RunTest". | |
| 2397 | ||
| 2398 | 8. Added patches from David Burgess to enable PCRE to run on EBCDIC systems. | |
| 2399 | ||
| 2400 | 9. The prototype for memmove() for systems that don't have it was using | |
| 2401 | size_t, but the inclusion of the header that defines size_t was later. I've | |
| 2402 | moved the #includes for the C headers earlier to avoid this. | |
| 2403 | ||
| 2404 | 10. Added some adjustments to the code to make it easier to compiler on certain | |
| 2405 | special systems: | |
| 2406 | ||
| 2407 | (a) Some "const" qualifiers were missing. | |
| 2408 | (b) Added the macro EXPORT before all exported functions; by default this | |
| 2409 | is defined to be empty. | |
| 2410 | (c) Changed the dftables auxiliary program (that builds chartables.c) so | |
| 2411 | that it reads its output file name as an argument instead of writing | |
| 2412 | to the standard output and assuming this can be redirected. | |
| 2413 | ||
| 2414 | 11. In UTF-8 mode, if a recursive reference (e.g. (?1)) followed a character | |
| 2415 | class containing characters with values greater than 255, PCRE compilation | |
| 2416 | went into a loop. | |
| 2417 | ||
| 2418 | 12. A recursive reference to a subpattern that was within another subpattern | |
| 2419 | that had a minimum quantifier of zero caused PCRE to crash. For example, | |
| 2420 | (x(y(?2))z)? provoked this bug with a subject that got as far as the | |
| 2421 | recursion. If the recursively-called subpattern itself had a zero repeat, | |
| 2422 | that was OK. | |
| 2423 | ||
| 2424 | 13. In pcretest, the buffer for reading a data line was set at 30K, but the | |
| 2425 | buffer into which it was copied (for escape processing) was still set at | |
| 2426 | 1024, so long lines caused crashes. | |
| 2427 | ||
| 2428 | 14. A pattern such as /[ab]{1,3}+/ failed to compile, giving the error | |
| 2429 | "internal error: code overflow...". This applied to any character class | |
| 2430 | that was followed by a possessive quantifier. | |
| 2431 | ||
| 2432 | 15. Modified the Makefile to add libpcre.la as a prerequisite for | |
| 2433 | libpcreposix.la because I was told this is needed for a parallel build to | |
| 2434 | work. | |
| 2435 | ||
| 2436 | 16. If a pattern that contained .* following optional items at the start was | |
| 2437 | studied, the wrong optimizing data was generated, leading to matching | |
| 2438 | errors. For example, studying /[ab]*.*c/ concluded, erroneously, that any | |
| 2439 | matching string must start with a or b or c. The correct conclusion for | |
| 2440 | this pattern is that a match can start with any character. | |
| 2441 | ||
| 2442 | ||
| 2443 | Version 4.4 13-Aug-03 | |
| 2444 | --------------------- | |
| 2445 | ||
| 2446 | 1. In UTF-8 mode, a character class containing characters with values between | |
| 2447 | 127 and 255 was not handled correctly if the compiled pattern was studied. | |
| 2448 | In fixing this, I have also improved the studying algorithm for such | |
| 2449 | classes (slightly). | |
| 2450 | ||
| 2451 | 2. Three internal functions had redundant arguments passed to them. Removal | |
| 2452 | might give a very teeny performance improvement. | |
| 2453 | ||
| 2454 | 3. Documentation bug: the value of the capture_top field in a callout is *one | |
| 2455 | more than* the number of the hightest numbered captured substring. | |
| 2456 | ||
| 2457 | 4. The Makefile linked pcretest and pcregrep with -lpcre, which could result | |
| 2458 | in incorrectly linking with a previously installed version. They now link | |
| 2459 | explicitly with libpcre.la. | |
| 2460 | ||
| 2461 | 5. configure.in no longer needs to recognize Cygwin specially. | |
| 2462 | ||
| 2463 | 6. A problem in pcre.in for Windows platforms is fixed. | |
| 2464 | ||
| 2465 | 7. If a pattern was successfully studied, and the -d (or /D) flag was given to | |
| 2466 | pcretest, it used to include the size of the study block as part of its | |
| 2467 | output. Unfortunately, the structure contains a field that has a different | |
| 2468 | size on different hardware architectures. This meant that the tests that | |
| 2469 | showed this size failed. As the block is currently always of a fixed size, | |
| 2470 | this information isn't actually particularly useful in pcretest output, so | |
| 2471 | I have just removed it. | |
| 2472 | ||
| 2473 | 8. Three pre-processor statements accidentally did not start in column 1. | |
| 2474 | Sadly, there are *still* compilers around that complain, even though | |
| 2475 | standard C has not required this for well over a decade. Sigh. | |
| 2476 | ||
| 2477 | 9. In pcretest, the code for checking callouts passed small integers in the | |
| 2478 | callout_data field, which is a void * field. However, some picky compilers | |
| 2479 | complained about the casts involved for this on 64-bit systems. Now | |
| 2480 | pcretest passes the address of the small integer instead, which should get | |
| 2481 | rid of the warnings. | |
| 2482 | ||
| 2483 | 10. By default, when in UTF-8 mode, PCRE now checks for valid UTF-8 strings at | |
| 2484 | both compile and run time, and gives an error if an invalid UTF-8 sequence | |
| 2485 | is found. There is a option for disabling this check in cases where the | |
| 2486 | string is known to be correct and/or the maximum performance is wanted. | |
| 2487 | ||
| 2488 | 11. In response to a bug report, I changed one line in Makefile.in from | |
| 2489 | ||
| 2490 | -Wl,--out-implib,.libs/lib@WIN_PREFIX@pcreposix.dll.a \ | |
| 2491 | to | |
| 2492 | -Wl,--out-implib,.libs/@WIN_PREFIX@libpcreposix.dll.a \ | |
| 2493 | ||
| 2494 | to look similar to other lines, but I have no way of telling whether this | |
| 2495 | is the right thing to do, as I do not use Windows. No doubt I'll get told | |
| 2496 | if it's wrong... | |
| 2497 | ||
| 2498 | ||
| 2499 | Version 4.3 21-May-03 | |
| 2500 | --------------------- | |
| 2501 | ||
| 2502 | 1. Two instances of @WIN_PREFIX@ omitted from the Windows targets in the | |
| 2503 | Makefile. | |
| 2504 | ||
| 2505 | 2. Some refactoring to improve the quality of the code: | |
| 2506 | ||
| 2507 | (i) The utf8_table... variables are now declared "const". | |
| 2508 | ||
| 2509 | (ii) The code for \cx, which used the "case flipping" table to upper case | |
| 2510 | lower case letters, now just substracts 32. This is ASCII-specific, | |
| 2511 | but the whole concept of \cx is ASCII-specific, so it seems | |
| 2512 | reasonable. | |
| 2513 | ||
| 2514 | (iii) PCRE was using its character types table to recognize decimal and | |
| 2515 | hexadecimal digits in the pattern. This is silly, because it handles | |
| 2516 | only 0-9, a-f, and A-F, but the character types table is locale- | |
| 2517 | specific, which means strange things might happen. A private | |
| 2518 | table is now used for this - though it costs 256 bytes, a table is | |
| 2519 | much faster than multiple explicit tests. Of course, the standard | |
| 2520 | character types table is still used for matching digits in subject | |
| 2521 | strings against \d. | |
| 2522 | ||
| 2523 | (iv) Strictly, the identifier ESC_t is reserved by POSIX (all identifiers | |
| 2524 | ending in _t are). So I've renamed it as ESC_tee. | |
| 2525 | ||
| 2526 | 3. The first argument for regexec() in the POSIX wrapper should have been | |
| 2527 | defined as "const". | |
| 2528 | ||
| 2529 | 4. Changed pcretest to use malloc() for its buffers so that they can be | |
| 2530 | Electric Fenced for debugging. | |
| 2531 | ||
| 2532 | 5. There were several places in the code where, in UTF-8 mode, PCRE would try | |
| 2533 | to read one or more bytes before the start of the subject string. Often this | |
| 2534 | had no effect on PCRE's behaviour, but in some circumstances it could | |
| 2535 | provoke a segmentation fault. | |
| 2536 | ||
| 2537 | 6. A lookbehind at the start of a pattern in UTF-8 mode could also cause PCRE | |
| 2538 | to try to read one or more bytes before the start of the subject string. | |
| 2539 | ||
| 2540 | 7. A lookbehind in a pattern matched in non-UTF-8 mode on a PCRE compiled with | |
| 2541 | UTF-8 support could misbehave in various ways if the subject string | |
| 2542 | contained bytes with the 0x80 bit set and the 0x40 bit unset in a lookbehind | |
| 2543 | area. (PCRE was not checking for the UTF-8 mode flag, and trying to move | |
| 2544 | back over UTF-8 characters.) | |
| 2545 | ||
| 2546 | ||
| 2547 | Version 4.2 14-Apr-03 | |
| 2548 | --------------------- | |
| 2549 | ||
| 2550 | 1. Typo "#if SUPPORT_UTF8" instead of "#ifdef SUPPORT_UTF8" fixed. | |
| 2551 | ||
| 2552 | 2. Changes to the building process, supplied by Ronald Landheer-Cieslak | |
| 2553 | [ON_WINDOWS]: new variable, "#" on non-Windows platforms | |
| 2554 | [NOT_ON_WINDOWS]: new variable, "#" on Windows platforms | |
| 2555 | [WIN_PREFIX]: new variable, "cyg" for Cygwin | |
| 2556 | * Makefile.in: use autoconf substitution for OBJEXT, EXEEXT, BUILD_OBJEXT | |
| 2557 | and BUILD_EXEEXT | |
| 2558 | Note: automatic setting of the BUILD variables is not yet working | |
| 2559 | set CPPFLAGS and BUILD_CPPFLAGS (but don't use yet) - should be used at | |
| 2560 | compile-time but not at link-time | |
| 2561 | [LINK]: use for linking executables only | |
| 2562 | make different versions for Windows and non-Windows | |
| 2563 | [LINKLIB]: new variable, copy of UNIX-style LINK, used for linking | |
| 2564 | libraries | |
| 2565 | [LINK_FOR_BUILD]: new variable | |
| 2566 | [OBJEXT]: use throughout | |
| 2567 | [EXEEXT]: use throughout | |
| 2568 | <winshared>: new target | |
| 2569 | <wininstall>: new target | |
| 2570 | <dftables.o>: use native compiler | |
| 2571 | <dftables>: use native linker | |
| 2572 | <install>: handle Windows platform correctly | |
| 2573 | <clean>: ditto | |
| 2574 | <check>: ditto | |
| 2575 | copy DLL to top builddir before testing | |
| 2576 | ||
| 2577 | As part of these changes, -no-undefined was removed again. This was reported | |
| 2578 | to give trouble on HP-UX 11.0, so getting rid of it seems like a good idea | |
| 2579 | in any case. | |
| 2580 | ||
| 2581 | 3. Some tidies to get rid of compiler warnings: | |
| 2582 | ||
| 2583 | . In the match_data structure, match_limit was an unsigned long int, whereas | |
| 2584 | match_call_count was an int. I've made them both unsigned long ints. | |
| 2585 | ||
| 2586 | . In pcretest the fact that a const uschar * doesn't automatically cast to | |
| 2587 | a void * provoked a warning. | |
| 2588 | ||
| 2589 | . Turning on some more compiler warnings threw up some "shadow" variables | |
| 2590 | and a few more missing casts. | |
| 2591 | ||
| 2592 | 4. If PCRE was complied with UTF-8 support, but called without the PCRE_UTF8 | |
| 2593 | option, a class that contained a single character with a value between 128 | |
| 2594 | and 255 (e.g. /[\xFF]/) caused PCRE to crash. | |
| 2595 | ||
| 2596 | 5. If PCRE was compiled with UTF-8 support, but called without the PCRE_UTF8 | |
| 2597 | option, a class that contained several characters, but with at least one | |
| 2598 | whose value was between 128 and 255 caused PCRE to crash. | |
| 2599 | ||
| 2600 | ||
| 2601 | Version 4.1 12-Mar-03 | |
| 2602 | --------------------- | |
| 2603 | ||
| 2604 | 1. Compiling with gcc -pedantic found a couple of places where casts were | |
| 2605 | needed, and a string in dftables.c that was longer than standard compilers are | |
| 2606 | required to support. | |
| 2607 | ||
| 2608 | 2. Compiling with Sun's compiler found a few more places where the code could | |
| 2609 | be tidied up in order to avoid warnings. | |
| 2610 | ||
| 2611 | 3. The variables for cross-compiling were called HOST_CC and HOST_CFLAGS; the | |
| 2612 | first of these names is deprecated in the latest Autoconf in favour of the name | |
| 2613 | CC_FOR_BUILD, because "host" is typically used to mean the system on which the | |
| 2614 | compiled code will be run. I can't find a reference for HOST_CFLAGS, but by | |
| 2615 | analogy I have changed it to CFLAGS_FOR_BUILD. | |
| 2616 | ||
| 2617 | 4. Added -no-undefined to the linking command in the Makefile, because this is | |
| 2618 | apparently helpful for Windows. To make it work, also added "-L. -lpcre" to the | |
| 2619 | linking step for the pcreposix library. | |
| 2620 | ||
| 2621 | 5. PCRE was failing to diagnose the case of two named groups with the same | |
| 2622 | name. | |
| 2623 | ||
| 2624 | 6. A problem with one of PCRE's optimizations was discovered. PCRE remembers a | |
| 2625 | literal character that is needed in the subject for a match, and scans along to | |
| 2626 | ensure that it is present before embarking on the full matching process. This | |
| 2627 | saves time in cases of nested unlimited repeats that are never going to match. | |
| 2628 | Problem: the scan can take a lot of time if the subject is very long (e.g. | |
| 2629 | megabytes), thus penalizing straightforward matches. It is now done only if the | |
| 2630 | amount of subject to be scanned is less than 1000 bytes. | |
| 2631 | ||
| 2632 | 7. A lesser problem with the same optimization is that it was recording the | |
| 2633 | first character of an anchored pattern as "needed", thus provoking a search | |
| 2634 | right along the subject, even when the first match of the pattern was going to | |
| 2635 | fail. The "needed" character is now not set for anchored patterns, unless it | |
| 2636 | follows something in the pattern that is of non-fixed length. Thus, it still | |
| 2637 | fulfils its original purpose of finding quick non-matches in cases of nested | |
| 2638 | unlimited repeats, but isn't used for simple anchored patterns such as /^abc/. | |
| 2639 | ||
| 2640 | ||
| 2641 | Version 4.0 17-Feb-03 | |
| 2642 | --------------------- | |
| 2643 | ||
| 2644 | 1. If a comment in an extended regex that started immediately after a meta-item | |
| 2645 | extended to the end of string, PCRE compiled incorrect data. This could lead to | |
| 2646 | all kinds of weird effects. Example: /#/ was bad; /()#/ was bad; /a#/ was not. | |
| 2647 | ||
| 2648 | 2. Moved to autoconf 2.53 and libtool 1.4.2. | |
| 2649 | ||
| 2650 | 3. Perl 5.8 no longer needs "use utf8" for doing UTF-8 things. Consequently, | |
| 2651 | the special perltest8 script is no longer needed - all the tests can be run | |
| 2652 | from a single perltest script. | |
| 2653 | ||
| 2654 | 4. From 5.004, Perl has not included the VT character (0x0b) in the set defined | |
| 2655 | by \s. It has now been removed in PCRE. This means it isn't recognized as | |
| 2656 | whitespace in /x regexes too, which is the same as Perl. Note that the POSIX | |
| 2657 | class [:space:] *does* include VT, thereby creating a mess. | |
| 2658 | ||
| 2659 | 5. Added the class [:blank:] (a GNU extension from Perl 5.8) to match only | |
| 2660 | space and tab. | |
| 2661 | ||
| 2662 | 6. Perl 5.005 was a long time ago. It's time to amalgamate the tests that use | |
| 2663 | its new features into the main test script, reducing the number of scripts. | |
| 2664 | ||
| 2665 | 7. Perl 5.8 has changed the meaning of patterns like /a(?i)b/. Earlier versions | |
| 2666 | were backward compatible, and made the (?i) apply to the whole pattern, as if | |
| 2667 | /i were given. Now it behaves more logically, and applies the option setting | |
| 2668 | only to what follows. PCRE has been changed to follow suit. However, if it | |
| 2669 | finds options settings right at the start of the pattern, it extracts them into | |
| 2670 | the global options, as before. Thus, they show up in the info data. | |
| 2671 | ||
| 2672 | 8. Added support for the \Q...\E escape sequence. Characters in between are | |
| 2673 | treated as literals. This is slightly different from Perl in that $ and @ are | |
| 2674 | also handled as literals inside the quotes. In Perl, they will cause variable | |
| 2675 | interpolation. Note the following examples: | |
| 2676 | ||
| 2677 | Pattern PCRE matches Perl matches | |
| 2678 | ||
| 2679 | \Qabc$xyz\E abc$xyz abc followed by the contents of $xyz | |
| 2680 | \Qabc\$xyz\E abc\$xyz abc\$xyz | |
| 2681 | \Qabc\E\$\Qxyz\E abc$xyz abc$xyz | |
| 2682 | ||
| 2683 | For compatibility with Perl, \Q...\E sequences are recognized inside character | |
| 2684 | classes as well as outside them. | |
| 2685 | ||
| 2686 | 9. Re-organized 3 code statements in pcretest to avoid "overflow in | |
| 2687 | floating-point constant arithmetic" warnings from a Microsoft compiler. Added a | |
| 2688 | (size_t) cast to one statement in pcretest and one in pcreposix to avoid | |
| 2689 | signed/unsigned warnings. | |
| 2690 | ||
| 2691 | 10. SunOS4 doesn't have strtoul(). This was used only for unpicking the -o | |
| 2692 | option for pcretest, so I've replaced it by a simple function that does just | |
| 2693 | that job. | |
| 2694 | ||
| 2695 | 11. pcregrep was ending with code 0 instead of 2 for the commands "pcregrep" or | |
| 2696 | "pcregrep -". | |
| 2697 | ||
| 2698 | 12. Added "possessive quantifiers" ?+, *+, ++, and {,}+ which come from Sun's | |
| 2699 | Java package. This provides some syntactic sugar for simple cases of what my | |
| 2700 | documentation calls "once-only subpatterns". A pattern such as x*+ is the same | |
| 2701 | as (?>x*). In other words, if what is inside (?>...) is just a single repeated | |
| 2702 | item, you can use this simplified notation. Note that only makes sense with | |
| 2703 | greedy quantifiers. Consequently, the use of the possessive quantifier forces | |
| 2704 | greediness, whatever the setting of the PCRE_UNGREEDY option. | |
| 2705 | ||
| 2706 | 13. A change of greediness default within a pattern was not taking effect at | |
| 2707 | the current level for patterns like /(b+(?U)a+)/. It did apply to parenthesized | |
| 2708 | subpatterns that followed. Patterns like /b+(?U)a+/ worked because the option | |
| 2709 | was abstracted outside. | |
| 2710 | ||
| 2711 | 14. PCRE now supports the \G assertion. It is true when the current matching | |
| 2712 | position is at the start point of the match. This differs from \A when the | |
| 2713 | starting offset is non-zero. Used with the /g option of pcretest (or similar | |
| 2714 | code), it works in the same way as it does for Perl's /g option. If all | |
| 2715 | alternatives of a regex begin with \G, the expression is anchored to the start | |
| 2716 | match position, and the "anchored" flag is set in the compiled expression. | |
| 2717 | ||
| 2718 | 15. Some bugs concerning the handling of certain option changes within patterns | |
| 2719 | have been fixed. These applied to options other than (?ims). For example, | |
| 2720 | "a(?x: b c )d" did not match "XabcdY" but did match "Xa b c dY". It should have | |
| 2721 | been the other way round. Some of this was related to change 7 above. | |
| 2722 | ||
| 2723 | 16. PCRE now gives errors for /[.x.]/ and /[=x=]/ as unsupported POSIX | |
| 2724 | features, as Perl does. Previously, PCRE gave the warnings only for /[[.x.]]/ | |
| 2725 | and /[[=x=]]/. PCRE now also gives an error for /[:name:]/ because it supports | |
| 2726 | POSIX classes only within a class (e.g. /[[:alpha:]]/). | |
| 2727 | ||
| 2728 | 17. Added support for Perl's \C escape. This matches one byte, even in UTF8 | |
| 2729 | mode. Unlike ".", it always matches newline, whatever the setting of | |
| 2730 | PCRE_DOTALL. However, PCRE does not permit \C to appear in lookbehind | |
| 2731 | assertions. Perl allows it, but it doesn't (in general) work because it can't | |
| 2732 | calculate the length of the lookbehind. At least, that's the case for Perl | |
| 2733 | 5.8.0 - I've been told they are going to document that it doesn't work in | |
| 2734 | future. | |
| 2735 | ||
| 2736 | 18. Added an error diagnosis for escapes that PCRE does not support: these are | |
| 2737 | \L, \l, \N, \P, \p, \U, \u, and \X. | |
| 2738 | ||
| 2739 | 19. Although correctly diagnosing a missing ']' in a character class, PCRE was | |
| 2740 | reading past the end of the pattern in cases such as /[abcd/. | |
| 2741 | ||
| 2742 | 20. PCRE was getting more memory than necessary for patterns with classes that | |
| 2743 | contained both POSIX named classes and other characters, e.g. /[[:space:]abc/. | |
| 2744 | ||
| 2745 | 21. Added some code, conditional on #ifdef VPCOMPAT, to make life easier for | |
| 2746 | compiling PCRE for use with Virtual Pascal. | |
| 2747 | ||
| 2748 | 22. Small fix to the Makefile to make it work properly if the build is done | |
| 2749 | outside the source tree. | |
| 2750 | ||
| 2751 | 23. Added a new extension: a condition to go with recursion. If a conditional | |
| 2752 | subpattern starts with (?(R) the "true" branch is used if recursion has | |
| 2753 | happened, whereas the "false" branch is used only at the top level. | |
| 2754 | ||
| 2755 | 24. When there was a very long string of literal characters (over 255 bytes | |
| 2756 | without UTF support, over 250 bytes with UTF support), the computation of how | |
| 2757 | much memory was required could be incorrect, leading to segfaults or other | |
| 2758 | strange effects. | |
| 2759 | ||
| 2760 | 25. PCRE was incorrectly assuming anchoring (either to start of subject or to | |
| 2761 | start of line for a non-DOTALL pattern) when a pattern started with (.*) and | |
| 2762 | there was a subsequent back reference to those brackets. This meant that, for | |
| 2763 | example, /(.*)\d+\1/ failed to match "abc123bc". Unfortunately, it isn't | |
| 2764 | possible to check for precisely this case. All we can do is abandon the | |
| 2765 | optimization if .* occurs inside capturing brackets when there are any back | |
| 2766 | references whatsoever. (See below for a better fix that came later.) | |
| 2767 | ||
| 2768 | 26. The handling of the optimization for finding the first character of a | |
| 2769 | non-anchored pattern, and for finding a character that is required later in the | |
| 2770 | match were failing in some cases. This didn't break the matching; it just | |
| 2771 | failed to optimize when it could. The way this is done has been re-implemented. | |
| 2772 | ||
| 2773 | 27. Fixed typo in error message for invalid (?R item (it said "(?p"). | |
| 2774 | ||
| 2775 | 28. Added a new feature that provides some of the functionality that Perl | |
| 2776 | provides with (?{...}). The facility is termed a "callout". The way it is done | |
| 2777 | in PCRE is for the caller to provide an optional function, by setting | |
| 2778 | pcre_callout to its entry point. Like pcre_malloc and pcre_free, this is a | |
| 2779 | global variable. By default it is unset, which disables all calling out. To get | |
| 2780 | the function called, the regex must include (?C) at appropriate points. This | |
| 2781 | is, in fact, equivalent to (?C0), and any number <= 255 may be given with (?C). | |
| 2782 | This provides a means of identifying different callout points. When PCRE | |
| 2783 | reaches such a point in the regex, if pcre_callout has been set, the external | |
| 2784 | function is called. It is provided with data in a structure called | |
| 2785 | pcre_callout_block, which is defined in pcre.h. If the function returns 0, | |
| 2786 | matching continues; if it returns a non-zero value, the match at the current | |
| 2787 | point fails. However, backtracking will occur if possible. [This was changed | |
| 2788 | later and other features added - see item 49 below.] | |
| 2789 | ||
| 2790 | 29. pcretest is upgraded to test the callout functionality. It provides a | |
| 2791 | callout function that displays information. By default, it shows the start of | |
| 2792 | the match and the current position in the text. There are some new data escapes | |
| 2793 | to vary what happens: | |
| 2794 | ||
| 2795 | \C+ in addition, show current contents of captured substrings | |
| 2796 | \C- do not supply a callout function | |
| 2797 | \C!n return 1 when callout number n is reached | |
| 2798 | \C!n!m return 1 when callout number n is reached for the mth time | |
| 2799 | ||
| 2800 | 30. If pcregrep was called with the -l option and just a single file name, it | |
| 2801 | output "<stdin>" if a match was found, instead of the file name. | |
| 2802 | ||
| 2803 | 31. Improve the efficiency of the POSIX API to PCRE. If the number of capturing | |
| 2804 | slots is less than POSIX_MALLOC_THRESHOLD, use a block on the stack to pass to | |
| 2805 | pcre_exec(). This saves a malloc/free per call. The default value of | |
| 2806 | POSIX_MALLOC_THRESHOLD is 10; it can be changed by --with-posix-malloc-threshold | |
| 2807 | when configuring. | |
| 2808 | ||
| 2809 | 32. The default maximum size of a compiled pattern is 64K. There have been a | |
| 2810 | few cases of people hitting this limit. The code now uses macros to handle the | |
| 2811 | storing of links as offsets within the compiled pattern. It defaults to 2-byte | |
| 2812 | links, but this can be changed to 3 or 4 bytes by --with-link-size when | |
| 2813 | configuring. Tests 2 and 5 work only with 2-byte links because they output | |
| 2814 | debugging information about compiled patterns. | |
| 2815 | ||
| 2816 | 33. Internal code re-arrangements: | |
| 2817 | ||
| 2818 | (a) Moved the debugging function for printing out a compiled regex into | |
| 2819 | its own source file (printint.c) and used #include to pull it into | |
| 2820 | pcretest.c and, when DEBUG is defined, into pcre.c, instead of having two | |
| 2821 | separate copies. | |
| 2822 | ||
| 2823 | (b) Defined the list of op-code names for debugging as a macro in | |
| 2824 | internal.h so that it is next to the definition of the opcodes. | |
| 2825 | ||
| 2826 | (c) Defined a table of op-code lengths for simpler skipping along compiled | |
| 2827 | code. This is again a macro in internal.h so that it is next to the | |
| 2828 | definition of the opcodes. | |
| 2829 | ||
| 2830 | 34. Added support for recursive calls to individual subpatterns, along the | |
| 2831 | lines of Robin Houston's patch (but implemented somewhat differently). | |
| 2832 | ||
| 2833 | 35. Further mods to the Makefile to help Win32. Also, added code to pcregrep to | |
| 2834 | allow it to read and process whole directories in Win32. This code was | |
| 2835 | contributed by Lionel Fourquaux; it has not been tested by me. | |
| 2836 | ||
| 2837 | 36. Added support for named subpatterns. The Python syntax (?P<name>...) is | |
| 2838 | used to name a group. Names consist of alphanumerics and underscores, and must | |
| 2839 | be unique. Back references use the syntax (?P=name) and recursive calls use | |
| 2840 | (?P>name) which is a PCRE extension to the Python extension. Groups still have | |
| 2841 | numbers. The function pcre_fullinfo() can be used after compilation to extract | |
| 2842 | a name/number map. There are three relevant calls: | |
| 2843 | ||
| 2844 | PCRE_INFO_NAMEENTRYSIZE yields the size of each entry in the map | |
| 2845 | PCRE_INFO_NAMECOUNT yields the number of entries | |
| 2846 | PCRE_INFO_NAMETABLE yields a pointer to the map. | |
| 2847 | ||
| 2848 | The map is a vector of fixed-size entries. The size of each entry depends on | |
| 2849 | the length of the longest name used. The first two bytes of each entry are the | |
| 2850 | group number, most significant byte first. There follows the corresponding | |
| 2851 | name, zero terminated. The names are in alphabetical order. | |
| 2852 | ||
| 2853 | 37. Make the maximum literal string in the compiled code 250 for the non-UTF-8 | |
| 2854 | case instead of 255. Making it the same both with and without UTF-8 support | |
| 2855 | means that the same test output works with both. | |
| 2856 | ||
| 2857 | 38. There was a case of malloc(0) in the POSIX testing code in pcretest. Avoid | |
| 2858 | calling malloc() with a zero argument. | |
| 2859 | ||
| 2860 | 39. Change 25 above had to resort to a heavy-handed test for the .* anchoring | |
| 2861 | optimization. I've improved things by keeping a bitmap of backreferences with | |
| 2862 | numbers 1-31 so that if .* occurs inside capturing brackets that are not in | |
| 2863 | fact referenced, the optimization can be applied. It is unlikely that a | |
| 2864 | relevant occurrence of .* (i.e. one which might indicate anchoring or forcing | |
| 2865 | the match to follow \n) will appear inside brackets with a number greater than | |
| 2866 | 31, but if it does, any back reference > 31 suppresses the optimization. | |
| 2867 | ||
| 2868 | 40. Added a new compile-time option PCRE_NO_AUTO_CAPTURE. This has the effect | |
| 2869 | of disabling numbered capturing parentheses. Any opening parenthesis that is | |
| 2870 | not followed by ? behaves as if it were followed by ?: but named parentheses | |
| 2871 | can still be used for capturing (and they will acquire numbers in the usual | |
| 2872 | way). | |
| 2873 | ||
| 2874 | 41. Redesigned the return codes from the match() function into yes/no/error so | |
| 2875 | that errors can be passed back from deep inside the nested calls. A malloc | |
| 2876 | failure while inside a recursive subpattern call now causes the | |
| 2877 | PCRE_ERROR_NOMEMORY return instead of quietly going wrong. | |
| 2878 | ||
| 2879 | 42. It is now possible to set a limit on the number of times the match() | |
| 2880 | function is called in a call to pcre_exec(). This facility makes it possible to | |
| 2881 | limit the amount of recursion and backtracking, though not in a directly | |
| 2882 | obvious way, because the match() function is used in a number of different | |
| 2883 | circumstances. The count starts from zero for each position in the subject | |
| 2884 | string (for non-anchored patterns). The default limit is, for compatibility, a | |
| 2885 | large number, namely 10 000 000. You can change this in two ways: | |
| 2886 | ||
| 2887 | (a) When configuring PCRE before making, you can use --with-match-limit=n | |
| 2888 | to set a default value for the compiled library. | |
| 2889 | ||
| 2890 | (b) For each call to pcre_exec(), you can pass a pcre_extra block in which | |
| 2891 | a different value is set. See 45 below. | |
| 2892 | ||
| 2893 | If the limit is exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT. | |
| 2894 | ||
| 2895 | 43. Added a new function pcre_config(int, void *) to enable run-time extraction | |
| 2896 | of things that can be changed at compile time. The first argument specifies | |
| 2897 | what is wanted and the second points to where the information is to be placed. | |
| 2898 | The current list of available information is: | |
| 2899 | ||
| 2900 | PCRE_CONFIG_UTF8 | |
| 2901 | ||
| 2902 | The output is an integer that is set to one if UTF-8 support is available; | |
| 2903 | otherwise it is set to zero. | |
| 2904 | ||
| 2905 | PCRE_CONFIG_NEWLINE | |
| 2906 | ||
| 2907 | The output is an integer that it set to the value of the code that is used for | |
| 2908 | newline. It is either LF (10) or CR (13). | |
| 2909 | ||
| 2910 | PCRE_CONFIG_LINK_SIZE | |
| 2911 | ||
| 2912 | The output is an integer that contains the number of bytes used for internal | |
| 2913 | linkage in compiled expressions. The value is 2, 3, or 4. See item 32 above. | |
| 2914 | ||
| 2915 | PCRE_CONFIG_POSIX_MALLOC_THRESHOLD | |
| 2916 | ||
| 2917 | The output is an integer that contains the threshold above which the POSIX | |
| 2918 | interface uses malloc() for output vectors. See item 31 above. | |
| 2919 | ||
| 2920 | PCRE_CONFIG_MATCH_LIMIT | |
| 2921 | ||
| 2922 | The output is an unsigned integer that contains the default limit of the number | |
| 2923 | of match() calls in a pcre_exec() execution. See 42 above. | |
| 2924 | ||
| 2925 | 44. pcretest has been upgraded by the addition of the -C option. This causes it | |
| 2926 | to extract all the available output from the new pcre_config() function, and to | |
| 2927 | output it. The program then exits immediately. | |
| 2928 | ||
| 2929 | 45. A need has arisen to pass over additional data with calls to pcre_exec() in | |
| 2930 | order to support additional features. One way would have been to define | |
| 2931 | pcre_exec2() (for example) with extra arguments, but this would not have been | |
| 2932 | extensible, and would also have required all calls to the original function to | |
| 2933 | be mapped to the new one. Instead, I have chosen to extend the mechanism that | |
| 2934 | is used for passing in "extra" data from pcre_study(). | |
| 2935 | ||
| 2936 | The pcre_extra structure is now exposed and defined in pcre.h. It currently | |
| 2937 | contains the following fields: | |
| 2938 | ||
| 2939 | flags a bitmap indicating which of the following fields are set | |
| 2940 | study_data opaque data from pcre_study() | |
| 2941 | match_limit a way of specifying a limit on match() calls for a specific | |
| 2942 | call to pcre_exec() | |
| 2943 | callout_data data for callouts (see 49 below) | |
| 2944 | ||
| 2945 | The flag bits are also defined in pcre.h, and are | |
| 2946 | ||
| 2947 | PCRE_EXTRA_STUDY_DATA | |
| 2948 | PCRE_EXTRA_MATCH_LIMIT | |
| 2949 | PCRE_EXTRA_CALLOUT_DATA | |
| 2950 | ||
| 2951 | The pcre_study() function now returns one of these new pcre_extra blocks, with | |
| 2952 | the actual study data pointed to by the study_data field, and the | |
| 2953 | PCRE_EXTRA_STUDY_DATA flag set. This can be passed directly to pcre_exec() as | |
| 2954 | before. That is, this change is entirely upwards-compatible and requires no | |
| 2955 | change to existing code. | |
| 2956 | ||
| 2957 | If you want to pass in additional data to pcre_exec(), you can either place it | |
| 2958 | in a pcre_extra block provided by pcre_study(), or create your own pcre_extra | |
| 2959 | block. | |
| 2960 | ||
| 2961 | 46. pcretest has been extended to test the PCRE_EXTRA_MATCH_LIMIT feature. If a | |
| 2962 | data string contains the escape sequence \M, pcretest calls pcre_exec() several | |
| 2963 | times with different match limits, until it finds the minimum value needed for | |
| 2964 | pcre_exec() to complete. The value is then output. This can be instructive; for | |
| 2965 | most simple matches the number is quite small, but for pathological cases it | |
| 2966 | gets very large very quickly. | |
| 2967 | ||
| 2968 | 47. There's a new option for pcre_fullinfo() called PCRE_INFO_STUDYSIZE. It | |
| 2969 | returns the size of the data block pointed to by the study_data field in a | |
| 2970 | pcre_extra block, that is, the value that was passed as the argument to | |
| 2971 | pcre_malloc() when PCRE was getting memory in which to place the information | |
| 2972 | created by pcre_study(). The fourth argument should point to a size_t variable. | |
| 2973 | pcretest has been extended so that this information is shown after a successful | |
| 2974 | pcre_study() call when information about the compiled regex is being displayed. | |
| 2975 | ||
| 2976 | 48. Cosmetic change to Makefile: there's no need to have / after $(DESTDIR) | |
| 2977 | because what follows is always an absolute path. (Later: it turns out that this | |
| 2978 | is more than cosmetic for MinGW, because it doesn't like empty path | |
| 2979 | components.) | |
| 2980 | ||
| 2981 | 49. Some changes have been made to the callout feature (see 28 above): | |
| 2982 | ||
| 2983 | (i) A callout function now has three choices for what it returns: | |
| 2984 | ||
| 2985 | 0 => success, carry on matching | |
| 2986 | > 0 => failure at this point, but backtrack if possible | |
| 2987 | < 0 => serious error, return this value from pcre_exec() | |
| 2988 | ||
| 2989 | Negative values should normally be chosen from the set of PCRE_ERROR_xxx | |
| 2990 | values. In particular, returning PCRE_ERROR_NOMATCH forces a standard | |
| 2991 | "match failed" error. The error number PCRE_ERROR_CALLOUT is reserved for | |
| 2992 | use by callout functions. It will never be used by PCRE itself. | |
| 2993 | ||
| 2994 | (ii) The pcre_extra structure (see 45 above) has a void * field called | |
| 2995 | callout_data, with corresponding flag bit PCRE_EXTRA_CALLOUT_DATA. The | |
| 2996 | pcre_callout_block structure has a field of the same name. The contents of | |
| 2997 | the field passed in the pcre_extra structure are passed to the callout | |
| 2998 | function in the corresponding field in the callout block. This makes it | |
| 2999 | easier to use the same callout-containing regex from multiple threads. For | |
| 3000 | testing, the pcretest program has a new data escape | |
| 3001 | ||
| 3002 | \C*n pass the number n (may be negative) as callout_data | |
| 3003 | ||
| 3004 | If the callout function in pcretest receives a non-zero value as | |
| 3005 | callout_data, it returns that value. | |
| 3006 | ||
| 3007 | 50. Makefile wasn't handling CFLAGS properly when compiling dftables. Also, | |
| 3008 | there were some redundant $(CFLAGS) in commands that are now specified as | |
| 3009 | $(LINK), which already includes $(CFLAGS). | |
| 3010 | ||
| 3011 | 51. Extensions to UTF-8 support are listed below. These all apply when (a) PCRE | |
| 3012 | has been compiled with UTF-8 support *and* pcre_compile() has been compiled | |
| 3013 | with the PCRE_UTF8 flag. Patterns that are compiled without that flag assume | |
| 3014 | one-byte characters throughout. Note that case-insensitive matching applies | |
| 3015 | only to characters whose values are less than 256. PCRE doesn't support the | |
| 3016 | notion of cases for higher-valued characters. | |
| 3017 | ||
| 3018 | (i) A character class whose characters are all within 0-255 is handled as | |
| 3019 | a bit map, and the map is inverted for negative classes. Previously, a | |
| 3020 | character > 255 always failed to match such a class; however it should | |
| 3021 | match if the class was a negative one (e.g. [^ab]). This has been fixed. | |
| 3022 | ||
| 3023 | (ii) A negated character class with a single character < 255 is coded as | |
| 3024 | "not this character" (OP_NOT). This wasn't working properly when the test | |
| 3025 | character was multibyte, either singly or repeated. | |
| 3026 | ||
| 3027 | (iii) Repeats of multibyte characters are now handled correctly in UTF-8 | |
| 3028 | mode, for example: \x{100}{2,3}. | |
| 3029 | ||
| 3030 | (iv) The character escapes \b, \B, \d, \D, \s, \S, \w, and \W (either | |
| 3031 | singly or repeated) now correctly test multibyte characters. However, | |
| 3032 | PCRE doesn't recognize any characters with values greater than 255 as | |
| 3033 | digits, spaces, or word characters. Such characters always match \D, \S, | |
| 3034 | and \W, and never match \d, \s, or \w. | |
| 3035 | ||
| 3036 | (v) Classes may now contain characters and character ranges with values | |
| 3037 | greater than 255. For example: [ab\x{100}-\x{400}]. | |
| 3038 | ||
| 3039 | (vi) pcregrep now has a --utf-8 option (synonym -u) which makes it call | |
| 3040 | PCRE in UTF-8 mode. | |
| 3041 | ||
| 3042 | 52. The info request value PCRE_INFO_FIRSTCHAR has been renamed | |
| 3043 | PCRE_INFO_FIRSTBYTE because it is a byte value. However, the old name is | |
| 3044 | retained for backwards compatibility. (Note that LASTLITERAL is also a byte | |
| 3045 | value.) | |
| 3046 | ||
| 3047 | 53. The single man page has become too large. I have therefore split it up into | |
| 3048 | a number of separate man pages. These also give rise to individual HTML pages; | |
| 3049 | these are now put in a separate directory, and there is an index.html page that | |
| 3050 | lists them all. Some hyperlinking between the pages has been installed. | |
| 3051 | ||
| 3052 | 54. Added convenience functions for handling named capturing parentheses. | |
| 3053 | ||
| 3054 | 55. Unknown escapes inside character classes (e.g. [\M]) and escapes that | |
| 3055 | aren't interpreted therein (e.g. [\C]) are literals in Perl. This is now also | |
| 3056 | true in PCRE, except when the PCRE_EXTENDED option is set, in which case they | |
| 3057 | are faulted. | |
| 3058 | ||
| 3059 | 56. Introduced HOST_CC and HOST_CFLAGS which can be set in the environment when | |
| 3060 | calling configure. These values are used when compiling the dftables.c program | |
| 3061 | which is run to generate the source of the default character tables. They | |
| 3062 | default to the values of CC and CFLAGS. If you are cross-compiling PCRE, | |
| 3063 | you will need to set these values. | |
| 3064 | ||
| 3065 | 57. Updated the building process for Windows DLL, as provided by Fred Cox. | |
| 3066 | ||
| 3067 | ||
| 3068 | Version 3.9 02-Jan-02 | |
| 3069 | --------------------- | |
| 3070 | ||
| 3071 | 1. A bit of extraneous text had somehow crept into the pcregrep documentation. | |
| 3072 | ||
| 3073 | 2. If --disable-static was given, the building process failed when trying to | |
| 3074 | build pcretest and pcregrep. (For some reason it was using libtool to compile | |
| 3075 | them, which is not right, as they aren't part of the library.) | |
| 3076 | ||
| 3077 | ||
| 3078 | Version 3.8 18-Dec-01 | |
| 3079 | --------------------- | |
| 3080 | ||
| 3081 | 1. The experimental UTF-8 code was completely screwed up. It was packing the | |
| 3082 | bytes in the wrong order. How dumb can you get? | |
| 3083 | ||
| 3084 | ||
| 3085 | Version 3.7 29-Oct-01 | |
| 3086 | --------------------- | |
| 3087 | ||
| 3088 | 1. In updating pcretest to check change 1 of version 3.6, I screwed up. | |
| 3089 | This caused pcretest, when used on the test data, to segfault. Unfortunately, | |
| 3090 | this didn't happen under Solaris 8, where I normally test things. | |
| 3091 | ||
| 3092 | 2. The Makefile had to be changed to make it work on BSD systems, where 'make' | |
| 3093 | doesn't seem to recognize that ./xxx and xxx are the same file. (This entry | |
| 3094 | isn't in ChangeLog distributed with 3.7 because I forgot when I hastily made | |
| 3095 | this fix an hour or so after the initial 3.7 release.) | |
| 3096 | ||
| 3097 | ||
| 3098 | Version 3.6 23-Oct-01 | |
| 3099 | --------------------- | |
| 3100 | ||
| 3101 | 1. Crashed with /(sens|respons)e and \1ibility/ and "sense and sensibility" if | |
| 3102 | offsets passed as NULL with zero offset count. | |
| 3103 | ||
| 3104 | 2. The config.guess and config.sub files had not been updated when I moved to | |
| 3105 | the latest autoconf. | |
| 3106 | ||
| 3107 | ||
| 3108 | Version 3.5 15-Aug-01 | |
| 3109 | --------------------- | |
| 3110 | ||
| 3111 | 1. Added some missing #if !defined NOPOSIX conditionals in pcretest.c that | |
| 3112 | had been forgotten. | |
| 3113 | ||
| 3114 | 2. By using declared but undefined structures, we can avoid using "void" | |
| 3115 | definitions in pcre.h while keeping the internal definitions of the structures | |
| 3116 | private. | |
| 3117 | ||
| 3118 | 3. The distribution is now built using autoconf 2.50 and libtool 1.4. From a | |
| 3119 | user point of view, this means that both static and shared libraries are built | |
| 3120 | by default, but this can be individually controlled. More of the work of | |
| 3121 | handling this static/shared cases is now inside libtool instead of PCRE's make | |
| 3122 | file. | |
| 3123 | ||
| 3124 | 4. The pcretest utility is now installed along with pcregrep because it is | |
| 3125 | useful for users (to test regexs) and by doing this, it automatically gets | |
| 3126 | relinked by libtool. The documentation has been turned into a man page, so | |
| 3127 | there are now .1, .txt, and .html versions in /doc. | |
| 3128 | ||
| 3129 | 5. Upgrades to pcregrep: | |
| 3130 | (i) Added long-form option names like gnu grep. | |
| 3131 | (ii) Added --help to list all options with an explanatory phrase. | |
| 3132 | (iii) Added -r, --recursive to recurse into sub-directories. | |
| 3133 | (iv) Added -f, --file to read patterns from a file. | |
| 3134 | ||
| 3135 | 6. pcre_exec() was referring to its "code" argument before testing that | |
| 3136 | argument for NULL (and giving an error if it was NULL). | |
| 3137 | ||
| 3138 | 7. Upgraded Makefile.in to allow for compiling in a different directory from | |
| 3139 | the source directory. | |
| 3140 | ||
| 3141 | 8. Tiny buglet in pcretest: when pcre_fullinfo() was called to retrieve the | |
| 3142 | options bits, the pointer it was passed was to an int instead of to an unsigned | |
| 3143 | long int. This mattered only on 64-bit systems. | |
| 3144 | ||
| 3145 | 9. Fixed typo (3.4/1) in pcre.h again. Sigh. I had changed pcre.h (which is | |
| 3146 | generated) instead of pcre.in, which it its source. Also made the same change | |
| 3147 | in several of the .c files. | |
| 3148 | ||
| 3149 | 10. A new release of gcc defines printf() as a macro, which broke pcretest | |
| 3150 | because it had an ifdef in the middle of a string argument for printf(). Fixed | |
| 3151 | by using separate calls to printf(). | |
| 3152 | ||
| 3153 | 11. Added --enable-newline-is-cr and --enable-newline-is-lf to the configure | |
| 3154 | script, to force use of CR or LF instead of \n in the source. On non-Unix | |
| 3155 | systems, the value can be set in config.h. | |
| 3156 | ||
| 3157 | 12. The limit of 200 on non-capturing parentheses is a _nesting_ limit, not an | |
| 3158 | absolute limit. Changed the text of the error message to make this clear, and | |
| 3159 | likewise updated the man page. | |
| 3160 | ||
| 3161 | 13. The limit of 99 on the number of capturing subpatterns has been removed. | |
| 3162 | The new limit is 65535, which I hope will not be a "real" limit. | |
| 3163 | ||
| 3164 | ||
| 3165 | Version 3.4 22-Aug-00 | |
| 3166 | --------------------- | |
| 3167 | ||
| 3168 | 1. Fixed typo in pcre.h: unsigned const char * changed to const unsigned char *. | |
| 3169 | ||
| 3170 | 2. Diagnose condition (?(0) as an error instead of crashing on matching. | |
| 3171 | ||
| 3172 | ||
| 3173 | Version 3.3 01-Aug-00 | |
| 3174 | --------------------- | |
| 3175 | ||
| 3176 | 1. If an octal character was given, but the value was greater than \377, it | |
| 3177 | was not getting masked to the least significant bits, as documented. This could | |
| 3178 | lead to crashes in some systems. | |
| 3179 | ||
| 3180 | 2. Perl 5.6 (if not earlier versions) accepts classes like [a-\d] and treats | |
| 3181 | the hyphen as a literal. PCRE used to give an error; it now behaves like Perl. | |
| 3182 | ||
| 3183 | 3. Added the functions pcre_free_substring() and pcre_free_substring_list(). | |
| 3184 | These just pass their arguments on to (pcre_free)(), but they are provided | |
| 3185 | because some uses of PCRE bind it to non-C systems that can call its functions, | |
| 3186 | but cannot call free() or pcre_free() directly. | |
| 3187 | ||
| 3188 | 4. Add "make test" as a synonym for "make check". Corrected some comments in | |
| 3189 | the Makefile. | |
| 3190 | ||
| 3191 | 5. Add $(DESTDIR)/ in front of all the paths in the "install" target in the | |
| 3192 | Makefile. | |
| 3193 | ||
| 3194 | 6. Changed the name of pgrep to pcregrep, because Solaris has introduced a | |
| 3195 | command called pgrep for grepping around the active processes. | |
| 3196 | ||
| 3197 | 7. Added the beginnings of support for UTF-8 character strings. | |
| 3198 | ||
| 3199 | 8. Arranged for the Makefile to pass over the settings of CC, CFLAGS, and | |
| 3200 | RANLIB to ./ltconfig so that they are used by libtool. I think these are all | |
| 3201 | the relevant ones. (AR is not passed because ./ltconfig does its own figuring | |
| 3202 | out for the ar command.) | |
| 3203 | ||
| 3204 | ||
| 3205 | Version 3.2 12-May-00 | |
| 3206 | --------------------- | |
| 3207 | ||
| 3208 | This is purely a bug fixing release. | |
| 3209 | ||
| 3210 | 1. If the pattern /((Z)+|A)*/ was matched agained ZABCDEFG it matched Z instead | |
| 3211 | of ZA. This was just one example of several cases that could provoke this bug, | |
| 3212 | which was introduced by change 9 of version 2.00. The code for breaking | |
| 3213 | infinite loops after an iteration that matches an empty string was't working | |
| 3214 | correctly. | |
| 3215 | ||
| 3216 | 2. The pcretest program was not imitating Perl correctly for the pattern /a*/g | |
| 3217 | when matched against abbab (for example). After matching an empty string, it | |
| 3218 | wasn't forcing anchoring when setting PCRE_NOTEMPTY for the next attempt; this | |
| 3219 | caused it to match further down the string than it should. | |
| 3220 | ||
| 3221 | 3. The code contained an inclusion of sys/types.h. It isn't clear why this | |
| 3222 | was there because it doesn't seem to be needed, and it causes trouble on some | |
| 3223 | systems, as it is not a Standard C header. It has been removed. | |
| 3224 | ||
| 3225 | 4. Made 4 silly changes to the source to avoid stupid compiler warnings that | |
| 3226 | were reported on the Macintosh. The changes were from | |
| 3227 | ||
| 3228 | while ((c = *(++ptr)) != 0 && c != '\n'); | |
| 3229 | to | |
| 3230 | while ((c = *(++ptr)) != 0 && c != '\n') ; | |
| 3231 | ||
| 3232 | Totally extraordinary, but if that's what it takes... | |
| 3233 | ||
| 3234 | 5. PCRE is being used in one environment where neither memmove() nor bcopy() is | |
| 3235 | available. Added HAVE_BCOPY and an autoconf test for it; if neither | |
| 3236 | HAVE_MEMMOVE nor HAVE_BCOPY is set, use a built-in emulation function which | |
| 3237 | assumes the way PCRE uses memmove() (always moving upwards). | |
| 3238 | ||
| 3239 | 6. PCRE is being used in one environment where strchr() is not available. There | |
| 3240 | was only one use in pcre.c, and writing it out to avoid strchr() probably gives | |
| 3241 | faster code anyway. | |
| 3242 | ||
| 3243 | ||
| 3244 | Version 3.1 09-Feb-00 | |
| 3245 | --------------------- | |
| 3246 | ||
| 3247 | The only change in this release is the fixing of some bugs in Makefile.in for | |
| 3248 | the "install" target: | |
| 3249 | ||
| 3250 | (1) It was failing to install pcreposix.h. | |
| 3251 | ||
| 3252 | (2) It was overwriting the pcre.3 man page with the pcreposix.3 man page. | |
| 3253 | ||
| 3254 | ||
| 3255 | Version 3.0 01-Feb-00 | |
| 3256 | --------------------- | |
| 3257 | ||
| 3258 | 1. Add support for the /+ modifier to perltest (to output $` like it does in | |
| 3259 | pcretest). | |
| 3260 | ||
| 3261 | 2. Add support for the /g modifier to perltest. | |
| 3262 | ||
| 3263 | 3. Fix pcretest so that it behaves even more like Perl for /g when the pattern | |
| 3264 | matches null strings. | |
| 3265 | ||
| 3266 | 4. Fix perltest so that it doesn't do unwanted things when fed an empty | |
| 3267 | pattern. Perl treats empty patterns specially - it reuses the most recent | |
| 3268 | pattern, which is not what we want. Replace // by /(?#)/ in order to avoid this | |
| 3269 | effect. | |
| 3270 | ||
| 3271 | 5. The POSIX interface was broken in that it was just handing over the POSIX | |
| 3272 | captured string vector to pcre_exec(), but (since release 2.00) PCRE has | |
| 3273 | required a bigger vector, with some working space on the end. This means that | |
| 3274 | the POSIX wrapper now has to get and free some memory, and copy the results. | |
| 3275 | ||
| 3276 | 6. Added some simple autoconf support, placing the test data and the | |
| 3277 | documentation in separate directories, re-organizing some of the | |
| 3278 | information files, and making it build pcre-config (a GNU standard). Also added | |
| 3279 | libtool support for building PCRE as a shared library, which is now the | |
| 3280 | default. | |
| 3281 | ||
| 3282 | 7. Got rid of the leading zero in the definition of PCRE_MINOR because 08 and | |
| 3283 | 09 are not valid octal constants. Single digits will be used for minor values | |
| 3284 | less than 10. | |
| 3285 | ||
| 3286 | 8. Defined REG_EXTENDED and REG_NOSUB as zero in the POSIX header, so that | |
| 3287 | existing programs that set these in the POSIX interface can use PCRE without | |
| 3288 | modification. | |
| 3289 | ||
| 3290 | 9. Added a new function, pcre_fullinfo() with an extensible interface. It can | |
| 3291 | return all that pcre_info() returns, plus additional data. The pcre_info() | |
| 3292 | function is retained for compatibility, but is considered to be obsolete. | |
| 3293 | ||
| 3294 | 10. Added experimental recursion feature (?R) to handle one common case that | |
| 3295 | Perl 5.6 will be able to do with (?p{...}). | |
| 3296 | ||
| 3297 | 11. Added support for POSIX character classes like [:alpha:], which Perl is | |
| 3298 | adopting. | |
| 3299 | ||
| 3300 | ||
| 3301 | Version 2.08 31-Aug-99 | |
| 3302 | ---------------------- | |
| 3303 | ||
| 3304 | 1. When startoffset was not zero and the pattern began with ".*", PCRE was not | |
| 3305 | trying to match at the startoffset position, but instead was moving forward to | |
| 3306 | the next newline as if a previous match had failed. | |
| 3307 | ||
| 3308 | 2. pcretest was not making use of PCRE_NOTEMPTY when repeating for /g and /G, | |
| 3309 | and could get into a loop if a null string was matched other than at the start | |
| 3310 | of the subject. | |
| 3311 | ||
| 3312 | 3. Added definitions of PCRE_MAJOR and PCRE_MINOR to pcre.h so the version can | |
| 3313 | be distinguished at compile time, and for completeness also added PCRE_DATE. | |
| 3314 | ||
| 3315 | 5. Added Paul Sokolovsky's minor changes to make it easy to compile a Win32 DLL | |
| 3316 | in GnuWin32 environments. | |
| 3317 | ||
| 3318 | ||
| 3319 | Version 2.07 29-Jul-99 | |
| 3320 | ---------------------- | |
| 3321 | ||
| 3322 | 1. The documentation is now supplied in plain text form and HTML as well as in | |
| 3323 | the form of man page sources. | |
| 3324 | ||
| 3325 | 2. C++ compilers don't like assigning (void *) values to other pointer types. | |
| 3326 | In particular this affects malloc(). Although there is no problem in Standard | |
| 3327 | C, I've put in casts to keep C++ compilers happy. | |
| 3328 | ||
| 3329 | 3. Typo on pcretest.c; a cast of (unsigned char *) in the POSIX regexec() call | |
| 3330 | should be (const char *). | |
| 3331 | ||
| 3332 | 4. If NOPOSIX is defined, pcretest.c compiles without POSIX support. This may | |
| 3333 | be useful for non-Unix systems who don't want to bother with the POSIX stuff. | |
| 3334 | However, I haven't made this a standard facility. The documentation doesn't | |
| 3335 | mention it, and the Makefile doesn't support it. | |
| 3336 | ||
| 3337 | 5. The Makefile now contains an "install" target, with editable destinations at | |
| 3338 | the top of the file. The pcretest program is not installed. | |
| 3339 | ||
| 3340 | 6. pgrep -V now gives the PCRE version number and date. | |
| 3341 | ||
| 3342 | 7. Fixed bug: a zero repetition after a literal string (e.g. /abcde{0}/) was | |
| 3343 | causing the entire string to be ignored, instead of just the last character. | |
| 3344 | ||
| 3345 | 8. If a pattern like /"([^\\"]+|\\.)*"/ is applied in the normal way to a | |
| 3346 | non-matching string, it can take a very, very long time, even for strings of | |
| 3347 | quite modest length, because of the nested recursion. PCRE now does better in | |
| 3348 | some of these cases. It does this by remembering the last required literal | |
| 3349 | character in the pattern, and pre-searching the subject to ensure it is present | |
| 3350 | before running the real match. In other words, it applies a heuristic to detect | |
| 3351 | some types of certain failure quickly, and in the above example, if presented | |
| 3352 | with a string that has no trailing " it gives "no match" very quickly. | |
| 3353 | ||
| 3354 | 9. A new runtime option PCRE_NOTEMPTY causes null string matches to be ignored; | |
| 3355 | other alternatives are tried instead. | |
| 3356 | ||
| 3357 | ||
| 3358 | Version 2.06 09-Jun-99 | |
| 3359 | ---------------------- | |
| 3360 | ||
| 3361 | 1. Change pcretest's output for amount of store used to show just the code | |
| 3362 | space, because the remainder (the data block) varies in size between 32-bit and | |
| 3363 | 64-bit systems. | |
| 3364 | ||
| 3365 | 2. Added an extra argument to pcre_exec() to supply an offset in the subject to | |
| 3366 | start matching at. This allows lookbehinds to work when searching for multiple | |
| 3367 | occurrences in a string. | |
| 3368 | ||
| 3369 | 3. Added additional options to pcretest for testing multiple occurrences: | |
| 3370 | ||
| 3371 | /+ outputs the rest of the string that follows a match | |
| 3372 | /g loops for multiple occurrences, using the new startoffset argument | |
| 3373 | /G loops for multiple occurrences by passing an incremented pointer | |
| 3374 | ||
| 3375 | 4. PCRE wasn't doing the "first character" optimization for patterns starting | |
| 3376 | with \b or \B, though it was doing it for other lookbehind assertions. That is, | |
| 3377 | it wasn't noticing that a match for a pattern such as /\bxyz/ has to start with | |
| 3378 | the letter 'x'. On long subject strings, this gives a significant speed-up. | |
| 3379 | ||
| 3380 | ||
| 3381 | Version 2.05 21-Apr-99 | |
| 3382 | ---------------------- | |
| 3383 | ||
| 3384 | 1. Changed the type of magic_number from int to long int so that it works | |
| 3385 | properly on 16-bit systems. | |
| 3386 | ||
| 3387 | 2. Fixed a bug which caused patterns starting with .* not to work correctly | |
| 3388 | when the subject string contained newline characters. PCRE was assuming | |
| 3389 | anchoring for such patterns in all cases, which is not correct because .* will | |
| 3390 | not pass a newline unless PCRE_DOTALL is set. It now assumes anchoring only if | |
| 3391 | DOTALL is set at top level; otherwise it knows that patterns starting with .* | |
| 3392 | must be retried after every newline in the subject. | |
| 3393 | ||
| 3394 | ||
| 3395 | Version 2.04 18-Feb-99 | |
| 3396 | ---------------------- | |
| 3397 | ||
| 3398 | 1. For parenthesized subpatterns with repeats whose minimum was zero, the | |
| 3399 | computation of the store needed to hold the pattern was incorrect (too large). | |
| 3400 | If such patterns were nested a few deep, this could multiply and become a real | |
| 3401 | problem. | |
| 3402 | ||
| 3403 | 2. Added /M option to pcretest to show the memory requirement of a specific | |
| 3404 | pattern. Made -m a synonym of -s (which does this globally) for compatibility. | |
| 3405 | ||
| 3406 | 3. Subpatterns of the form (regex){n,m} (i.e. limited maximum) were being | |
| 3407 | compiled in such a way that the backtracking after subsequent failure was | |
| 3408 | pessimal. Something like (a){0,3} was compiled as (a)?(a)?(a)? instead of | |
| 3409 | ((a)((a)(a)?)?)? with disastrous performance if the maximum was of any size. | |
| 3410 | ||
| 3411 | ||
| 3412 | Version 2.03 02-Feb-99 | |
| 3413 | ---------------------- | |
| 3414 | ||
| 3415 | 1. Fixed typo and small mistake in man page. | |
| 3416 | ||
| 3417 | 2. Added 4th condition (GPL supersedes if conflict) and created separate | |
| 3418 | LICENCE file containing the conditions. | |
| 3419 | ||
| 3420 | 3. Updated pcretest so that patterns such as /abc\/def/ work like they do in | |
| 3421 | Perl, that is the internal \ allows the delimiter to be included in the | |
| 3422 | pattern. Locked out the use of \ as a delimiter. If \ immediately follows | |
| 3423 | the final delimiter, add \ to the end of the pattern (to test the error). | |
| 3424 | ||
| 3425 | 4. Added the convenience functions for extracting substrings after a successful | |
| 3426 | match. Updated pcretest to make it able to test these functions. | |
| 3427 | ||
| 3428 | ||
| 3429 | Version 2.02 14-Jan-99 | |
| 3430 | ---------------------- | |
| 3431 | ||
| 3432 | 1. Initialized the working variables associated with each extraction so that | |
| 3433 | their saving and restoring doesn't refer to uninitialized store. | |
| 3434 | ||
| 3435 | 2. Put dummy code into study.c in order to trick the optimizer of the IBM C | |
| 3436 | compiler for OS/2 into generating correct code. Apparently IBM isn't going to | |
| 3437 | fix the problem. | |
| 3438 | ||
| 3439 | 3. Pcretest: the timing code wasn't using LOOPREPEAT for timing execution | |
| 3440 | calls, and wasn't printing the correct value for compiling calls. Increased the | |
| 3441 | default value of LOOPREPEAT, and the number of significant figures in the | |
| 3442 | times. | |
| 3443 | ||
| 3444 | 4. Changed "/bin/rm" in the Makefile to "-rm" so it works on Windows NT. | |
| 3445 | ||
| 3446 | 5. Renamed "deftables" as "dftables" to get it down to 8 characters, to avoid | |
| 3447 | a building problem on Windows NT with a FAT file system. | |
| 3448 | ||
| 3449 | ||
| 3450 | Version 2.01 21-Oct-98 | |
| 3451 | ---------------------- | |
| 3452 | ||
| 3453 | 1. Changed the API for pcre_compile() to allow for the provision of a pointer | |
| 3454 | to character tables built by pcre_maketables() in the current locale. If NULL | |
| 3455 | is passed, the default tables are used. | |
| 3456 | ||
| 3457 | ||
| 3458 | Version 2.00 24-Sep-98 | |
| 3459 | ---------------------- | |
| 3460 | ||
| 3461 | 1. Since the (>?) facility is in Perl 5.005, don't require PCRE_EXTRA to enable | |
| 3462 | it any more. | |
| 3463 | ||
| 3464 | 2. Allow quantification of (?>) groups, and make it work correctly. | |
| 3465 | ||
| 3466 | 3. The first character computation wasn't working for (?>) groups. | |
| 3467 | ||
| 3468 | 4. Correct the implementation of \Z (it is permitted to match on the \n at the | |
| 3469 | end of the subject) and add 5.005's \z, which really does match only at the | |
| 3470 | very end of the subject. | |
| 3471 | ||
| 3472 | 5. Remove the \X "cut" facility; Perl doesn't have it, and (?> is neater. | |
| 3473 | ||
| 3474 | 6. Remove the ability to specify CASELESS, MULTILINE, DOTALL, and | |
| 3475 | DOLLAR_END_ONLY at runtime, to make it possible to implement the Perl 5.005 | |
| 3476 | localized options. All options to pcre_study() were also removed. | |
| 3477 | ||
| 3478 | 7. Add other new features from 5.005: | |
| 3479 | ||
| 3480 | $(?<= positive lookbehind | |
| 3481 | $(?<! negative lookbehind | |
| 3482 | (?imsx-imsx) added the unsetting capability | |
| 3483 | such a setting is global if at outer level; local otherwise | |
| 3484 | (?imsx-imsx:) non-capturing groups with option setting | |
| 3485 | (?(cond)re|re) conditional pattern matching | |
| 3486 | ||
| 3487 | A backreference to itself in a repeated group matches the previous | |
| 3488 | captured string. | |
| 3489 | ||
| 3490 | 8. General tidying up of studying (both automatic and via "study") | |
| 3491 | consequential on the addition of new assertions. | |
| 3492 | ||
| 3493 | 9. As in 5.005, unlimited repeated groups that could match an empty substring | |
| 3494 | are no longer faulted at compile time. Instead, the loop is forcibly broken at | |
| 3495 | runtime if any iteration does actually match an empty substring. | |
| 3496 | ||
| 3497 | 10. Include the RunTest script in the distribution. | |
| 3498 | ||
| 3499 | 11. Added tests from the Perl 5.005_02 distribution. This showed up a few | |
| 3500 | discrepancies, some of which were old and were also with respect to 5.004. They | |
| 3501 | have now been fixed. | |
| 3502 | ||
| 3503 | ||
| 3504 | Version 1.09 28-Apr-98 | |
| 3505 | ---------------------- | |
| 3506 | ||
| 3507 | 1. A negated single character class followed by a quantifier with a minimum | |
| 3508 | value of one (e.g. [^x]{1,6} ) was not compiled correctly. This could lead to | |
| 3509 | program crashes, or just wrong answers. This did not apply to negated classes | |
| 3510 | containing more than one character, or to minima other than one. | |
| 3511 | ||
| 3512 | ||
| 3513 | Version 1.08 27-Mar-98 | |
| 3514 | ---------------------- | |
| 3515 | ||
| 3516 | 1. Add PCRE_UNGREEDY to invert the greediness of quantifiers. | |
| 3517 | ||
| 3518 | 2. Add (?U) and (?X) to set PCRE_UNGREEDY and PCRE_EXTRA respectively. The | |
| 3519 | latter must appear before anything that relies on it in the pattern. | |
| 3520 | ||
| 3521 | ||
| 3522 | Version 1.07 16-Feb-98 | |
| 3523 | ---------------------- | |
| 3524 | ||
| 3525 | 1. A pattern such as /((a)*)*/ was not being diagnosed as in error (unlimited | |
| 3526 | repeat of a potentially empty string). | |
| 3527 | ||
| 3528 | ||
| 3529 | Version 1.06 23-Jan-98 | |
| 3530 | ---------------------- | |
| 3531 | ||
| 3532 | 1. Added Markus Oberhumer's little patches for C++. | |
| 3533 | ||
| 3534 | 2. Literal strings longer than 255 characters were broken. | |
| 3535 | ||
| 3536 | ||
| 3537 | Version 1.05 23-Dec-97 | |
| 3538 | ---------------------- | |
| 3539 | ||
| 3540 | 1. Negated character classes containing more than one character were failing if | |
| 3541 | PCRE_CASELESS was set at run time. | |
| 3542 | ||
| 3543 | ||
| 3544 | Version 1.04 19-Dec-97 | Version 1.04 19-Dec-97 |
| 3545 | ---------------------- | ---------------------- |