--- code/trunk/ChangeLog 2011/09/06 09:16:32 681 +++ code/trunk/ChangeLog 2011/10/21 09:04:01 738 @@ -1,43 +1,119 @@ ChangeLog for PCRE ------------------ -Version 8.20 ------------- +Version 8.20 21-Oct-2011 +------------------------ + +1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had + a POSIX class. After further experiments with Perl, which convinced me that + Perl has bugs and confusions, a closing square bracket is no longer allowed + in a POSIX name. This bug also affected patterns with classes that started + with full stops. + +2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no + captured substring, but while checking the failing first alternative, + substring 1 is temporarily captured. If the output vector supplied to + pcre_exec() was not big enough for this capture, the yield of the function + was still zero ("insufficient space for captured substrings"). This cannot + be totally fixed without adding another stack variable, which seems a lot + of expense for a edge case. However, I have improved the situation in cases + such as /(a)(b)x|abc/ matched against "abc", where the return code + indicates that fewer than the maximum number of slots in the ovector have + been set. + +3. Related to (2) above: when there are more back references in a pattern than + slots in the output vector, pcre_exec() uses temporary memory during + matching, and copies in the captures as far as possible afterwards. It was + using the entire output vector, but this conflicts with the specification + that only 2/3 is used for passing back captured substrings. Now it uses + only the first 2/3, for compatibility. This is, of course, another edge + case. -1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had - a POSIX class. After further experiments with Perl, which convinced me that - Perl has bugs and confusions, a closing square bracket is no longer allowed - in a POSIX name. - -2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no captured - substring, but while checking the failing first alternative, substring 1 is - temporarily captured. If the output vector supplied to pcre_exec() was not - big enough for this capture, the yield of the function was still zero - ("insufficient space for captured substrings"). This cannot be totally fixed - without adding another stack variable, which seems a lot of expense for a - edge case. However, I have improved the situation in cases such as - /(a)(b)x|abc/ matched against "abc", where the return code indicates that - fewer than the maximum number of slots in the ovector have been set. - -3. Related to (2) above: when there are more back references in a pattern than - slots in the output vector, pcre_exec() uses temporary memory during - matching, and copies in the captures as far as possible afterwards. It was - using the entire output vector, but this conflicts with the specification - that only 2/3 is used for passing back captured substrings. Now it uses only - the first 2/3, for compatibility. This is, of course, another edge case. - -4. Zoltan Herczeg's just-in-time compiler support has been integrated into the - main code base, and can be used by building with --enable-jit. - -5. When the number of matches in a pcre_dfa_exec() run exactly filled the - ovector, the return from the function was zero, implying that there were - other matches that did not fit. The correct "exactly full" value is now - returned. - -6. If a subpattern that was called recursively or as a subroutine contained - (*PRUNE) or any other control that caused it to give a non-standard return, - invalid errors such as "Error -26 (nested recursion at the same subject - position)" or even infinite loops could occur. +4. Zoltan Herczeg's just-in-time compiler support has been integrated into the + main code base, and can be used by building with --enable-jit. When this is + done, pcregrep automatically uses it unless --disable-pcregrep-jit or the + runtime --no-jit option is given. + +5. When the number of matches in a pcre_dfa_exec() run exactly filled the + ovector, the return from the function was zero, implying that there were + other matches that did not fit. The correct "exactly full" value is now + returned. + +6. If a subpattern that was called recursively or as a subroutine contained + (*PRUNE) or any other control that caused it to give a non-standard return, + invalid errors such as "Error -26 (nested recursion at the same subject + position)" or even infinite loops could occur. + +7. If a pattern such as /a(*SKIP)c|b(*ACCEPT)|/ was studied, it stopped + computing the minimum length on reaching *ACCEPT, and so ended up with the + wrong value of 1 rather than 0. Further investigation indicates that + computing a minimum subject length in the presence of *ACCEPT is difficult + (think back references, subroutine calls), and so I have changed the code + so that no minimum is registered for a pattern that contains *ACCEPT. + +8. If (*THEN) was present in the first (true) branch of a conditional group, + it was not handled as intended. [But see 16 below.] + +9. Replaced RunTest.bat and CMakeLists.txt with improved versions provided by + Sheri Pierce. + +10. A pathological pattern such as /(*ACCEPT)a/ was miscompiled, thinking that + the first byte in a match must be "a". + +11. Change 17 for 8.13 increased the recursion depth for patterns like + /a(?:.)*?a/ drastically. I've improved things by remembering whether a + pattern contains any instances of (*THEN). If it does not, the old + optimizations are restored. It would be nice to do this on a per-group + basis, but at the moment that is not feasible. + +12. In some environments, the output of pcretest -C is CRLF terminated. This + broke RunTest's code that checks for the link size. A single white space + character after the value is now allowed for. + +13. RunTest now checks for the "fr" locale as well as for "fr_FR" and "french". + For "fr", it uses the Windows-specific input and output files. + +14. If (*THEN) appeared in a group that was called recursively or as a + subroutine, it did not work as intended. [But see next item.] + +15. Consider the pattern /A (B(*THEN)C) | D/ where A, B, C, and D are complex + pattern fragments (but not containing any | characters). If A and B are + matched, but there is a failure in C so that it backtracks to (*THEN), PCRE + was behaving differently to Perl. PCRE backtracked into A, but Perl goes to + D. In other words, Perl considers parentheses that do not contain any | + characters to be part of a surrounding alternative, whereas PCRE was + treading (B(*THEN)C) the same as (B(*THEN)C|(*FAIL)) -- which Perl handles + differently. PCRE now behaves in the same way as Perl, except in the case + of subroutine/recursion calls such as (?1) which have in any case always + been different (but PCRE had them first :-). + +16. Related to 15 above: Perl does not treat the | in a conditional group as + creating alternatives. Such a group is treated in the same way as an + ordinary group without any | characters when processing (*THEN). PCRE has + been changed to match Perl's behaviour. + +17. If a user had set PCREGREP_COLO(U)R to something other than 1:31, the + RunGrepTest script failed. + +18. Change 22 for version 13 caused atomic groups to use more stack. This is + inevitable for groups that contain captures, but it can lead to a lot of + stack use in large patterns. The old behaviour has been restored for atomic + groups that do not contain any capturing parentheses. + +19. If the PCRE_NO_START_OPTIMIZE option was set for pcre_compile(), it did not + suppress the check for a minimum subject length at run time. (If it was + given to pcre_exec() or pcre_dfa_exec() it did work.) + +20. Fixed an ASCII-dependent infelicity in pcretest that would have made it + fail to work when decoding hex characters in data strings in EBCDIC + environments. + +21. It appears that in at least one Mac OS environment, the isxdigit() function + is implemented as a macro that evaluates to its argument more than once, + contravening the C 90 Standard (I haven't checked a later standard). There + was an instance in pcretest which caused it to go wrong when processing + \x{...} escapes in subject strings. The has been rewritten to avoid using + things like p++ in the argument of isxdigit(). Version 8.13 16-Aug-2011 @@ -137,7 +213,7 @@ tail recursion to cut down on stack usage. Unfortunately, now that there is the possibility of (*THEN) occurring in these branches, tail recursion is no longer possible because the return has to be checked for (*THEN). These - two optimizations have therefore been removed. + two optimizations have therefore been removed. [But see 8.20/11 above.] 18. If a pattern containing \R was studied, it was assumed that \R always matched two bytes, thus causing the minimum subject length to be