--- code/trunk/ChangeLog 2011/05/09 08:54:11 600 +++ code/trunk/ChangeLog 2011/07/29 15:56:39 643 @@ -43,6 +43,175 @@ documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the behaviour of \R* different to (\R)*, which also seems wrong. The behaviour has been changed. + +9. Some internal refactoring has changed the processing so that the handling + of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile + time (the PCRE_DOTALL option was changed this way some time ago: version + 7.7 change 16). This has made it possible to abolish the OP_OPT op code, + which was always a bit of a fudge. It also means that there is one less + argument for the match() function, which reduces its stack requirements + slightly. This change also fixes an incompatibility with Perl: the pattern + (?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. + +10. More internal refactoring has drastically reduced the number of recursive + calls to match() for possessively repeated groups such as (abc)++ when + using pcre_exec(). + +11. While implementing 10, a number of bugs in the handling of groups were + discovered and fixed: + + (?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). + (a|)*(?1) gave a compile-time internal error. + ((a|)+)+ did not notice that the outer group could match an empty string. + (^a|^)+ was not marked as anchored. + (.*a|.*)+ was not marked as matching at start or after a newline. + +12. Yet more internal refactoring has removed another argument from the match() + function. Special calls to this function are now indicated by setting a + value in a variable in the "match data" data block. + +13. Be more explicit in pcre_study() instead of relying on "default" for + opcodes that mean there is no starting character; this means that when new + ones are added and accidentally left out of pcre_study(), testing should + pick them up. + +14. The -s option of pcretest has been documented for ages as being an old + synonym of -m (show memory usage). I have changed it to mean "force study + for every regex", that is, assume /S for every regex. This is similar to -i + and -d etc. It's slightly incompatible, but I'm hoping nobody is still + using it. It makes it easier to run collections of tests with and without + study enabled, and thereby test pcre_study() more easily. All the standard + tests are now run with and without -s (but some patterns can be marked as + "never study" - see 20 below). + +15. When (*ACCEPT) was used in a subpattern that was called recursively, the + restoration of the capturing data to the outer values was not happening + correctly. + +16. If a recursively called subpattern ended with (*ACCEPT) and matched an + empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole + pattern had matched an empty string, and so incorrectly returned a no + match. + +17. There was optimizing code for the last branch of non-capturing parentheses, + and also for the obeyed branch of a conditional subexpression, which used + tail recursion to cut down on stack usage. Unfortunately, not that there is + the possibility of (*THEN) occurring in these branches, tail recursion is + no longer possible because the return has to be checked for (*THEN). These + two optimizations have therefore been removed. + +18. If a pattern containing \R was studied, it was assumed that \R always + matched two bytes, thus causing the minimum subject length to be + incorrectly computed because \R can also match just one byte. + +19. If a pattern containing (*ACCEPT) was studied, the minimum subject length + was incorrectly computed. + +20. If /S is present twice on a test pattern in pcretest input, it *disables* + studying, thereby overriding the use of -s on the command line. This is + necessary for one or two tests to keep the output identical in both cases. + +21. When (*ACCEPT) was used in an assertion that matched an empty string and + PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. + +22. When an atomic group that contained a capturing parenthesis was + successfully matched, but the branch in which it appeared failed, the + capturing was not being forgotten if a higher numbered group was later + captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing + group 1 to "a", when in fact it should be unset. This applied to multi- + branched capturing and non-capturing groups, repeated or not, and also to + positive assertions (capturing in negative assertions is not well defined + in PCRE) and also to nested atomic groups. + +23. Add the ++ qualifier feature to pcretest, to show the remainder of the + subject after a captured substring (to make it easier to tell which of a + number of identical substrings has been captured). + +24. The way atomic groups are processed by pcre_exec() has been changed so that + if they are repeated, backtracking one repetition now resets captured + values correctly. For example, if ((?>(a+)b)+aabab) is matched against + "aaaabaaabaabab" the value of captured group 2 is now correctly recorded as + "aaa". Previously, it would have been "a". As part of this code + refactoring, the way recursive calls are handled has also been changed. + +24. If an assertion condition captured any substrings, they were not passed + back unless some other capturing happened later. For example, if + (?(?=(a))a) was matched against "a", no capturing was returned. + +25. When studying a pattern that contained subroutine calls or assertions, + the code for finding the minimum length of a possible match was handling + direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where + group 1 called group 2 while simultaneously a separate group 2 called group + 1). A stack overflow occurred in this case. I have fixed this by limiting + the recursion depth to 10. + +26. Updated RunTest.bat in the distribution to the version supplied by Tom + Fortmann. This supports explicit test numbers on the command line, and has + argument validation and error reporting. + +27. An instance of \X with an unlimited repeat could fail if at any point the + first character it looked at was a mark character. + +28. Some minor code refactoring concerning Unicode properties and scripts + should reduce the stack requirement of match() slightly. + +29. Added the '=' option to pcretest to check the setting of unused capturing + slots at the end of the pattern, which are documented as being -1, but are + not included in the return count. + +30. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE + compiled something random. Now it gives a compile-time error (as does + Perl). + +31. A *MARK encountered during the processing of a positive assertion is now + recorded and passed back (compatible with Perl). + +32. If --only-matching or --colour was set on a pcregrep call whose pattern + had alternative anchored branches, the search for a second match in a line + was done as if at the line start. Thus, for example, /^01|^02/ incorrectly + matched the line "0102" twice. The same bug affected patterns that started + with a backwards assertion. For example /\b01|\b02/ also matched "0102" + twice. + +33. Previously, PCRE did not allow quantification of assertions. However, Perl + does, and because of capturing effects, quantifying parenthesized + assertions may at times be useful. Quantifiers are now allowed for + parenthesized assertions. + +34. A minor code tidy in pcre_compile() when checking options for \R usage. + +35. \g was being checked for fancy things in a character class, when it should + just be a literal "g". + +36. PCRE was rejecting [:a[:digit:]] whereas Perl was not. It seems that the + appearance of a nested POSIX class supersedes an apparent external class. + For example, [:a[:digit:]b:] matches "a", "b", ":", or a digit. Also, + unescaped square brackets may also appear as part of class names. For + example, [:a[:abc]b:] gives unknown class "[:abc]b:]". PCRE now behaves + more like Perl. + +37. PCRE was giving an error for \N with a braced quantifier such as {1,} (this + was because it thought it was \N{name}, which is not supported). + +38. Add minix to OS list not supporting the -S option in pcretest. + +39. PCRE tries to detect cases of infinite recursion at compile time, but it + cannot analyze patterns in sufficient detail to catch mutual recursions + such as ((?1))((?2)). There is now a runtime test that gives an error if a + subgroup is called recursively as a subpattern for a second time at the + same position in the subject string. In previous releases this might have + been caught by the recursion limit, or it might have run out of stack. + +40. A pattern such as /(?(R)a+|(?R)b)/ is quite safe, as the recursion can + happen only once. PCRE was, however incorrectly giving a compile time error + "recursive call could loop indefinitely" because it cannot analyze the + pattern in sufficient detail. The compile time test no longer happens when + PCRE is compiling a conditional subpattern, but actual runaway loops are + now caught at runtime (see 39 above). + +41. It seems that Perl allows any characters other than a closing parenthesis + to be part of the NAME in (*MARK:NAME) and other backtracking verbs. PCRE + has been changed to be the same. Version 8.12 15-Jan-2011