| 1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
| 2 |
------------------ |
------------------ |
| 3 |
|
|
| 4 |
|
Version 6.5 01-Feb-06 |
| 5 |
|
--------------------- |
| 6 |
|
|
| 7 |
|
1. When using the partial match feature with pcre_dfa_exec(), it was not |
| 8 |
|
anchoring the second and subsequent partial matches at the new starting |
| 9 |
|
point. This could lead to incorrect results. For example, with the pattern |
| 10 |
|
/1234/, partially matching against "123" and then "a4" gave a match. |
| 11 |
|
|
| 12 |
|
2. Changes to pcregrep: |
| 13 |
|
|
| 14 |
|
(a) All non-match returns from pcre_exec() were being treated as failures |
| 15 |
|
to match the line. Now, unless the error is PCRE_ERROR_NOMATCH, an |
| 16 |
|
error message is output. Some extra information is given for the |
| 17 |
|
PCRE_ERROR_MATCHLIMIT and PCRE_ERROR_RECURSIONLIMIT errors, which are |
| 18 |
|
probably the only errors that are likely to be caused by users (by |
| 19 |
|
specifying a regex that has nested indefinite repeats, for instance). |
| 20 |
|
If there are more than 20 of these errors, pcregrep is abandoned. |
| 21 |
|
|
| 22 |
|
(b) A binary zero was treated as data while matching, but terminated the |
| 23 |
|
output line if it was written out. This has been fixed: binary zeroes |
| 24 |
|
are now no different to any other data bytes. |
| 25 |
|
|
| 26 |
|
(c) Whichever of the LC_ALL or LC_CTYPE environment variables is set is |
| 27 |
|
used to set a locale for matching. The --locale=xxxx long option has |
| 28 |
|
been added (no short equivalent) to specify a locale explicitly on the |
| 29 |
|
pcregrep command, overriding the environment variables. |
| 30 |
|
|
| 31 |
|
(d) When -B was used with -n, some line numbers in the output were one less |
| 32 |
|
than they should have been. |
| 33 |
|
|
| 34 |
|
(e) Added the -o (--only-matching) option. |
| 35 |
|
|
| 36 |
|
(f) If -A or -C was used with -c (count only), some lines of context were |
| 37 |
|
accidentally printed for the final match. |
| 38 |
|
|
| 39 |
|
(g) Added the -H (--with-filename) option. |
| 40 |
|
|
| 41 |
|
(h) The combination of options -rh failed to suppress file names for files |
| 42 |
|
that were found from directory arguments. |
| 43 |
|
|
| 44 |
|
(i) Added the -D (--devices) and -d (--directories) options. |
| 45 |
|
|
| 46 |
|
(j) Added the -F (--fixed-strings) option. |
| 47 |
|
|
| 48 |
|
(k) Allow "-" to be used as a file name for -f as well as for a data file. |
| 49 |
|
|
| 50 |
|
(l) Added the --colo(u)r option. |
| 51 |
|
|
| 52 |
|
(m) Added Jeffrey Friedl's -S testing option, but within #ifdefs so that it |
| 53 |
|
is not present by default. |
| 54 |
|
|
| 55 |
|
3. A nasty bug was discovered in the handling of recursive patterns, that is, |
| 56 |
|
items such as (?R) or (?1), when the recursion could match a number of |
| 57 |
|
alternatives. If it matched one of the alternatives, but subsequently, |
| 58 |
|
outside the recursion, there was a failure, the code tried to back up into |
| 59 |
|
the recursion. However, because of the way PCRE is implemented, this is not |
| 60 |
|
possible, and the result was an incorrect result from the match. |
| 61 |
|
|
| 62 |
|
In order to prevent this happening, the specification of recursion has |
| 63 |
|
been changed so that all such subpatterns are automatically treated as |
| 64 |
|
atomic groups. Thus, for example, (?R) is treated as if it were (?>(?R)). |
| 65 |
|
|
| 66 |
|
4. I had overlooked the fact that, in some locales, there are characters for |
| 67 |
|
which isalpha() is true but neither isupper() nor islower() are true. In |
| 68 |
|
the fr_FR locale, for instance, the \xAA and \xBA characters (ordmasculine |
| 69 |
|
and ordfeminine) are like this. This affected the treatment of \w and \W |
| 70 |
|
when they appeared in character classes, but not when they appeared outside |
| 71 |
|
a character class. The bit map for "word" characters is now created |
| 72 |
|
separately from the results of isalnum() instead of just taking it from the |
| 73 |
|
upper, lower, and digit maps. (Plus the underscore character, of course.) |
| 74 |
|
|
| 75 |
|
5. The above bug also affected the handling of POSIX character classes such as |
| 76 |
|
[[:alpha:]] and [[:alnum:]]. These do not have their own bit maps in PCRE's |
| 77 |
|
permanent tables. Instead, the bit maps for such a class were previously |
| 78 |
|
created as the appropriate unions of the upper, lower, and digit bitmaps. |
| 79 |
|
Now they are created by subtraction from the [[:word:]] class, which has |
| 80 |
|
its own bitmap. |
| 81 |
|
|
| 82 |
|
6. The [[:blank:]] character class matches horizontal, but not vertical space. |
| 83 |
|
It is created by subtracting the vertical space characters (\x09, \x0a, |
| 84 |
|
\x0b, \x0c) from the [[:space:]] bitmap. Previously, however, the |
| 85 |
|
subtraction was done in the overall bitmap for a character class, meaning |
| 86 |
|
that a class such as [\x0c[:blank:]] was incorrect because \x0c would not |
| 87 |
|
be recognized. This bug has been fixed. |
| 88 |
|
|
| 89 |
|
7. Patches from the folks at Google: |
| 90 |
|
|
| 91 |
|
(a) pcrecpp.cc: "to handle a corner case that may or may not happen in |
| 92 |
|
real life, but is still worth protecting against". |
| 93 |
|
|
| 94 |
|
(b) pcrecpp.cc: "corrects a bug when negative radixes are used with |
| 95 |
|
regular expressions". |
| 96 |
|
|
| 97 |
|
(c) pcre_scanner.cc: avoid use of std::count() because not all systems |
| 98 |
|
have it. |
| 99 |
|
|
| 100 |
|
(d) Split off pcrecpparg.h from pcrecpp.h and had the former built by |
| 101 |
|
"configure" and the latter not, in order to fix a problem somebody had |
| 102 |
|
with compiling the Arg class on HP-UX. |
| 103 |
|
|
| 104 |
|
(e) Improve the error-handling of the C++ wrapper a little bit. |
| 105 |
|
|
| 106 |
|
(f) New tests for checking recursion limiting. |
| 107 |
|
|
| 108 |
|
8. The pcre_memmove() function, which is used only if the environment does not |
| 109 |
|
have a standard memmove() function (and is therefore rarely compiled), |
| 110 |
|
contained two bugs: (a) use of int instead of size_t, and (b) it was not |
| 111 |
|
returning a result (though PCRE never actually uses the result). |
| 112 |
|
|
| 113 |
|
9. In the POSIX regexec() interface, if nmatch is specified as a ridiculously |
| 114 |
|
large number - greater than INT_MAX/(3*sizeof(int)) - REG_ESPACE is |
| 115 |
|
returned instead of calling malloc() with an overflowing number that would |
| 116 |
|
most likely cause subsequent chaos. |
| 117 |
|
|
| 118 |
|
10. The debugging option of pcretest was not showing the NO_AUTO_CAPTURE flag. |
| 119 |
|
|
| 120 |
|
11. The POSIX flag REG_NOSUB is now supported. When a pattern that was compiled |
| 121 |
|
with this option is matched, the nmatch and pmatch options of regexec() are |
| 122 |
|
ignored. |
| 123 |
|
|
| 124 |
|
12. Added REG_UTF8 to the POSIX interface. This is not defined by POSIX, but is |
| 125 |
|
provided in case anyone wants to the the POSIX interface with UTF-8 |
| 126 |
|
strings. |
| 127 |
|
|
| 128 |
|
13. Added CXXLDFLAGS to the Makefile parameters to provide settings only on the |
| 129 |
|
C++ linking (needed for some HP-UX environments). |
| 130 |
|
|
| 131 |
|
14. Avoid compiler warnings in get_ucpname() when compiled without UCP support |
| 132 |
|
(unused parameter) and in the pcre_printint() function (omitted "default" |
| 133 |
|
switch label when the default is to do nothing). |
| 134 |
|
|
| 135 |
|
15. Added some code to make it possible, when PCRE is compiled as a C++ |
| 136 |
|
library, to replace subject pointers for pcre_exec() with a smart pointer |
| 137 |
|
class, thus making it possible to process discontinuous strings. |
| 138 |
|
|
| 139 |
|
16. The two macros PCRE_EXPORT and PCRE_DATA_SCOPE are confusing, and perform |
| 140 |
|
much the same function. They were added by different people who were trying |
| 141 |
|
to make PCRE easy to compile on non-Unix systems. It has been suggested |
| 142 |
|
that PCRE_EXPORT be abolished now that there is more automatic apparatus |
| 143 |
|
for compiling on Windows systems. I have therefore replaced it with |
| 144 |
|
PCRE_DATA_SCOPE. This is set automatically for Windows; if not set it |
| 145 |
|
defaults to "extern" for C or "extern C" for C++, which works fine on |
| 146 |
|
Unix-like systems. It is now possible to override the value of PCRE_DATA_ |
| 147 |
|
SCOPE with something explicit in config.h. In addition: |
| 148 |
|
|
| 149 |
|
(a) pcreposix.h still had just "extern" instead of either of these macros; |
| 150 |
|
I have replaced it with PCRE_DATA_SCOPE. |
| 151 |
|
|
| 152 |
|
(b) Functions such as _pcre_xclass(), which are internal to the library, |
| 153 |
|
but external in the C sense, all had PCRE_EXPORT in their definitions. |
| 154 |
|
This is apparently wrong for the Windows case, so I have removed it. |
| 155 |
|
(It makes no difference on Unix-like systems.) |
| 156 |
|
|
| 157 |
|
17. Added a new limit, MATCH_LIMIT_RECURSION, which limits the depth of nesting |
| 158 |
|
of recursive calls to match(). This is different to MATCH_LIMIT because |
| 159 |
|
that limits the total number of calls to match(), not all of which increase |
| 160 |
|
the depth of recursion. Limiting the recursion depth limits the amount of |
| 161 |
|
stack (or heap if NO_RECURSE is set) that is used. The default can be set |
| 162 |
|
when PCRE is compiled, and changed at run time. A patch from Google adds |
| 163 |
|
this functionality to the C++ interface. |
| 164 |
|
|
| 165 |
|
18. Changes to the handling of Unicode character properties: |
| 166 |
|
|
| 167 |
|
(a) Updated the table to Unicode 4.1.0. |
| 168 |
|
|
| 169 |
|
(b) Recognize characters that are not in the table as "Cn" (undefined). |
| 170 |
|
|
| 171 |
|
(c) I revised the way the table is implemented to a much improved format |
| 172 |
|
which includes recognition of ranges. It now supports the ranges that |
| 173 |
|
are defined in UnicodeData.txt, and it also amalgamates other |
| 174 |
|
characters into ranges. This has reduced the number of entries in the |
| 175 |
|
table from around 16,000 to around 3,000, thus reducing its size |
| 176 |
|
considerably. I realized I did not need to use a tree structure after |
| 177 |
|
all - a binary chop search is just as efficient. Having reduced the |
| 178 |
|
number of entries, I extended their size from 6 bytes to 8 bytes to |
| 179 |
|
allow for more data. |
| 180 |
|
|
| 181 |
|
(d) Added support for Unicode script names via properties such as \p{Han}. |
| 182 |
|
|
| 183 |
|
19. In UTF-8 mode, a backslash followed by a non-Ascii character was not |
| 184 |
|
matching that character. |
| 185 |
|
|
| 186 |
|
20. When matching a repeated Unicode property with a minimum greater than zero, |
| 187 |
|
(for example \pL{2,}), PCRE could look past the end of the subject if it |
| 188 |
|
reached it while seeking the minimum number of characters. This could |
| 189 |
|
happen only if some of the characters were more than one byte long, because |
| 190 |
|
there is a check for at least the minimum number of bytes. |
| 191 |
|
|
| 192 |
|
21. Refactored the implementation of \p and \P so as to be more general, to |
| 193 |
|
allow for more different types of property in future. This has changed the |
| 194 |
|
compiled form incompatibly. Anybody with saved compiled patterns that use |
| 195 |
|
\p or \P will have to recompile them. |
| 196 |
|
|
| 197 |
|
22. Added "Any" and "L&" to the supported property types. |
| 198 |
|
|
| 199 |
|
23. Recognize \x{...} as a code point specifier, even when not in UTF-8 mode, |
| 200 |
|
but give a compile time error if the value is greater than 0xff. |
| 201 |
|
|
| 202 |
|
24. The man pages for pcrepartial, pcreprecompile, and pcre_compile2 were |
| 203 |
|
accidentally not being installed or uninstalled. |
| 204 |
|
|
| 205 |
|
25. The pcre.h file was built from pcre.h.in, but the only changes that were |
| 206 |
|
made were to insert the current release number. This seemed silly, because |
| 207 |
|
it made things harder for people building PCRE on systems that don't run |
| 208 |
|
"configure". I have turned pcre.h into a distributed file, no longer built |
| 209 |
|
by "configure", with the version identification directly included. There is |
| 210 |
|
no longer a pcre.h.in file. |
| 211 |
|
|
| 212 |
|
However, this change necessitated a change to the pcre-config script as |
| 213 |
|
well. It is built from pcre-config.in, and one of the substitutions was the |
| 214 |
|
release number. I have updated configure.ac so that ./configure now finds |
| 215 |
|
the release number by grepping pcre.h. |
| 216 |
|
|
| 217 |
|
26. Added the ability to run the tests under valgrind. |
| 218 |
|
|
| 219 |
|
|
| 220 |
Version 6.4 05-Sep-05 |
Version 6.4 05-Sep-05 |
| 221 |
--------------------- |
--------------------- |
| 222 |
|
|
Parent Directory
| 
Revision Log
| 
Patch