/[pcre]/code/trunk/doc/pcre.txt

Diff of /code/trunk/doc/pcre.txt

Parent Directory | Revision Log | View Patch Patch

-revision 469 by ph10,
Mon Oct 19 14:38:48 2009 UTC
+revision 487 by ph10,
Wed Jan  6 10:26:55 2010 UTC
 Line 6958 
 PCRE DISCUSSION OF STACK USAGE
         restarted instead.
         The pcre_dfa_exec() function operates in an entirely different way, and
-        hardly uses recursion at all. The limit on its complexity is the amount
+        uses recursion only when there is a  regular  expression  recursion  or
-        of  workspace  it  is  given.  The comments that follow do NOT apply to
+        subroutine  call in the pattern. This includes the processing of asser-
-        pcre_dfa_exec(); they are relevant only for pcre_exec().
+        tion and "once-only" subpatterns, which  are  handled  like  subroutine
+        calls.  Normally,  these are never very deep, and the limit on the com-
-        You can set limits on the number of times that match() is called,  both
+        plexity of pcre_dfa_exec() is controlled by the amount of workspace  it
-        in  total  and  recursively. If the limit is exceeded, an error occurs.
+        is  given. However, it is possible to write patterns with runaway infi-
-        For details, see the section on  extra  data  for  pcre_exec()  in  the
+        nite recursions; such patterns will cause pcre_dfa_exec() to run out of
-        pcreapi documentation.
+        stack. At present, there is no protection against this.
-        Each  time  that match() is actually called recursively, it uses memory
+        The comments that follow do NOT apply to pcre_dfa_exec(); they are rel-
-        from the process stack. For certain kinds of  pattern  and  data,  very
+        evant only for pcre_exec().
-        large  amounts of stack may be needed, despite the recognition of "tail
-        recursion".  You can often reduce the amount of recursion,  and  there-
+    Reducing pcre_exec()'s stack usage
-        fore  the  amount of stack used, by modifying the pattern that is being
+        Each time that match() is actually called recursively, it  uses  memory
+        from  the  process  stack.  For certain kinds of pattern and data, very
+        large amounts of stack may be needed, despite the recognition of  "tail
+        recursion".   You  can often reduce the amount of recursion, and there-
+        fore the amount of stack used, by modifying the pattern that  is  being
         matched. Consider, for example, this pattern:
           ([^<]|<(?!inet))+
-        It matches from wherever it starts until it encounters "<inet"  or  the
+        It  matches  from wherever it starts until it encounters "<inet" or the
-        end  of  the  data,  and is the kind of pattern that might be used when
+        end of the data, and is the kind of pattern that  might  be  used  when
         processing an XML file. Each iteration of the outer parentheses matches
-        either  one  character that is not "<" or a "<" that is not followed by
+        either one character that is not "<" or a "<" that is not  followed  by
-        "inet". However, each time a  parenthesis  is  processed,  a  recursion
+        "inet".  However,  each  time  a  parenthesis is processed, a recursion
         occurs, so this formulation uses a stack frame for each matched charac-
-        ter. For a long string, a lot of stack is required. Consider  now  this
+        ter.  For  a long string, a lot of stack is required. Consider now this
         rewritten pattern, which matches exactly the same strings:
           ([^<]++|<(?!inet))+
-        This  uses very much less stack, because runs of characters that do not
+        This uses very much less stack, because runs of characters that do  not
-        contain "<" are "swallowed" in one item inside the parentheses.  Recur-
+        contain  "<" are "swallowed" in one item inside the parentheses. Recur-
-        sion  happens  only when a "<" character that is not followed by "inet"
+        sion happens only when a "<" character that is not followed  by  "inet"
-        is encountered (and we assume this is relatively  rare).  A  possessive
+        is  encountered  (and  we assume this is relatively rare). A possessive
-        quantifier  is  used  to stop any backtracking into the runs of non-"<"
+        quantifier is used to stop any backtracking into the  runs  of  non-"<"
         characters, but that is not related to stack usage.
-        This example shows that one way of avoiding stack problems when  match-
+        This  example shows that one way of avoiding stack problems when match-
         ing long subject strings is to write repeated parenthesized subpatterns
         to match more than one character whenever possible.
-    Compiling PCRE to use heap instead of stack
+    Compiling PCRE to use heap instead of stack for pcre_exec()
+        In  environments  where  stack memory is constrained, you might want to
+        compile PCRE to use heap memory instead of stack for remembering  back-
+        up  points  when  pcre_exec()  is running. This makes it run a lot more
+        slowly, however.  Details of how to do this are given in the  pcrebuild
+        documentation. When built in this way, instead of using the stack, PCRE
+        obtains and frees memory by calling the functions that are  pointed  to
+        by  the  pcre_stack_malloc  and  pcre_stack_free variables. By default,
+        these point to malloc() and free(), but you can replace the pointers to
+        cause  PCRE to use your own functions. Since the block sizes are always
+        the same, and are always freed in reverse order, it may be possible  to
+        implement  customized  memory handlers that are more efficient than the
+        standard functions.
-        In environments where stack memory is constrained, you  might  want  to
+    Limiting pcre_exec()'s stack usage
-        compile  PCRE to use heap memory instead of stack for remembering back-
-        up points. This makes it run a lot more slowly, however. Details of how
+        You can set limits on the number of times that match() is called,  both
-        to do this are given in the pcrebuild documentation. When built in this
+        in  total  and recursively. If a limit is exceeded, pcre_exec() returns
-        way, instead of using the stack, PCRE obtains and frees memory by call-
+        an error code. Setting suitable limits should prevent it  from  running
-        ing  the  functions  that  are  pointed to by the pcre_stack_malloc and
+        out  of  stack.  The  default  values of the limits are very large, and
-        pcre_stack_free variables. By default,  these  point  to  malloc()  and
+        unlikely ever to operate. They can be changed when PCRE is  built,  and
-        free(),  but you can replace the pointers to cause PCRE to use your own
+        they  can  also be set when pcre_exec() is called. For details of these
-        functions. Since the block sizes are always the same,  and  are  always
+        interfaces, see the pcrebuild documentation and the  section  on  extra
-        freed in reverse order, it may be possible to implement customized mem-
+        data for pcre_exec() in the pcreapi documentation.
-        ory handlers that are more efficient than the standard functions.
-    Limiting PCRE's stack usage
-        PCRE has an internal counter that can be used to  limit  the  depth  of
-        recursion,  and  thus cause pcre_exec() to give an error code before it
-        runs out of stack. By default, the limit is very  large,  and  unlikely
-        ever  to operate. It can be changed when PCRE is built, and it can also
-        be set when pcre_exec() is called. For details of these interfaces, see
-        the pcrebuild and pcreapi documentation.
         As a very rough rule of thumb, you should reckon on about 500 bytes per
         recursion. Thus, if you want to limit your  stack  usage  to  8Mb,  you
         should  set  the  limit at 16000 recursions. A 64Mb stack, on the other
-        hand, can support around 128000 recursions. The pcretest  test  program
+        hand, can support around 128000 recursions.
-        has a command line option (-S) that can be used to increase the size of
-        its stack.
+        In Unix-like environments, the pcretest test program has a command line
+        option (-S) that can be used to increase the size of its stack. As long
+        as the stack is large enough, another option (-M) can be used  to  find
+        the  smallest  limits  that allow a particular pattern to match a given
+        subject string. This is done by  calling  pcre_exec()  repeatedly  with
+        different limits.
     Changing stack size in Unix-like systems
-        In Unix-like environments, there is not often a problem with the  stack
+        In  Unix-like environments, there is not often a problem with the stack
-        unless  very  long  strings  are  involved, though the default limit on
+        unless very long strings are involved,  though  the  default  limit  on
-        stack size varies from system to system. Values from 8Mb  to  64Mb  are
+        stack  size  varies  from system to system. Values from 8Mb to 64Mb are
         common. You can find your default limit by running the command:
           ulimit -s
-        Unfortunately,  the  effect  of  running out of stack is often SIGSEGV,
+        Unfortunately, the effect of running out of  stack  is  often  SIGSEGV,
-        though sometimes a more explicit error message is given. You  can  nor-
+        though  sometimes  a more explicit error message is given. You can nor-
         mally increase the limit on stack size by code such as this:
           struct rlimit rlim;
-Line 7046 
 PCRE DISCUSSION OF STACK USAGE
+Line 7059 
 PCRE DISCUSSION OF STACK USAGE
           rlim.rlim_cur = 100*1024*1024;
           setrlimit(RLIMIT_STACK, &rlim);
-        This  reads  the current limits (soft and hard) using getrlimit(), then
+        This reads the current limits (soft and hard) using  getrlimit(),  then
-        attempts to increase the soft limit to  100Mb  using  setrlimit().  You
+        attempts  to  increase  the  soft limit to 100Mb using setrlimit(). You
         must do this before calling pcre_exec().
     Changing stack size in Mac OS X
         Using setrlimit(), as described above, should also work on Mac OS X. It
         is also possible to set a stack size when linking a program. There is a
-        discussion   about   stack  sizes  in  Mac  OS  X  at  this  web  site:
+        discussion  about  stack  sizes  in  Mac  OS  X  at  this   web   site:
         http://developer.apple.com/qa/qa2005/qa1419.html.
-Line 7067 
 AUTHOR
+Line 7080 
 AUTHOR
  REVISION
-        Last updated: 09 July 2008
+        Last updated: 03 January 2010
-        Copyright (c) 1997-2008 University of Cambridge.
+        Copyright (c) 1997-2010 University of Cambridge.
  ------------------------------------------------------------------------------

 Legend:



Removed from v.469
 


changed lines


 
Added in v.487
 Legend:



Removed from v.469
 


changed lines


 
Added in v.487
-Removed from v.469
+Added in v.487

webmaster@exim.org	ViewVC Help
Powered by ViewVC 1.1.12