• DanielDiniz

Hi :)

After downloading, compiling and installing the libraries, add this snippet to your Cython source:

cdef extern from "google/profiler.h":
    cdef int ProfilerStart(char* fname)
    cdef void ProfilerStop()


def start():
    ProfilerStart('parse.prof')

def stop():
    ProfilerStop()

Add the path to profiler.h to gcc's include path and link against libprofiler, as in this example:

gcc  "..." -fPIC -I/usr/include/python2.5 -I/usr/local/include/google -c parse.c -o build/parse.o

gcc  "..." -lprofiler  build/parse.o -o parse.so

A sample ipython session shows how easy it is to gather profile data:

In [1]: import parse

In [2]: parse.start()

In [3]: %timeit j = parse.And([parse.Keyword('a'), parse.Literal('b')]).searchString('a b' * 900)
10 loops, best of 3: 69.2 ms per loop

In [4]: parse.stop()
PROFILE: interrupts/evictions/bytes = 231/0/38816

Here's the kind of information that you can get:

~/code/cy$ pprof parse.so parse.prof --text |head
Total: 231 samples
      14   6.1%   6.1%       14   6.1% strlen
      10   4.3%  10.4%       14   6.1% vfprintf
       8   3.5%  13.9%       84  36.4% __Pyx_AddTraceback
       7   3.0%  16.9%      168  72.7% __pyx_f_5parse_13ParserElement__parse
       7   3.0%  19.9%        7   3.0% malloc_set_state
       6   2.6%  22.5%        6   2.6% 0x0809174a
       4   1.7%  24.2%       44  19.0% __pyx_pf_5parse_13ParserElement___getattr__
       4   1.7%  26.0%        4   1.7% memcpy
       4   1.7%  27.7%        4   1.7% _IO_default_xsputn

So pyx_f_5parse_13ParserElementparse is pretty busy. Here's a snippet (minus error checking):

~/code/cy$ pprof --line --list=__pyx_f_5parse_13ParserElement__parse parse.so parse.prof

ROUTINE ====================== __pyx_f_5parse_13ParserElement__parse in code/cy/parse.c
samples    426 Total 15155 (flat / cumulative)
"..."
 7    7 11343: static  PyObject
               *__pyx_f_5parse_13ParserElement__parse(
 .    .     struct __pyx_obj_5parse_ParserElement *__pyx_v_self,
 .    .     PyObject *__pyx_v_instring, int __pyx_v_loc, int __pyx_v_doActions,
 .    .     int __pyx_v_callPreParse, int __pyx_skip_dispatch) {
"..."
 .    . 11419:   /* "code/cy/parse.pyx":744
 .    . 11420:  *   instrlen = len(instring)
 .    . 11421:  *   debugging = ( self.debug ) #and doActions )
 .    . 11422:  *   if debugging or self.failAction:     # <<<<<<<<<<<<<<
 .    . 11425:  */
 .    . 11426:   __pyx_8 = __Pyx_PyObject_IsTrue(__pyx_v_debugging);
 .    . 11427:   __pyx_7 = __pyx_8;
 4    4 11428:   if (!__pyx_7) {
 7  107 11429: __pyx_4 = PyObject_GetAttr(((PyObject *)__pyx_v_self), __pyx_kp_failAction);
 8    8 11430:     __pyx_7 = __Pyx_PyObject_IsTrue(__pyx_4);
 7    7 11431:     Py_DECREF(__pyx_4); __pyx_4 = 0;
 .    . 11432:   }

It looks like the sampling (done 100 times per second by default) misses some lines.

CategoryHomepage