Using the Cython Compiler to write fast Python code

Dr. Stefan Behnel

http://cython.org/

cython-dev@codespeak.net

cython-users@googlegroups.com

About myself

Part 1: Intro to Cython

What is Cython?

Cython is the missing link
between the simplicity of Python
and the speed of C / C++ / Fortran.
SimplicityVsSpeed1.png

What is Cython?

Cython is the missing link
between the simplicity of Python
and the speed of C / C++ / Fortran.
SimplicityVsSpeed.png

What is Cython?

Cython is

Major Cython Core Developers

How to use Cython

Example: compiling Python code

# file: worker.py

class HardWorker(object):
    u"Almost Sisyphos"

    def __init__(self, task):
        self.task = task

    def work_hard(self, repeat=100):
        for i in range(repeat):
            self.task()


def add_simple_stuff():
    x = 1+1

HardWorker(add_simple_stuff).work_hard()

Example: compiling Python code

Portable Code

... the fastest way to port Python 2 code to Py3 ;-)

Python language feature support

Python features in work

Planned Cython features

... as usual: great ideas, little time

Currently unsupported

... all certainly on the TODO list for 1.0.

Speed

Cython generates very efficient C code:

Type declarations

Cython supports optional type declarations that

Part 2: Building Cython modules

Ways to build Cython code

To compile Python code (.py) or Cython code (.pyx)

Example: distutils

Example: pyximport

Build and import Cython code files (.pyx) on the fly

$ ls
worker.pyx
$ PYTHONPATH=. python

Python 2.6.2 (r262:71600, Apr 17 2009, 11:29:30)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyximport
>>> pyximport.install()

>>> import worker
>>> worker
<module 'worker' from '~/.pyxbld/.../worker.so'>
>>> worker.HardWorker
<class 'worker.HardWorker'>
>>> worker.HardWorker(worker.add_simple_stuff).work_hard()

pyximporting Python modules

>>> import pyximport
>>> pyximport.install(pyimport = True)
>>> import shlex
[lots of compiler errors from different modules ...]
>>> help(shlex)

Writing executable programs

# file: hw.py

def hello_world():
    import sys
    print "Welcome to Python %d.%d!" % sys.version_info[:2]

if __name__ == '__main__':
    hello_world()

Writing executable programs

# file: hw.py

def hello_world():
    import sys
    print "Welcome to Python %d.%d!" % sys.version_info[:2]

if __name__ == '__main__':
    hello_world()

Compile, link and run:

$ cython --embed hw.py   # <- embed a main() function

$ gcc $CFLAGS -I/usr/include/python2.6 \
   -o hw hw.c -lpython2.6 -lpthread -lm -lutil -ldl

$ ./hw
Welcome to Python 2.6!

Part 3: Writing fast code

A simple example

# integrate_py.py

from math import sin

def f(x):
    return sin(x**2)

def integrate_f(a, b, N):
    dx = (b-a)/N
    s = 0
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

Type declarations in Cython

Function arguments are easy

Type declarations in Cython

»cdef« keyword declares

Functions: def vs. cdef vs. cpdef

Typed arguments and return values

A simple example: Python

# integrate_py.py

from math import sin

def f(x):
    return sin(x**2)

def integrate_f(a, b, N):
    dx = (b-a)/N
    s = 0
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

A simple example: Cython

# integrate_cy.pyx

cdef extern from "math.h":
    double sin(double x)

cdef double f(double x):
    return sin(x**2)

cpdef double integrate_f(double a, double b, int N):
    cdef double dx, s
    cdef int i

    dx = (b-a)/N
    s = 0
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

Overriding declarations in .pxd

# integrate_py.py

from math import sin

def f(x):
    return sin(x**2)

def integrate_f(a, b, N):
    dx = (b-a)/N
    s = 0
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

Overriding declarations in .pxd

Python integrate_py.py Cython integrate_py.pxd 
# integrate_py.py

from math import sin

def f(x):
    return sin(x**2)


def integrate_f(a, b, N):

    dx = (b-a)/N
    s = 0
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

# integrate_py.pxd
cimport cython


cpdef double f(double x)

@cython.locals(
    dx=double, s=double, i=int)
cpdef integrate_f(
    double a, double b, int N)

The .pxd file used

# integrate_py.pxd

cimport cython

cpdef double f(double x):
    return sin(x**2)

cpdef double integrate_f(double a, double b, int N)

Overriding declarations in .pxd

Typing in Python syntax

# integrate_py.py

from math import sin

def f(x):
    return sin(x**2)

def integrate_f(a, b, N):
    dx = (b-a)/N
    s = 0
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

Typing in Python syntax

from math import sin
import cython

@cython.locals(x=cython.double)
def f(x):
    return sin(x**2)

@cython.locals(a=cython.double, b=cython.double,
               N=cython.Py_ssize_t, dx=cython.double,
               s=cython.double, i=cython.Py_ssize_t)
def integrate_f(a, b, N):
    dx = (b-a)/N
    s = 0
    for i in range(N):
        s += f(a+i*dx)
    return s * dx

Declaring Python types

Declaring Python types: dict

def filter_a(d):
    return { key : value
             for key, value in d.iteritems()
             if 'a' not in value }

import string
d = { s:s for s in string.ascii_letters }
print filter_a(d)

Declaring Python types: dict

def filter_a(dict d):       # <====
    return { key : value
             for key, value in d.iteritems()
             if 'a' not in value }

import string
d = { s:s for s in string.ascii_letters }
print filter_a(d)

Declaring Python types: dict

def filter_a(dict d):       # <====
    return { key : value
             for key, value in d.iteritems()
             if 'a' not in value }

import string
d = { s:s for s in string.ascii_letters }
print filter_a(d)

Think twice before you type

SimplicityVsSpeed.png

Classes

cdef classes - when to use them?

Part 4: Talking to other extensions

Talking to other extensions

Python 3 buffer protocol

def inplace_invert_2D_buffer(
                object[unsigned char, 2] image):

    cdef int i, j
    for i in range(image.shape[0]):
        for j in range(image.shape[1]):
            image[i, j] = 255 - image[i, j]

Conclusion

... but Cython is also

Cython

Cython

C-Extensions in Python

... use it, and join the project!

http://cython.org/