17 SWIG and Python

Caution: This chapter is under repair!

This chapter describes SWIG's support of Python. SWIG is compatible with most recent Python versions including Python 2.2 as well as older versions dating back to Python 1.5. The original release of SWIG was developed for Python 1.3 so current versions may still work with that release. However, this hasn't been tested for quite some time and your mileage might vary. For the best results, consider using Python 2.0 or newer.

This chapter covers most SWIG features, but general features are cover in less depth than is found in earlier chapters. At the very least, make sure you read the "SWIG Basics" chapter.

17.1 Overview

To build Python extension modules, SWIG uses a layered approach. At the lowest level, simple procedural wrappers are generated for functions, classes, methods, and other declarations in the input file. Then, for structures and classes, an optional collection of Python proxy classes are generated in order to provide a more natural Python interface to the objects in your program. These proxy classes simply build upon the low-level interface.

This layered approach to wrapping has a number of benefits. First, the low-level procedural interface is relatively simple and can be packaged into an extension module that works with virtually every version of Python. In addition, the use of proxy classes means that C/C++ objects are wrapped by real Python classes which are free from the limitations that arise when classes are wrapped into new built-in types (a common approach to object wrapping prior to Python 2.2).

In describing Python interface, this chapter starts by covering the most critical topics. First, the problem of configuration, compiling, and installing Python modules is discussed. Next, the low-level procedural interface is presented. Finally, proxy classes are described. Advanced customization features, typemaps, and other options are found near the end of the chapter.

17.2 Preliminaries

To build a Python module, run SWIG using the -python option :

$ swig -python example.i
If building a C++ extension, add the -c++ option:

$ swig -c++ -python example.i

This creates a file example_wrap.c or example_wrap.cxx that contains all of the code needed to build a Python extension module. To finish building the module, you need to compile this file and link it with the rest of your program.

17.2.1 Getting the right header files

In order to compile the wrapper code, the compiler needs the Python.h header file. This file is usually contained in a directory such as

/usr/local/include/python2.0
The exact location may vary on your machine, but the above location is typical. If you are not entirely sure where Python is installed, you can run Python to find out. For example:
$ python
Python 2.1.1 (#1, Jul 23 2001, 14:36:06)
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.prefix
/usr/local
>>>           

17.2.2 Compiling a dynamic module

The preferred approach to building an extension module is to compile it into a shared object file or DLL. To do this, you will need to compile your program using comands like this (shown for Linux):

$ swig -python example.i
$ gcc -c example.c
$ gcc -c example_wrap.c -I/usr/local/include/python2.0
$ gcc -shared example.o example_wrap.o -o examplemodule.so
The exact commands for doing this vary from platform to platform. SWIG tries to guess the right options when it is installed. Therefore, you may want to start with one of the examples in the SWIG/Examples/python directory. If that doesn't work, you will need to read the man-pages for your compiler and linker to get the right set of options. You might also check the SWIG Wiki for additional information.

When linking the module, the name of the output file has to match the name of the module. If the name of your SWIG module is "example", the name of the corresponding object file should be "examplemodule.so" or "example.so". The name of the module is specified using the %module directive or the -module command line option.

17.2.3 Using distutils

17.2.4 Static linking

An alternative approach to dynamic linking is to rebuild the Python interpreter with your extension module added to it. In the past, this approach was sometimes necesssary due to limitations in dynamic loading support on certain machines. However, the situation has improved greatly over the last few years and you should not consider this approach unless there is really no other option.

The usual procedure for adding a new module to Python involves finding the Python source, adding an entry to the Modules/Setup file, and rebuilding the interpreter using the Python Makefile. However, newer Python versions have changed the build process. You may need to edit the 'setup.py' file in the Python distribution instead.

In earlier versions of SWIG, the embed.i library file could be used to rebuild the interpreter. For example:

%module example

extern int fact(int);
extern int mod(int, int);
extern double My_variable;

%include embed.i       // Include code for a static version of Python

The embed.i library file includes supporting code that contains everything needed to rebuild Python. To rebuild the interpreter, you simply do something like this:

$ swig -python example.i
$ gcc example.c example_wrap.c \
        -Xlinker -export-dynamic \
        -DHAVE_CONFIG_H -I/usr/local/include/python2.1 \
	-I/usr/local/lib/python2.1/config \
	-L/usr/local/lib/python2.1/config -lpython2.1 -lm -ldl \
	-o mypython

You will need to supply the same libraries that were used to build Python the first time. This may include system libraries such as -lsocket, -lnsl, and -lpthread. Assuming this actually works, the new version of Python should be identical to the default version except that your extension module will be a built-in part of the interpreter.

Comment: In practice, you should probably try to avoid static linking if possible. Some programmers may be inclined to use static linking in the interest of getting better performance. However, the performance gained by static linking tends to be rather minimal in most situations (and quite frankly not worth the extra hassle in the opinion of this author).

Compatibility note: The embed.i library file is deprecated and has not been maintained for several years. Even though it appears to "work" with Python 2.1, no future support is guaranteed. If using static linking, you might want to rely on a different approach (perhaps using distutils).

17.2.5 Using your module

To use your module, simply use the Python import statement. If all goes well, you will be able to this:

$ python
>>> import example
>>> example.fact(4)
24
>>>
A common error received by first-time users is the following:
>>> import example
Traceback (most recent call last):
  File "", line 1, in ?
ImportError: dynamic module does not define init function (initexample)
>>>                                                               
This error is almost always caused when the name of the shared object file doesn't match the name of the module supplied using the SWIG %module directive. Double-check the interface to make sure the module name and the shared object file match. Another possible cause of this error is forgetting to link the SWIG-generated wrapper code with the rest of your application when creating the extension module.

Another common error is something similar to the following:

Traceback (most recent call last):
  File "example.py", line 3, in ?
    import example
ImportError: ./examplemodule.so: undefined symbol: fact
This error usually indicates that you forgot to include some object files or libraries in the linking of the shared library file. Make sure you compile both the SWIG wrapper file and your original program into a shared library file. Make sure you pass all of the required libraries to the linker.

Sometimes unresolved symbols occur because a wrapper has been created for a function that doesn't actually exist in a library. This usually occurs when a header file includes a declaration for a function that was never actually implemented or it was removed from a library without updating the header file. To fix this, you can either edit the SWIG input file to remove the offending declaration or you can use the %ignore directive to ignore the declaration.

Finally, suppose that your extension module is linked with another library like this:

$ gcc -shared example.o example_wrap.o -L/home/beazley/projects/lib -lfoo \
      -o examplemodule.so
If the foo library is compiled as a shared library, you might get the following problem when you try to use your module:
>>> import example
Traceback (most recent call last):
  File "", line 1, in ?
ImportError: libfoo.so: cannot open shared object file: No such file or directory
>>>                 
This error is generated because the dynamic linker can't locate the libfoo.so library. When shared libraries are loaded, the system normally only checks a few standard locations such as /usr/lib and /usr/local/lib. To fix this problem, there are several things you can do. First, you can recompile your extension module with extra path information. For example, on Linux you can do this:
$ gcc -shared example.o example_wrap.o -L/home/beazley/projects/lib -lfoo \
      -Xlinker -rpath /home/beazley/projects/lib \
      -o examplemodule.so
Alternatively, you can set the LD_LIBRARY_PATH environment variable to include the directory with your shared libraries. If setting LD_LIBRARY_PATH, be aware that setting this variable can introduce a noticeable performance impact on all other applications that you run. To set it only for Python, you might want to do this instead:
$ env LD_LIBRARY_PATH=/home/beazley/projects/lib python
Finally, you can use a command such as ldconfig to add additional search paths to the default system configuration (this requires root access and you will need to read the man pages).

17.2.6 Compilation of C++ extensions

Compilation of C++ extensions has traditionally been a tricky problem. Since the Python interpreter is written in C, you need to take steps to make sure C++ is properly initialized and that modules are compiled correctly.

On most machines, C++ extension modules should be linked using the C++ compiler. For example:

% swig -c++ -python example.i
% g++ -c example.cxx
% g++ -c example_wrap.cxx -I/usr/local/include/python2.0
% g++ -shared example.o example_wrap.o -o examplemodule.so
In addition to this, you may need to include additional library files to make it work. For example, if you are using the Sun C++ compiler on Solaris, you often need to add an extra library -lCrun like this:

% swig -c++ -python example.i
% CC -c example.cxx
% CC -c example_wrap.cxx -I/usr/local/include/python2.0
% CC -G example.o example_wrap.o -L/opt/SUNWspro/lib -o examplemodule.so -lCrun
Of course, the extra libraries to use are completely non-portable---you will probably need to do some experimentation.

Sometimes people have suggested that it is necessary to relink the Python interpreter using the C++ compiler to make C++ extension modules work. In the experience of this author, this has never actually appeared to be necessary. Relinking the interpreter with C++ really only includes the special run-time libraries described above---as long as you link your extension modules with these libraries, it should not be necessary to rebuild Python.

If you aren't entirely sure about the linking of a C++ extension, you might look at an existing C++ program. On many Unix machines, the ldd command will list library dependencies. This should give you some clues about what you might have to include when you link your extension module. For example:

$ ldd swig
        libstdc++-libc6.1-1.so.2 => /usr/lib/libstdc++-libc6.1-1.so.2 (0x40019000)
        libm.so.6 => /lib/libm.so.6 (0x4005b000)
        libc.so.6 => /lib/libc.so.6 (0x40077000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
$

As a final complication, a major weakness of C++ is that it does not define any sort of standard for binary linking of libraries. This means that C++ code compiled by different compilers will not link together properly as libraries nor is the memory layout of classes and data structures implemented in any kind of portable manner. In a monolithic C++ program, this problem may be unnoticed. However, in Python, it is possible for different extension modules to be compiled with different C++ compilers. As long as these modules are self-contained, this probably won't matter. However, if these modules start sharing data, you will need to take steps to avoid segmentation faults and other erratic program behavior. If working with lots of software components, you might want to investigate using a more formal standard such as COM.

17.2.7 Compiling for 64-bit platforms

On platforms that support 64-bit applications (Solaris, Irix, etc.), special care is required when building extension modules. On these machines, 64-bit applications are compiled and linked using a different set of compiler/linker options. In addition, it is not generally possible to mix 32-bit and 64-bit code together in the same application.

To utilize 64-bits, the Python executable will need to be recompiled as a 64-bit application. In addition, all libraries, wrapper code, and every other part of your application will need to be compiled for 64-bits. If you plan to use other third-party extension modules, they will also have to be recompiled as 64-bit extensions.

If you are wrapping commercial software for which you have no source code, you will be forced to use the same linking standard as used by that software. This may prevent the use of 64-bit extensions. It may also introduce problems on platforms that support more than one linking standard (e.g., -o32 and -n32 on Irix).

17.2.8 Building Python Extensions under Windows

Building a SWIG extension to Python under Windows is roughly similar to the process used with Unix. You will need to create a DLL that can be loaded into the interpreter. This section briefly describes the use of SWIG with Microsoft Visual C++. As a starting point, many of SWIG's examples include project files. You might want to take a quick look at these in addition to reading this section.

In Developer Studio, SWIG should be invoked as a custom build option. This is usually done as follows:

If all went well, SWIG will be automatically invoked whenever you build your project. Any changes made to the interface file will result in SWIG being automatically executed to produce a new version of the wrapper file.

To run your new Python extension, simply run Python and use the import command as normal. For example :

MSDOS > python
>>> import example
>>> print example.fact(4)
24
>>>
If you get an ImportError exception when importing the module, you may have forgotten to include aditional library files when you built your module. If you get an access violation or some kind of general protection fault immediately upon import, you have a more serious problem. This is often caused by linking your extension module against the wrong set of Win32 debug or thread libraries. You will have to fiddle around with the build options of project to try and track this down.

Some users have reported success in building extension modules using Cygwin and other compilers. However, the problem of building usable DLLs with these compilers tends to be rather problematic. For the latest information, you may want to consult the SWIG Wiki.

17.3 The low-level procedural interface

At its core, the Python module uses a simple low-level interface to C function, variables, constants, and classes. This low-level interface can be used to control your application. However, it is also used to construct more user-friendly proxy classes as described in the next section.

17.3.1 Modules

The SWIG %module directive specifies the name of the Python module. If you specify `%module example', then everything is wrapped into a Python 'example' module. When choosing a module name, make sure you don't use the same name as a built-in Python command or standard module name. Otherwise, the results may be unpredictable.

17.3.2 Functions

Global functions are wrapped as new Python built-in functions. For example,

%module example
int fact(int n);

creates a built-in function example.fact(n) that works like this:

>>> import example
>>> print example.fact(4)
24
>>>

17.3.3 Variable Linking

C/C++ global variables are fully supported by SWIG. However, the underlying mechanism is somewhat different than you might expect due to the way that Python assignment works. When you type the following in Python

a = 3.4

"a" becomes a name for an object containing the value 3.4. If you later type

b = a

then "a" and "b" are both names for the object containing the value 3.4. Thus, there is only one object containing 3.4 and "a" and "b" are both names that refer to it. This is quite different than C where a variable name refers to a memory location in which a value is stored (and assignment copies data into that location). Because of this, there is no direct way to map variable assignment in C to variable assignment in Python.

To provide access to C global variables, SWIG creates a special object called `cvar' that is added to each SWIG generated module. Global variables are then accessed as attributes of this object. For example, consider this interface

// SWIG interface file with global variables
%module example
...
extern int My_variable;
extern double density;
...

Now look at the Python interface:

>>> import example
>>> # Print out value of a C global variable
>>> print example.cvar.My_variable
4
>>> # Set the value of a C global variable
>>> example.cvar.density = 0.8442
>>> # Use in a math operation
>>> example.cvar.density = example.cvar.density*1.10

If you make an error in variable assignment, you will receive an error message. For example:

>>> example.cvar.density = "Hello"
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: C variable 'density (double )'
>>> 

If a variable is declared as const, it is wrapped as a read-only variable. Attempts to modify its value will result in an error.

To make ordinary variables read-only, you can also use the %immutable directive. For example:

%immutable;
extern char *path;
%mutable;
The %immutable directive stays in effect until it is explicitly disabled using %mutable.

If you would like to use a name other than "cvar", it can be changed using the -globals option :

% swig -python -globals myvar example.i

Finally, some care is in order when importing multiple SWIG modules. If you use the "from <file> import *" style of importing, you will get a name clash on the variable `cvar' and you will only be able to access global variables from the last module loaded. To prevent this, you might consider renaming cvar or making it private to the module by giving it a name that starts with a leading underscore. Also, SWIG does not create cvar if there are no global variables in a module.

17.3.4 Constants

C/C++ constants are installed as Python objects containing the appropriate value. To create a constant, use #define or the %constant directive. For example:
#define PI 3.14159
#define VERSION "1.0"

%constant int FOO = 42;
%constant const char *path = "/usr/local";
Note: C declarations declared as const are wrapped as read-only variables and will be accessed using the cvar object described in the previous section. They are not wrapped as constants.

Constants are not guaranteed to remain constant in Python---the name of the constant could be accidentally reassigned to refer to some other object. Unfortunately, there is no easy way for SWIG to generate code that prevents this. You will just have to be careful.

17.3.5 Pointers

Pointers to C/C++ objects are represented as encoded character strings such as the following :

_800f8e28_p_Vector

A NULL pointer is represented by the Python None object.

As an alternative to strings, SWIG can also encode pointers as a Python CObject type. CObjects are rarely discussed in most Python books or documentation. However, this is a special built-in type that can be used to hold raw C pointer values. Internally, a CObject is just a container that holds a raw void * along with some additional information such as a type-string.

If you want to use CObjects instead of strings, compile the SWIG wrapper code with the -DSWIG_COBJECT_TYPES option. For example:

% swig -python example.i
% gcc -c example.c
% gcc -c -DSWIG_COBJECT_TYPES example_wrap.c -I/usr/local/include/python2.0
% gcc -shared example.o example_wrap.o -o examplemodule.so
The choice of whether or not to use strings or CObjects is mostly a matter of personal preference. There is no significant performance difference between using one type or the other (strings actually appear to be ever-so-slightly faster on the author's machine). Although CObjects feel more natural to some programmers, a disadvantage of this approach is that it makes debugging more difficult. For example, if you are using CObjects, you will get code that works like this:
>>> import example
>>> a = example.new_Circle(10)
>>> b = example.new_Square(20)
>>> a
<PyCObject object at 0x80c5e60>
>>> b
<PyCObject object at 0x8107800>
>>> 
Notice how no clues regarding the type of a and b is shown. On the other hand, the string representation produces the following:
>>> a
'_88671008_p_Circle'
>>> b
'_605f0c08_p_Square'
>>> 
As much as you might be inclined to modify a pointer value directly from Python, don't. The hexadecimal encoding is not necessarily the same as the logical memory address of the underlying object. Instead it is the raw byte encoding of the pointer value. The encoding will vary depending on the native byte-ordering of the platform (i.e., big-endian vs. little-endian). Similarly, don't try to manually cast a pointer to a new type by simply replacing the type-string. This may not work like you expect, it is particularly dangerous when casting C++ objects, and it won't work if you switch to a new pointer representation such as CObjects. If you need to cast a pointer or change its value, consider writing some helper functions instead. For example:
%inline %{
/* C-style cast */
Bar *FooToBar(Foo *f) {
   return (Bar *) f;
}

/* C++-style cast */
Foo *BarToFoo(Bar *b) {
   return dynamic_cast<Foo*>(b);
}

Foo *IncrFoo(Foo *f, int i) {
    return f+i;
}
%}
Also, if working with C++, you should always try to use the new C++ style casts. For example, in the above code, the C-style cast may return a bogus result whereas as the C++-style cast will return None if the conversion can't be performed.

17.3.6 Structures

Access to the contents of a structure are provided through a set of low-level accessor functions as described in the "SWIG Basics" chapter. For example,

struct Vector {
	double x,y,z;
};

gets mapped into the following collection of accessor functions:

struct Vector *new_Vector();
void           delete_Vector(Vector *v);
double         Vector_x_get(Vector *obj)
void           Vector_x_set(Vector *obj, double x)
double         Vector_y_get(Vector *obj)
void           Vector_y_set(Vector *obj, double y)
double         Vector_z_get(Vector *obj)
void           Vector_z_set(Vector *obj, double z)

These functions are then used to access structure data from Python as follows:

>>> v = new_Vector()
>>> Vector_x_get(v)
3.5
>>> Vector_x_set(v,7.8)       # Change x component
>>> print Vector_x_get(v), Vector_y_get(v), Vector_z_get(v)
7.8 -4.5 0.0
>>> 

Similar access is provided for unions and the data members of C++ classes.

const members of a structure are read-only. Data members can also be forced to be read-only using the %immutable directive. For example:

struct Foo {
   ...
   %immutable;
   int x;        /* Read-only members */
   char *name;
   %mutable;
   ...
};

When char * members of a structure are wrapped, the contents are assumed to be dynamically allocated using malloc or new (depending on whether or not SWIG is run with the -c++ option). When the structure member is set, the old contents will be released and a new value created. If this is not the behavior you want, you will have to use a typemap (described shortly).

Array members are normally wrapped as read-only. For example,

struct Foo {
   int  x[50];
};
produces a single accessor function like this:
int *Foo_x_get(Foo *self) {
    return self->x;
};
If you want to set an array member, you will need to supply a "memberin" typemap described later in this chapter. As a special case, SWIG does generate code to set array members of type char (allowing you to store a Python string in the structure).

When structure members are wrapped, they are handled as pointers. For example,

struct Foo {
   ...
};

struct Bar {
   Foo f;
};
generates accessor functions such as this:
Foo *Bar_f_get(Bar *b) {
    return &b->f;
}

void Bar_f_set(Bar *b, Foo *val) {
    b->f = *val;
}

17.3.7 C++ classes

C++ classes are wrapped by building a set of low level accessor functions. Consider the following class :

class List {
public:
  List();
  ~List();
  int  search(char *item);
  void insert(char *item);
  void remove(char *item);
  char *get(int n);
  int  length;
static void print(List *l);
};

When wrapped by SWIG, the following functions are created :

List    *new_List();
void     delete_List(List *l);
int      List_search(List *l, char *item);
void     List_insert(List *l, char *item);
void     List_remove(List *l, char *item);
char    *List_get(List *l, int n);
int      List_length_get(List *l);
void     List_length_set(List *l, int n);
void     List_print(List *l);

In Python, these functions are used as follows:
>>> l = new_List()
>>> List_insert(l,"Ale")
>>> List_insert(l,"Stout")
>>> List_insert(l,"Lager")
>>> List_print(l)
Lager
Stout
Ale
>>> print List_length_get(l)
3
>>> print l
_80085608_p_List
>>>

At this low level, C++ objects are really just typed pointers. Member functions are accessed by calling a C-like wrapper with an instance pointer as the first argument. Although this interface is fairly primitive, it provides direct access to C++ objects. A higher level interface known as shadow classes can be built using these low-level accessors. This is described shortly.

17.3.8 C++ classes and type-checking

The SWIG type-checker is fully aware of C++ inheritance. Therefore, if you have classes like this
class Foo {
...
};

class Bar : public Foo {
...
};
and a function
void spam(Foo *f);
then the function spam() accepts Foo * or a pointer to any class derived from Foo. If necesssary, the type-checker also adjusts the value of the pointer (as is necessary when multiple inheritance is used).

17.3.9 C++ overloaded functions

If you have a C++ program with overloaded functions or methods, you will need to disambiguate those methods using %rename. For example:
/* Forward renaming declarations */
%rename(foo_i) foo(int); 
%rename(foo_d) foo(double);
...
void foo(int);           // Becomes 'foo_i'
void foo(char *c);       // Stays 'foo' (not renamed)

class Spam {
public:
   void foo(int);      // Becomes 'foo_i'
   void foo(double);   // Becomes 'foo_d'
   ...
};
Now, in Python, the methods are accessed as follows:
>>> import example
>>> example.foo_i(3)
>>> s = example.new_Spam()
>>> Spam_foo_i(s,3)
>>> Spam_foo_d(s,3.14)
Please refer to the "SWIG Basics" chapter for more information.

17.3.10 Operators

C++ operators can also be wrapped using the %rename directive. All you need to do is give the operator the name of a valid Python identifier. For example:
%rename(add_complex) operator+(Complex &, Complex &);
...
Complex operator+(Complex &, Complex &);
Now, in Python, you can do this:
>>> a = example.new_Complex(2,3)
>>> b = example.new_Complex(4,-1)
>>> c = example.add_complex(a,b)
More details about wrapping C++ operators into Python operators is discussed a little later.

17.4 Input and output parameters

A common problem in some C programs is handling parameters passed as simple pointers. For example:
void add(int x, int y, int *result) {
   *result = x + y;
}
or perhaps
int sub(int *x, int *y) {
   return *x+*y;
}
The easiest way to handle these situations is to use the typemaps.i file. For example:
%module example
%include "typemaps.i"

void add(int, int, int *OUTPUT);
int  sub(int *INPUT, int *INPUT);
In Python, this allows you to pass simple values. For example:
>>> a = add(3,4)
>>> print a
7
>>> b = sub(7,4)
>>> print b
3
>>>
Notice how the INPUT parameters allow integer values to be passed instead of pointers and how the OUTPUT parameter creates a return result.

If you don't want to use the names INPUT or OUTPUT, use the %apply directive. For example:

%module example
%include "typemaps.i"

%apply int *OUTPUT { int *result };
%apply int *INPUT  { int *x, int *y};

void add(int x, int y, int *result);
int  sub(int *x, int *y);

If a function mutates one of its parameters like this,

void negate(int *x) {
   *x = -(*x);
}
you can use INOUT like this:
%include "typemaps.i"
...
void negate(int *INOUT);
In Python, a mutated parameter shows up as a return value. For example:
>>> a = negate(3)
>>> print a
-3
>>>
Note: Since most primitive Python objects are immutable, it is not possible to perform in-place modification of a Python object passed as a parameter.

The most common use of these special typemap rules is to handle functions that return more than one value. For example, sometimes a function returns a result as well as a special error code:

/* send message, return number of bytes sent, along with success code */
int send_message(char *text, int len, int *success);
To wrap such a function, simply use the OUTPUT rule above. For example:
%module example
%include "typemaps.i"
%apply int *OUTPUT { int *success };
...
int send_message(char *text, int *success);
When used in Python, the function will return multiple values.
bytes, success = send_message("Hello World")
if not success:
    print "Whoa!"
else:
    print "Sent", bytes
Another common use of multiple return values are in query functions. For example:
void get_dimensions(Matrix *m, int *rows, int *columns);
To wrap this, you might use the following:
%module example
%include "typemaps.i"
%apply int *OUTPUT { int *rows, int *columns };
...
void get_dimensions(Matrix *m, int *rows, *columns);
Now, in Python:
>>> r,c = get_dimensions(m)

17.5 Simple exception handling

The SWIG %exception directive can be used to define a user-definable exception handler that can convert C/C++ errors into Python exceptions. The chapter on customization features contains more details, but suppose you have a C++ class like the following :

class RangeError {};   // Used for an exception

class DoubleArray {
  private:
    int n;
    double *ptr;
  public:
    // Create a new array of fixed size
    DoubleArray(int size) {
      ptr = new double[size];
      n = size;
    }
    // Destroy an array
    ~DoubleArray() {
       delete ptr;
    }
    // Return the length of the array
    int   length() {
      return n;
    }

    // Get an item from the array and perform bounds checking.
    double getitem(int i) {
      if ((i >= 0) && (i < n))
        return ptr[i];
      else
        throw RangeError();
    }
    // Set an item in the array and perform bounds checking.
    void setitem(int i, double val) {
      if ((i >= 0) && (i < n))
        ptr[i] = val;
      else {
        throw RangeError();
      }
    }
  };

Since several methods in this class can throw an exception for an out-of-bounds access, you might want to catch this in the Python extension by writing the following in an interface file:

%exception {
  try {
    $action
  }
  catch (RangeError) {
    PyErr_SetString(PyExc_IndexError,"index out-of-bounds");
    return NULL;
  }
}

class DoubleArray {
...
};
The exception handling code is inserted directly into generated wrapper functions. The $action variable is replaced with the C/C++ code being executed by the wrapper. When an exception handler is defined, errors can be caught and used to gracefully raise a Python exception instead of forcing the entire program to terminate with an uncaught error.

As shown, the exception handling code will be added to every wrapper function. Since this is somewhat inefficient. You might consider refining the exception handler to only apply to specific methods like this:

%exception getitem {
  try {
    $action
  }
  catch (RangeError) {
    PyErr_SetString(PyExc_IndexError,"index out-of-bounds");
    return NULL;
  }
}

%exception setitem {
  try {
    $action
  }
  catch (RangeError) {
    PyErr_SetString(PyExc_IndexError,"index out-of-bounds");
    return NULL;
  }
}
In this case, the exception handler is only attached to methods and functions named getitem and setitem.

If you had a lot of different methods, you can avoid extra typing by using a macro. For example:

%define RANGE_ERROR
{
  try {
    $action
  }
  catch (RangeError) {
    PyErr_SetString(PyExc_IndexError,"index out-of-bounds");
    return NULL;
  }
}
%enddef

%exception getitem RANGE_ERROR;
%exception setitem RANGE_ERROR;
Since SWIG's exception handling is user-definable, you are not limited to C++ exception handling. See the chapter on "Customization Features" for more examples.

When raising a Python exception from C, use the PyErr_SetString() function as shown above. The following exception types can be used as the first argument.

PyExc_ArithmeticError
PyExc_AssertionError
PyExc_AttributeError
PyExc_EnvironmentError
PyExc_EOFError
PyExc_Exception
PyExc_FloatingPointError
PyExc_ImportError
PyExc_IndexError
PyExc_IOError
PyExc_KeyError
PyExc_KeyboardInterrupt
PyExc_LookupError
PyExc_MemoryError
PyExc_NameError
PyExc_NotImplementedError
PyExc_OSError
PyExc_OverflowError
PyExc_RuntimeError
PyExc_StandardError
PyExc_SyntaxError
PyExc_SystemError
PyExc_TypeError
PyExc_UnicodeError
PyExc_ValueError
PyExc_ZeroDivisionError
These exceptions are actually organized into an hierarchy as shown below. Consult the Python Essential Reference for more details (shameless plug):
PyExc_Exception
    PyExc_SystemExit
    PyExc_StandardError
       PyExc_ArithmeticError
           PyExc_FloatingPointError
           PyExc_OverflowError
           PyExc_ZeroDivisionError
       PyExc_AssertionError
       PyExc_AttributeError
       PyExc_EnvironmentError
           PyExc_IOError  
           PyExc_OSError
       PyExc_EOFError
       PyExc_ImportError
       PyExc_KeyboardInterrupt
       PyExc_LookupError
           PyExc_IndexError
           PyExc_KeyError
       PyExc_MemoryError
       PyExc_NameError
       PyExc_RuntimeError
           PyExc_NotImplementedError
       PyExc_SyntaxError
       PyExc_SystemError
       PyExc_TypeError
       PyExc_ValueError
           PyExc_UnicodeError

Compatibility note: In SWIG1.1, exceptions were defined using the older %except directive:

%except(python) {
  try {
    $action
  }
  catch (RangeError) {
    PyErr_SetString(PyExc_IndexError,"index out-of-bounds");
    return NULL;
  }
}
This is still supported, but it is deprecated. The newer %exception directive provides the same functionality, but it has additional capabilities that make it more powerful.

17.6 Typemaps

This section describes how you can modify SWIG's default wrapping behavior for various C/C++ datatypes using the %typemap directive. This is an advanced topic that assumes familiarity with the Python C API as well as the material in the "Typemaps" chapter.

Before proceeding, it should be stressed that typemaps are not a required part of using SWIG---the default wrapping behavior is enough in most cases. Typemaps are only used if you want to change some aspect of the primitive C-Python interface.

17.6.1 What is a typemap?

A typemap is nothing more than a code generation rule that is attached to a specific C datatype. For example, to convert integers from Python to C, you might define a typemap like this:

%module example

%typemap(in) int {
	$1 = (int) PyLong_AsLong($input);
	printf("Received an integer : %d\n",$1);
}
extern int fact(int n);

Typemaps are always associated with some specific aspect of code generation. In this case, the "in" method refers to the conversion of input arguments to C/C++. The datatype int is the datatype to which the typemap will be applied. The supplied C code is used to convert values. In this code a number of special variable prefaced by a $ are used. The $1 variable is placeholder for a local variable of type int. The $input variable is the input object of type PyObject *.

When this example is compiled into a Python module, it operates as follows:

>>> from example import *
>>> fact(6)
Received an integer : 6
720

In this example, the typemap is applied to all occurrences of the int datatype. You can refine this by supplying an optional parameter name. For example:

%module example

%typemap(in) int n {
	$1 = (int) PyLong_AsLong($input);
	printf("n = %d\n",$1);
}
extern int fact(int n);
In this case, the typemap code is only attached to arguments that exactly match int n.

The application of a typemap to specific datatypes and argument names involves more than simple text-matching--typemaps are fully integrated into the SWIG type-system. When you define a typemap for int, that typemap applies to int and qualified variations such as const int. In addition, the typemap system follows typedef declarations. For example:

%typemap(in) int n {
	$1 = (int) PyLong_AsLong($input);
	printf("n = %d\n",$1);
}
typedef int Integer;
extern int fact(Integer n);    // Above typemap is applied
However, the matching of typedef only occurs in one direction. If you defined a typemap for Integer, it is not applied to arguments of type int.

Typemaps can also be defined for groups of consecutive arguments. For example:

%typemap(in) (char *str, int len) {
    $1 = PyString_AsString($input);
    $2 = PyString_Size($input);
};

int count(char c, char *str, int len);
When a multi-argument typemap is defined, the arguments are always handled as a single Python object. This allows the function to be used like this (notice how the length parameter is ommitted):
>>> example.count('e','Hello World')
1
>>>

17.6.2 Python typemaps

The previous section illustrated an "in" typemap for converting Python objects to C. A variety of different typemap methods are defined by the Python module. For example, to convert a C integer back into a Python object, you might define an "out" typemap like this:
%typemap(out) int {
    $result = PyInt_FromLong((long) $1);
}
The following list details all of the typemap methods that can be used by the Python module:

%typemap(in)

Converts Python objects to input function arguments

%typemap(out)
Converts return value of a C function to a Python object

%typemap(varin)
Assigns a C global variable from a Python object

%typemap(varout)
Returns a C global variable as a Python object

%typemap(freearg)
Cleans up a function argument (if necessary)

%typemap(argout)
Output argument processing

%typemap(ret)
Cleanup of function return values

%typemap(consttab)
Creation of Python constants (constant table)

%typemap(constcode)
Creation of Python constants (init function)
%typemap(memberin)
Setting of structure/class member data

%typemap(globalin)
Setting of C global variables

%typemap(check)
Checks function input values.

%typemap(default)
Set a default value for an argument (making it optional).
%typemap(ignore)
Ignore an argument, but initialize its value.
%typemap(arginit)
Initialize an argument to a value before any conversions occur.
Examples of these methods will appear shortly.

17.6.3 Typemap variables

Within typemap code, a number of special variables prefaced with a $ may appear. A full list of variables can be found in the "Typemaps" chapter. This is a list of the most common variables:

$1

A C local variable corresponding to the actual type specified in the %typemap directive. For input values, this is a C local variable that's supposed to hold an argument value. For output values, this is the raw result that's supposed to be returned to Python.

$input

A PyObject * holding a raw Python object with an argument or variable value.

$result

A PyObject * that holds the result to be returned to Python.

$1_name

The parameter name that was matched.

$1_type

The actual C datatype matched by the typemap.

$1_ltype

An assignable version of the datatype matched by the typemap (a type that can appear on the left-hand-side of a C assignment operation). This type is stripped of qualifiers and may be an altered version of $1_type. All arguments and local variables in wrapper functions are declared using this type so that their values can be properly assigned.
$symname
The Python name of the wrapper function being created.

17.6.4 Useful Functions

When you write a typemap, you usually have to work directly with Python objects. The following functions may prove to be useful.

Python Integer Functions

PyObject *PyInt_FromLong(long l);
long      PyInt_AsLong(PyObject *);
int       PyInt_Check(PyObject *);
Python Floating Point Functions
PyObject *PyFloat_FromDouble(double);
double    PyFloat_AsDouble(PyObject *);
int       PyFloat_Check(PyObject *);
Python String Functions
PyObject *PyString_FromString(char *);
PyObject *PyString_FromStringAndSize(char *, lint len);
int       PyString_Size(PyObject *);
char     *PyString_AsString(PyObject *);
int       PyString_Check(PyObject *);
Python List Functions
PyObject *PyList_New(int size);
int       PyList_Size(PyObject *list);
PyObject *PyList_GetItem(PyObject *list, int i);
int       PyList_SetItem(PyObject *list, int i, PyObject *item);
int       PyList_Insert(PyObject *list, int i, PyObject *item);
int       PyList_Append(PyObject *list, PyObject *item);
PyObject *PyList_GetSlice(PyObject *list, int i, int j);
int       PyList_SetSlice(PyObject *list, int i, int , PyObject *list2);
int       PyList_Sort(PyObject *list);
int       PyList_Reverse(PyObject *list);
PyObject *PyList_AsTuple(PyObject *list);
int       PyList_Check(PyObject *);
Python Tuple Functions
PyObject *PyTuple_New(int size);
int       PyTuple_Size(PyObject *);
PyObject *PyTuple_GetItem(PyObject *, int i);
int       PyTuple_SetItem(PyObject *, int i, pyObject *item);
PyObject *PyTuple_GetSlice(PyObject *t, int i, int j);
int       PyTuple_Check(PyObject *);
Python Dictionary Functions
write me
Python File Conversion Functions
PyObject *PyFile_FromFile(FILE *f);
FILE     *PyFile_AsFile(PyObject *);
int       PyFile_Check(PyObject *);
Abstract Object Interface
write me

17.7 Typemap Examples

This section includes a few examples of typemaps. For more examples, you might look at the files "python.swg" and "typemaps.i" in the SWIG library.

17.7.1 Converting Python list to a char **

A common problem in many C programs is the processing of command line arguments, which are usually passed in an array of NULL terminated strings. The following SWIG interface file allows a Python list object to be used as a char ** object.

%module argv

// This tells SWIG to treat char ** as a special case
%typemap(in) char ** {
  /* Check if is a list */
  if (PyList_Check($input)) {
    int size = PyList_Size($input);
    int i = 0;
    $1 = (char **) malloc((size+1)*sizeof(char *));
    for (i = 0; i < size; i++) {
      PyObject *o = PyList_GetItem($input,i);
      if (PyString_Check(o))
	$1[i] = PyString_AsString(PyList_GetItem($input,i));
      else {
	PyErr_SetString(PyExc_TypeError,"list must contain strings");
	free($1);
	return NULL;
      }
    }
    $1[i] = 0;
  } else {
    PyErr_SetString(PyExc_TypeError,"not a list");
    return NULL;
  }
}

// This cleans up the char ** array we malloc'd before the function call
%typemap(freearg) char ** {
  free((char *) $1);
}

// Now a test function
%inline %{
int print_args(char **argv) {
    int i = 0;
    while (argv[i]) {
         printf("argv[%d] = %s\n", i,argv[i]);
         i++;
    }
    return i;
}
%}

When this module is compiled, the wrapped C function now operates as follows :

>>> from argv import *
>>> print_args(["Dave","Mike","Mary","Jane","John"])
argv[0] = Dave
argv[1] = Mike
argv[2] = Mary
argv[3] = Jane
argv[4] = John
5
In the example, two different typemaps are used. The "in" typemap is used to receive an input argument and convert it to a C array. Since dynamic memory allocation is used to allocate memory for the array, the "freearg" typemap is used to later release this memory after the execution of the C function.

17.7.2 Expanding a Python object to multiple arguments

Suppose that you had a collection of C functions with arguments such as the following:
int foo(int argc, char **argv);
In the previous example, a typemap was written to pass a Python list as the char **argv. This allows the function to be used from Python as follows:
>>> foo(4, ["foo","bar","spam","1"])
Although this works, it's a little awkward to specify the argument count. To fix this, a multi-argument typemap can be defined. This is not very difficult--you only have to make slight modifications to the previous example:
%typemap(in) (int argc, char **argv) {
  /* Check if is a list */
  if (PyList_Check($input)) {
    int i;
    $1 = PyList_Size($input);
    $2 = (char **) malloc((size+1)*sizeof(char *));
    for (i = 0; i < $1; i++) {
      PyObject *o = PyList_GetItem($input,i);
      if (PyString_Check(o))
	$2[i] = PyString_AsString(PyList_GetItem($input,i));
      else {
	PyErr_SetString(PyExc_TypeError,"list must contain strings");
	free($2);
	return NULL;
      }
    }
    $2[i] = 0;
  } else {
    PyErr_SetString(PyExc_TypeError,"not a list");
    return NULL;
  }
}

%typemap(freearg) (int argc, char **argv) {
  free((char *) $2);
}
When writing a multiple-argument typemap, each of the types is referenced by a variable such as $1 or $2. The typemap code simply fills in the appropriate values from the supplied Python object.

With the above typemap in place, you will find it no longer necessary to supply the argument count. This is automatically set by the typemap code. For example:

>>> foo(["foo","bar","spam","1"])

17.7.3 Using typemaps to return arguments

A common problem in some C programs is that values may be returned in arguments rather than in the return value of a function. For example :

/* Returns a status value and two values in out1 and out2 */
int spam(double a, double b, double *out1, double *out2) {
	... Do a bunch of stuff ...
	*out1 = result1;
	*out2 = result2;
	return status;
};

A typemap can be used to handle this case as follows :

%module outarg

// This tells SWIG to treat an double * argument with name 'OutValue' as
// an output value.  We'll append the value to the current result which 
// is guaranteed to be a List object by SWIG.

%typemap(argout) double *OutValue {
    PyObject *o, *o2, *o3;
    o = PyFloat_FromDouble(*$1);
    if ((!$result) || ($result == Py_None)) {
        $result = o;
    } else {
        if (!PyTuple_Check($result)) {
            PyObject *o2 = $result;
            $result = PyTuple_New(1);
            PyTuple_SetItem(target,0,o2);
        }
        o3 = PyTuple_New(1);
        PyTuple_SetItem(o3,0,o);
        o2 = $result;
        $result = PySequence_Concat(o2,o3);
        Py_DECREF(o2);
        Py_DECREF(o3);
    }
}

int spam(double a, double b, double *OutValue, double *OutValue);

The typemap works as follows. First, a check is made to see if any previous result exists. If so, it is turned into a tuple and the new output value is concatenated to it. Otherwise, the result is returned normally. For the sample function spam(), there are three output values--meaning that the function will return a 3-tuple of the results.

As written, the function must accept 4 arguments as input values, last two being pointers to doubles. If these arguments are only used to hold output values (and have no meaningful input value), an additional typemap can be written. For example:

%typemap(ignore) double *OutValue(double temp) {
    $1 = &temp;
}

The ignore typemap forces the input value to be ignored. However, since the argument still has to be set to some meaningful value before calling C, it is set to point to a local variable temp. When the function stores its output value, it will simply be placed in this local variable. As a result, the function can now be used as follows:

>>> a = spam(4,5)
>>> print a
(0, 2.45, 5.0)
>>> x,y,z = spam(4,5)
>>>

17.7.4 Mapping Python tuples into small arrays

In some applications, it is sometimes desirable to pass small arrays of numbers as arguments. For example :

extern void set_direction(double a[4]);       // Set direction vector

This too, can be handled used typemaps as follows :

// Grab a 4 element array as a Python 4-tuple
%typemap(in) double[4](double temp[4]) {   // temp[4] becomes a local variable
  int i;
  if (PyTuple_Check($input)) {
    if (!PyArg_ParseTuple($input,"dddd",temp,temp+1,temp+2,temp+3)) {
      PyErr_SetString(PyExc_TypeError,"tuple must have 4 elements");
      return NULL;
    }
    $1 = &temp[0];
  } else {
    PyErr_SetString(PyExc_TypeError,"expected a tuple.");
    return NULL;
  }
}

This allows our set_direction function to be called from Python as follows :

>>> set_direction((0.5,0.0,1.0,-0.25))

Since our mapping copies the contents of a Python tuple into a C array, such an approach would not be recommended for huge arrays, but for small structures, this approach works fine.

17.7.5 Mapping sequences to C arrays

Suppose that you wanted to generalize the previous example to handle C arrays of different sizes. To do this, you might write a typemap as follows:
// Map a Python sequence into any sized C double array
%typemap(in) double[ANY](double temp[$1_dim0]) {
  int i;
  if (!PySequence_Check($input)) {
      PyErr_SetString(PyExc_TypeError,"Expecting a sequence");
      return NULL;
  }
  if (PyObject_Length($input) != $1_dim0) {
      PyErr_SetString(PyExc_ValueError,"Expecting a sequence with $1_dim0 elements");
      return NULL;
  }
  for (i =0; i < $1_dim0; i++) {
      PyObject *o = PySequence_GetItem($input,i);
      if (!PyFloat_Check(o)) {
         PyErr_SetString(PyExc_ValueError,"Expecting a sequence of floats");
         return NULL;
      }
      temp[i] = PyFloat_AsDouble(o);
  }
  $1 = &temp[0];
}
In this case, the variable $1_dim0 is expanded to match the array dimensions actually used in the C code. This allows the typemap to be applied to types such as:
void foo(double x[10]);
void bar(double a[4], double b[8]);
Since the above typemap code gets inserted into every wrapper function where used, it might make sense to use a helper function instead. This will greatly reduce the amount of wrapper code. For example:
%{
static int convert_darray(PyObject *input, double *ptr, int size) {
  int i;
  if (!PySequence_Check(input)) {
      PyErr_SetString(PyExc_TypeError,"Expecting a sequence");
      return 0;
  }
  if (PyObject_Length(input) != size) {
      PyErr_SetString(PyExc_ValueError,"Sequence size mismatch");
      return 0;
  }
  for (i =0; i < size; i++) {
      PyObject *o = PySequence_GetItem(input,i);
      if (!PyFloat_Check(o)) {
         PyErr_SetString(PyExc_ValueError,"Expecting a sequence of floats");
         return 0;
      }
      ptr[i] = PyFloat_AsDouble(o);
  }
  return 1;
}
%}

%typemap(in) double [ANY](double temp[$1_dim0]) {
   if (!convert_darray($input,temp,$1_dim0))) {
      return NULL;
   }
   $1 = &temp[0];
}

17.7.6 Accessing array structure members

Consider the following data structure :

#define SIZE  8
typedef struct {
    int   values[SIZE];
    ...
} Foo;

By default, SWIG doesn't know how to the handle the values structure member it's an array, not a pointer. In this case, SWIG makes the array member read-only. Reading will simply return a pointer to the first item in the array. To make the member writable, a "memberin" typemap can be used.

%typemap(memberin) int [SIZE] {
    int i;
    for (i = 0; i < SIZE; i++) {
        $1[i] = $input[i];
    }
}

Whenever a int [SIZE] member is encountered in a structure or class, this typemap provides a safe mechanism for setting its value.

As in the previous example, the typemap can be generalized for any dimension. For example:

%typemap(memberin) int [ANY] {
   int i;
   for (i = 0; i < $1_dim0; i++) {
      $1[i] = $input[i];
   }
}
When setting structure members, the input object is always assumed to be a C array of values that have already been converted from the target language. Because of this, the memberin typemap is almost always combined with the use of an "in" typemap. For example, the "in" typemap in the previous section would be used to convert an int[] array to C whereas the "memberin" typemap would be used to copy the converted array into a C data structure.

17.7.7 Pointer handling

Occasionally, it might be necessary to convert pointer values that have been stored using the SWIG typed-pointer representation. Since there are several ways in which pointers can be represented, the following two functions are used to safely perform this conversion:

int SWIG_ConvertPtr(PyObject *obj, void **ptr, swig_type_info *ty, int flags)

Converts a Python object obj to a C pointer. The result of the conversion is placed into the pointer located at ptr. ty is a SWIG type descriptor structure. flags is used to handle error checking and other aspects of conversion. If set, the function converts type-errors into a Python TypeError exception. If set to zero, no Python exception is raised. Returns 0 on success and -1 on error.

PyObject *Swig_NewPointerObj(void *ptr, swig_type_info *ty, int own)

Creates a new Python pointer object. ptr is the pointer to convert, ty is the SWIG type descriptor structure that describes the type, and own is a flag that indicates whether or not Python should take ownership of the pointer.
Both of these functions require the use of a special SWIG type-descriptor structure. This structure contains information about the mangled name of the datatype, type-equivalence information, as well as information about converting pointer values under C++ inheritance. For a type of Foo *, the type descriptor structure is usually accessed as follows:
Foo *f;
if (SWIG_ConvertPtr($input, (void **) &f, SWIGTYPE_p_Foo, 1) == -1) return NULL;

PyObject *obj;
obj = SWIG_NewPointerObj(f, SWIGTYPE_p_Foo, 0);
In a typemap, the type descriptor should always be accessed using the special typemap variable $1_descriptor. For example:
%typemap(in) Foo * {
   if ((SWIG_ConvertPtr($input,(void **) &$1, $1_descriptor,1)) == -1) return NULL;
}

If necessary, the descriptor for any type can be obtained using the $descriptor() macro in a typemap. For example:

%typemap(in) Foo * {
   if ((SWIG_ConvertPtr($input,(void **) &$1, $descriptor(Foo *), 1)) == -1) return NULL;
}
Although the pointer handling functions are primarily intended for manipulating low-level pointers, both functions are fully aware of Python shadow classes (described shortly). Specifically, SWIG_ConvertPtr() will retrieve a pointer from any object that has a this attribute. In addition, SWIG_NewPointerObj() can automatically generate a shadow class object (if applicable).

17.8 Other odds and ends

17.8.1 Adding native Python functions to a SWIG module

Sometimes it is desirable to add a native Python method to a SWIG wrapper file. Suppose you have the following Python/C function :

PyObject *spam_system(PyObject *self, PyObject *args) {
	char *command;
	int sts;
	if (!PyArg_ParseTuple(args,"s",&command))
		return NULL;
	sts = system(command);
	return Py_BuildValue("i",sts);
}

This function can be added to a SWIG module using the following declaration :

%native(system) spam_system;        // Create a command called `system'

Alternatively, you can use the full function declaration like this

%native(system) PyObject *spam_system(PyObject *self, PyObject *args);

or

%native(system) extern PyObject *spam_system(PyObject *self, PyObject *args);

17.9 Python shadow classes

Using the low-level C interface, it is possible to create a Python class that works like the original C++ class. In this case, the Python class is said to shadow the original C++ class. It's really just a wrapper around the C++ original class.

17.9.1 A simple example

One way to illustrate a shadow class is to write one by hand. For example:

class List:
    def __init__(self):
        self.this = new_List()
    def __del__(self):
        delete_List(self.this)
    def search(self,item):
        return List_search(self.this,item)
    def insert(self,item):
        List_insert(self.this,item)
    def remove(self,item):
        List_remove(self.this,item)
    def get(self,n):
        return List_get(self.this,n)
    def __getattr__(self,name):
        if name == "length" : return List_length_get(self.this))
        else : return self.__dict__[name]
    def __setattr__(self,name,value):
        if name == "length": List_length_set(self.this,value)
        else : self.__dict__[name] = value

In the shadow class, a reference to the underlying C++ object is kept in the .this attribute. Methods are then written so that they pass the .this attribute along with the other arguments to the low-level accessor function created by SWIG. This, in turn, allows the class to be used like this:


>>> l = List()
>>> l.insert("Ale")
>>> l.insert("Stout")
>>> l.insert("Lager")
>>> List_print(l.this)
Lager
Stout
Ale
>>> l.length
3

Clearly, this is a much nicer interface than before--and it only required a small amount of Python coding.

17.9.2 Why shadow classes?

Shadow classes are not the only approach to wrapping C++ classes. An alternative approach would be to wrap classes as new Python types in C. However, this requires a substantial amount of complicated C wrapper code. Furthermore, this approach makes it difficult to correctly handle inheritance and other advanced language features. Another more serious problem is that until recently, Python types created in C could not be subclassed or used in the same way as you would use a real Python class. As a result, C++ classes wrapped in this manner are crippled versions of what you would obtain by writing a real class in Python.

Shadow classes allow C++ objects to be easily wrapped by a real Python class. This means that all of the normal features such as inheritance work like you would expect. The fact that such classes are written in Python also simplifies coding since it is much easier to write a class interface in Python than it is in C. Such classes are also much easier to modify since changes can be made without having to recompile any of the low-level C extension code. The main downside to this approach is worse performance--a concern for some users.

17.9.3 Automatic shadow class generation

SWIG can automatically generate shadow classes if you use the -shadow option :

swig -python -shadow interface.i

This generates the usual wrapper file along with an extra file module.py that contains the Python shadow-class code. The name of this file is the same as specified by the %module directive.

Since the shadow class code needs to be placed in a different module than the low-level C wrappers, the primitive interface is placed into a Python extension module named modulec (a 'c' is appended to the module name). When a user imports the module, the module.py is loaded. This file, in turn, imports the low-level C wrapper module to gain access to the accessor functions. For example, in the list example, the shadow file might look roughly like this:

# example
import examplec

class List:
    def __init__(self):
        self.this = examplec.new_List()
    def __del__(self):
        examplec.delete_List(self.this)
    def search(self,item):
        return examplec.List_search(self.this,item)
    def insert(self,item):
        examplec.List_insert(self.this,item)
    def remove(self,item):
        examplec.List_remove(self.this,item)
    def get(self,n):
        return examplec.List_get(self.this,n)
    def __getattr__(self,name):
        if name == "length" : return examplec.List_length_get(self.this))
        else : return self.__dict__[name]
    def __setattr__(self,name,value):
        if name == "length": examplec.List_length_set(self.this,value)
        else : self.__dict__[name] = value
The choice of appending a 'c' to the module name is somewhat non-standard and may cause a module name conflict in certain cases. To fix this, you can run SWIG with the -interface option to change the name of the C module file. For example, this places the low-level C interface into a module named _example:
$ swig -python -shadow -c++ -interface _example example.i
When shadow classes are used, most users don't notice the existence of the low-level C accessors. In fact, there is very little reason to use the low-level functions directly as shadow classes provide all of the needed access.

17.9.4 Compiling modules with shadow classes

To compile a module involving shadow classes, you can use the same procedure as before except that the module name now has an extra `c' appended to the name. Thus, an interface file like this

%module example
... a bunch of declarations ...

might be compiled as follows :

% swig -python -shadow example.i
% gcc -c example.c example_wrap.c -I/usr/local/include/python1.4 \
	-I/usr/local/lib/python1.4/config -DHAVE_CONFIG_H
% ld -shared example.o example_wrap.o -o examplecmodule.so

Notice the naming of `examplecmodule.so' as opposed to `examplemodule.so' that would have been created without shadow classes.

17.9.5 Shadow classes and type-checking

Because shadow classes are used so frequently, the SWIG type-checker is programmed to accept a raw-pointer or any object that contains a this attribute (which is assumed to be a raw-pointer). Thus, the earlier example can be simplified as follows:
class List:
    def __init__(self):
        self.this = new_List()
    def __del__(self):
        delete_List(self)                  # Notice how self is passed instead of
    def search(self,item):                 # self.this
        return List_search(self,item)
    def insert(self,item):
        List_insert(self,item)
    def remove(self,item):
        List_remove(self,item)
    def get(self,n):
        return List_get(self,n)
    def __getattr__(self,name):
        if name == "length" : return List_length_get(self))
        else : return self.__dict__[name]
    def __setattr__(self,name,value):
        if name == "length": List_length_set(self,value)
        else : self.__dict__[name] = value
You can also observe this behavior of the type checker with a little experimentation. For example:
>>> import example
>>> a = example.new_List()
>>> a
'_90651008_p_List'
>>> example.List_insert(a,"Ale")
>>> class Blah: pass
...
>>> b = Blah()
>>> b.this = a
>>> example.List_insert(b,"Lager")
>>>                        
Further details of type-checking and shadow classes will be described later. Stay tuned.

17.9.6 A preview

Shadow classes are one of SWIG's most powerful features. However, to better understand some of the finer points, some other SWIG features need to first be described. Therefore, we return to the topic of shadow classes a little later in this chapter.

17.10 The gory details of shadow classes

This section describes the process by which SWIG creates shadow classes and some of the more subtle aspects of using them.

17.10.1 A simple shadow class

Consider the following declaration from our previous example :

%module pde
struct Grid2d {
  Grid2d(int ni, int nj);
  ~Grid2d();
  double **data;
  int      xpoints;
  int      ypoints;
};

The SWIG generated class for this structure looks like the following:

# This file was created automatically by SWIG.
import pdec
class Grid2dPtr :
    def __init__(self,this):
        self.this = this
        self.thisown = 0
    def __del__(self):
        if self.thisown == 1 :
            pdec.delete_Grid2d(self.this)
    def __setattr__(self,name,value):
        if name == "data" :
            pdec.Grid2d_data_set(self.this,value)
            return
        if name == "xpoints" :
            pdec.Grid2d_xpoints_set(self.this,value)
            return
        if name == "ypoints" :
            pdec.Grid2d_ypoints_set(self.this,value)
            return
        self.__dict__[name] = value
    def __getattr__(self,name):
        if name == "data" : 
            return pdec.Grid2d_data_get(self.this)
        if name == "xpoints" : 
            return pdec.Grid2d_xpoints_get(self.this)
        if name == "ypoints" : 
            return pdec.Grid2d_ypoints_get(self.this)
        return self.__dict__[name]
    def __repr__(self):
        return "<C Grid2d instance>"
class Grid2d(Grid2dPtr):
    def __init__(self,arg0,arg1) :
        self.this = pdec.new_Grid2d(arg0,arg1)
        self.thisown = 1

17.10.2 Module names

Shadow classes are built using the low-level SWIG generated C interface. This interface is named "modulec" where "module" is the name of the module specified in a SWIG interface file. The Python code for the shadow classes is created in a file "module.py". This is the file that should be loaded when a user wants to use the module.

17.10.3 Two classes

For each structure or class found in an interface file, SWIG creates two Python classes. If a class is named "Grid2d", one of these classes will be named "Grid2dPtr" and the other named "Grid2d". The Grid2dPtr class is used to turn wrap a Python class around an already preexisting Grid2d pointer. For example :

>>> gptr = create_grid2d()         # Returns a Grid2d from somewhere
>>> g = Grid2dPtr(gptr)            # Turn it into a Python class
>>> g.xpoints
50
>>>

The Grid2d class, on the other hand, is used when you want to create a new Grid2d object from Python. In reality, it inherits all of the attributes of a Grid2dPtr, except that its constructor calls the corresponding C++ constructor to create a new object. Thus, in Python, this would look something like the following :

>>> g = Grid2d(50,50)           # Create a new Grid2d
>>> g.xpoints
50
>>>

This two class model is a tradeoff. In order to support C/C++ properly, it is necessary to be able to create Python objects from both pre-existing C++ objects and to create entirely new C++ objects in Python. While this might be accomplished using a single class, it would complicate the handling of constructors considerably. The two class model, on the other hand, works, is consistent, and is relatively easy to use. In practice, you probably won't even be aware that there are two classes working behind the scenes.

17.10.4 The this pointer

Within each shadow class, the member "this" contains the actual C/C++ pointer to the object. You can check this out yourself by typing something like this :

>>> g = Grid2d(50,50)
>>> print g.this
_1008fe8_Grid2d_p
>>>

Direct manipulation of the "this" pointer is generally discouraged. In fact forget that you read this.

17.10.5 Object ownership

Ownership is a critical issue when mixing C++ and Python. For example, suppose I create a new object in C++, but later use it to create a Python object. If that object is being used elsewhere in the C++ code, we clearly don't want Python to delete the C++ object when the Python object is deleted. Similarly, what if I create a new object in Python, but C++ saves a pointer to it and starts using it repeatedly. Clearly, we need some notion of who owns what. Since sorting out all of the possibilities is probably impossible, SWIG shadow classes always have an attribute "thisown" that indicates whether or not Python owns an object. Whenever an object is created in Python, Python will be given ownership by setting thisown to 1. When a Python class is created from a pre-existing C/C++ pointer, ownership is assumed to belong to the C/C++ code and thisown will be set to 0.

Ownership of an object can be changed as necessary by changing the value of thisown. When set, Python will call the C/C++ destructor when the object is deleted. If it is zero, Python will never call the C/C++ destructor.

17.10.6 Constructors and Destructors

C++ constructors and destructors will be mapped into Python's __init__ and __del__ methods respectively. Shadow classes always contain these methods even if no constructors or destructors were available in the SWIG interface file. The Python destructor will only call a C/C++ destructor if self.thisown is set.

17.10.7 Member data

Member data of an object is accessed through Python's __getattr__ and __setattr__ methods.

17.10.8 Printing

SWIG automatically creates a Python __repr__ method for each class. This forces the class to be relatively well-behaved when printing or being used interactively in the Python interpreter.

17.10.9 Shadow Functions

Suppose you have the following declarations in an interface file :

%module vector
struct Vector {
	Vector();
	~Vector();
	double x,y,z;
};

Vector addv(Vector a, Vector b);

By default, the function addv will operate on Vector pointers, not Python classes. However, the SWIG Python module is smart enough to know that Vector has been wrapped into a Python class so it will create the following replacement for the addv() function.

def addv(a,b):
	result = VectorPtr(vectorc.addv(a.this,b.this))
	result.thisown = 1
	return result

Function arguments are modified to use the "this" pointer of a Python Vector object. The result is a pointer to the result which has been allocated by malloc or new (this behavior is described in the chapter on SWIG basics), so we simply create a new VectorPtr with the return value. Since the result involved an implicit malloc, we set the ownership to 1 indicating that the result is to be owned by Python and that it should be deleted when the Python object is deleted. As a result, operations like this are perfectly legal and result in no memory leaks :

>>> v = add(add(add(add(a,b),c),d),e)

Substitution of complex datatypes occurs for all functions and member functions involving structure or class definitions. It is rarely necessary to use the low-level C interface when working with shadow classes.

17.10.10 Nested objects

SWIG shadow classes support nesting of complex objects. For example, suppose you had the following interface file :

%module particle

typedef struct {
  Vector();
  double x,y,z;
} Vector;

typedef struct {
  Particle();
 ~Particle();
  Vector r;
  Vector v;
  Vector f;
  int    type;
} Particle;

In this case you will be able to access members as follows :

>>> p = Particle()
>>> p.r.x = 0.0
>>> p.r.y = -1.5
>>> p.r.z = 2.0
>>> p.v = addv(v1,v2)
>>> ...

Nesting of objects is implemented using Python's __setattr__ and __getattr__ functions. In this case, they would look like this :

class ParticlePtr:
	...
	def __getattr__(self,name):
		if name == "r":
			return particlec.VectorPtr(Particle_r_get(self.this))
		elif name == "v":
			return particlec.VectorPtr(Particle_v_get(self.this))
		...
	
	def __setattr__(self,name,value):
		if name == "r":
			particlec.Particle_r_set(self.this,value.this)
		elif name == "v":
			particlec.Particle_v_set(self.this,value.this)
		...

The attributes of any given object are only converted into a Python object when referenced. This approach is more memory efficient, faster if you have a large collection of objects that aren't examined very often, and works with recursive structure definitions such as :

struct Node {
	char *name;
	struct Node *next;
};

Nested structures such as the following are also supported by SWIG. These types of structures tend to arise frequently in database and information processing applications.

typedef struct {  
	unsigned int dataType;
	union {
		int       intval;
		double    doubleval;
		char     *charval;
		void     *ptrvalue;
		long      longval;
		struct {
			int    i;
			double f;
			void   *v;
			char name[32];
              } v;
	} u;
} ValueStruct;

Access is provided in an entirely natural manner,

>>> v = new_ValueStruct()       # Create a ValueStruct somehow
>>> v.dataType
1
>>> v.u.intval
45
>>> v.u.longval
45
>>> v.u.v.v = _0_void_p
>>>

To support the embedded structure definitions, SWIG has to extract the internal structure definitions and use them to create new Python classes. In this example, the following shadow classes are created :

# Class corresponding to union u member
class ValueStruct_u :
	...
# Class corresponding to struct v member of union u
class ValueStruct_u_v :
	...

The names of the new classes are formed by appending the member names of each embedded structure.

17.10.11 Inheritance and shadow classes

Since shadow classes are implemented in Python, you can use any of the automatically generated classes as a base class for more Python classes. However, you need to be extremely careful when using multiple inheritance. When multiple inheritance is used, at most ONE SWIG generated shadow class can be involved. If multiple SWIG generated classes are used in a multiple inheritance hierarchy, you will get name clashes on the this pointer, the __getattr__ and __setattr__ functions won't work properly and the whole thing will probably crash and burn. Perhaps it's best to think of multiple inheritance as a big hammer that can be used to solve alot of problems, but it hurts quite alot if you accidently drop it on your foot....

17.10.12 Methods that return new objects

By default SWIG assumes that constructors are the only functions returning new objects to Python. However, you may have other functions that return new objects as well. For example :

Vector *cross_product(Vector *v1, Vector *v2) {
	Vector *result = new Vector();
	result = ... compute cross product ...
	return result;
}

When the value is returned to Python, we want Python to assume ownership. The brute force way to do this is to simply change the value of thisown. For example :

>>> v = cross_product(a,b)
>>> v.thisown = 1                        # Now Python owns it

Unfortunately, this is ugly and it doesn't work if we use the result as a temporary value :

w = vector_add(cross_product(a,b),c)     # Results in a memory leak

However, you can provide a hint to SWIG when working with such a function as shown :

// C Function returning a new object
%new Vector *cross_product(Vector *v1, Vector *v2);

The %new directive only provides a hint that the function is returning a new object. The Python module will assign proper ownership of the object when this is used.

17.10.13 Performance concerns and hints

Shadow classing is primarily intended to be a convenient way of accessing C/C++ objects from Python. However, if you're directly manipulating huge arrays of complex objects from Python, performance may suffer greatly. In these cases, you should consider implementing the functions in C or thinking of ways to optimize the problem.

There are a number of ways to optimize programs that use shadow classes. Consider the following two code fragments involving the Particle data structure in a previous example :

def force1(p1,p2):
	dx = p2.r.x - p1.r.x
	dy = p2.r.y - p1.r.y
	dz = p2.r.z - p1.r.z
	r2 = dx*dx + dy*dy + dz*dz
	f = 1.0/(r2*math.sqrt(r2))
	p1.f.x = p1.f.x + f*dx
	p2.f.x = p2.f.x - f*dx
	p1.f.y = p1.f.y + f*dy
	p2.f.y = p2.f.y - f*dy
	p1.f.z = p1.f.z + f*dz
	p2.f.z = p2.f.z - f*dz

def force2(p1,p2):
	r1 = p1.r
	r2 = p2.r
	dx = r2.x - r1.x
	dy = r2.y - r1.y
	dz = r2.z - r1.z
	r2 = dx*dx + dy*dy + dz*dz
	f = 1.0/(r2*math.sqrt(r2))
	f1 = p1.f
	f2 = p2.f
	f1.x = f1.x + f*dx
	f2.x = f2.x - f*dx
	f1.y = f1.y + f*dy
	f2.y = f2.y - f*dy
	f1.z = f1.z + f*dz
	f2.z = f2.z - f*dz

The first calculation simply works with each Particle structure directly. Unfortunately, it performs alot of dereferencing of objects. If the calculation is restructured to use temporary variables as shown in force2, it will run significantly faster--in fact, on my machine, the second code fragment runs more than twice as fast as the first one.

If performance is even more critical you can use the low-level C interface which eliminates all of the overhead of going through Python's class mechanism (at the expense of coding simplicity). When Python shadow classes are used, the low level C interface can still be used by importing the `modulec' module where `module' is the name of the module you used in the SWIG interface file.


SWIG 1.3 - Last Modified : January 27, 2002