Boost.Python - June 2002 Progress Report

###  | # Boost.Python ## June 2002 Progress Report

## Contents

Introduction
`handle<T>`
`object`

`object` operators
`object` conversions

`list`
`Numerics`
Community
What's Next

## Introduction

July was mostly focused on allowing expressive manipulation of individual Python objects, or what Ralf Grosse-Kunstleve calls "Writing Python in C++". The work began with this posting, which outlines the issues and intention.

## `handle<T>`

The most basic element needed was a replacement for the `reference<>` class template and the `ref` typedef from Boost.Python v1, a simple smart pointer to a Python object. The old v1 typedef "`ref`" (for `reference<PyObject>`) had to be retired because I thought it would be too confusing given the importance of `boost::ref()` to this library. I began a discussionof possible names, and it was eventually decided to rename `reference` to `handle` and supply a default argument so that `ref` could be spelled `handle<>` without an additional typedef. There were also some interface changes to make it safer and more-efficient to interface with the raw `PyObject*`s forced on us by Python's 'C' API. A discussion of those protocols can be found here.

## `object`

It is intended that users will seldom need or want to work with `handle<>`; its major distinguishing features are that it gives direct access to the underlying object representation through `operator*` and `operator->`, and that can be `NULL`, both sources of danger. Instead the library provides a class called `object`, which encapsulates a valid Python object and provides a similar interface to Python's.

### `object` operators

The first challenge was to provide support for object manipulations using a Python-like syntax, mostly in the form of operator overloads:

Python | C++
--- | ---
`y = x.foo` | `y = x.attr("foo");`
`x.foo = 1` | `x.attr("foo") = 1;`
`y = x[z]` | `y = x[z];`
`x[z] = 1` | `x[z] = 1;`
`y = x[3:-1]` | `y = x.slice(3,-1);`
`y = x[3:]` | `y = x.slice(3,_);`
`y = x[:-2]` | `y = x.slice(_,-2);`
`z = x(1, y)` | `z = x(1, y);`
`z = x.f(1, y)` | `z = x.attr("f")(1, y);`
`not x` | `!x`
`x and y` | `x and y`

I'm still a unsatisfied with the interface for attribute access. There original proposal used a syntax like this one:

    y = x._("foo");
    x._("foo") = 1;

which was only marginally better than what we've got. Niki Spahiev then pointed out a potential conflict with the macro which GNU Gettext suggests people define. This unfortunate state of affairs forced us into using `attr` instead. I'd still like to find a better interface, but the lack of overloadable C++ operators which aren't already used in Python is an obstacle. The comma operator is still a possibility, but it has the wrong precedence:

    y = x,"foo"    // error
    x,"foo" = 1;   // error

    y = (x,"foo"); // ok
    (x,"foo") = 1; // ok

Well, I guess we could consider adding that to the interface without removing `attr()`, to see how it plays out...

### `object` conversions

The `object` class also provided an opportunity to replace Boost.Python v1's `to_python()` as a user-level interface. Instead, `object` has a templated constructor which can be used to convert any C++ object to Python using the same underlying mechanisms used for the arguments to `call<>`.

Incidentally, the implementation of operator and conversion support for object uncovered an inordinate number of compiler bugs in our targeted platforms. It was a lot more "interesting" than it should have been.

## `list`

With `object` implemented, it was time to begin replacing the ad-hoc implementations of `list`, `string`, and `dictionary` supplied by Boost.Python v1 with something more robust. I started with `list` as an example. Because `object` already provides all of the requisite operators, publicly deriving `list` from object seemed like a good choice. The remaining issues were what do do about the one-argument list constructor (which in Python attempts to convert its argument to a list), and how to deal converting with `list` arguments to wrapped functions. Some of the issues are laid out in this thread. Ultimately, it was decided that `list(x)` should do the same thing in C++ as in Python (conversion), while `list` arguments should only match Python `list`s (and `list` subclasses). The implementation worked well, and provided a roadmap for the protocol to be used for implementation of the other built-in types.

## Numerics

Support for C++ `long long` and `unsigned long long` (and `__int64` on MSVC) to/from python conversions was added this month. We also improved handling of numeric overflows when converting, e.g., a Python int to a type with a more limited range of representation.

## Community

- Ralf W. Grosse-Kunstleve and Nick Sauter have implemented multiplatform nightly build-and-test runs for Boost.Python V2 at LBL.
- Dave Hawkes has made significant progress on generating the Python built-in function and API wrappers
- Achim Domma has agreed to take up the job of implementing the `str`, `dict`, and `tuple` classes.

Deep thanks to all the Boost.Python contributors! This project wouldn't be possible without your participation.

## What's Next

As I write this we are already well into the month of July, so I suggest you consult the Mailing List Archive if you want to know what's been happening. Otherwise you'll just have to wait till next month (hopefully the beginning).

Revised 13 November, 2002

© Copyright Dave Abrahams 2002.