Jul 7
2011

Python: Throwing an exception into a generator

EDIT: Thanks to Andrew Bennets, this problem is solved -- Python supports exactly the feature I am after using the "throw" method of generator objects. Original post below for interest's sake.

I'm writing some code that uses an asynchronous coding style, using generators.

Here's how the code works:

def rpcClient():
    initialisation()

    result = yield someBlockingFunction
    doSomethingWith(result)

    result = yield someOtherBlockingFunction
    doSomethingWith(result)

Here, rpcClient does some initialisation. After initialisation, it would like to call someBlockingFunction, but it doesn't want to block. So, instead, it passes the name of the function to its caller, via yield. The function is then called somehow (details later). When the function returns, its return value is passed back to rpcClient as "result".

The problem with this scheme is: what happens if someBlockingFunction() raises an exception? What I would like to do is throw the exception back into rpcClient, so that (from the perspective of rpcClient) it looks like the yield itself generated the exception.

As far as I can tell, there isn't a way to do that. It's kind of unfortunate, because the method above is otherwise a very neat (as in, few LOC, relatively comprehensible) to handle blocking functions without "stack smashing" or introducing extra threads.

I can't even do something like define a function to do the yield for me, because in Python the decision about whether a function is a generator or not is made at compile time and is based on the presence of specific syntax (i.e. the word "yield").

(In my code, someBlockingFunction is actually an RPC call. When you call it, you pass a callback, which is then called when the function returns. So the code calling rpcClient can handle all that detail behind-the-scenes, without requiring the complexity of the RPC to "escape" into all RPC clients.)