A Python Metaclass for "extra bad" errors in Google App Engine

So now here we are, having tried to handle errors in Google App Engine...and failed all because silly DeadlineExceededError jumps over Exception in the inheritance chain and goes right for BaseException. How can we catch these in our handlers while staying Pythonic*?

First and foremost, in the case of a timeout, we need to explicitly catch a DeadlineExceededError. To do so, we can use a decorator (hey, that's Pythonic) in each and every handler for each and every HTTP verb. (Again, prepare yourselves, a bunch of code is about to happen. See the necessary imports at the bottom of the post.)

def deadline_decorator(method):

    def wrapped_method(self, *args, **kwargs):
        try:
            method(self, *args, **kwargs)
        except DeadlineExceededError:
            traceback_info = ''.join(format_exception(*sys.exc_info()))
            email_admins(traceback_info, defer_now=True)

            serve_500(self)

    return wrapped_method

Unfortunately, having to manually

is not so Pythonic. At this point I was stuck and wanted to give up, but asked for some advice on G+ and actually got what I needed from the all knowing Ali Afshar. What did I need? Metaclasses.

Before showing the super simple metaclass I wrote, you need to know one thing from StackOverflow user Kevin Samuel:

The main purpose of a metaclass is to change the class automatically, when it's created.

With the __new__ method, the type object in Python actually constructs a class (which is also an object) by taking into account the name of the class, the parents (or bases) and the class attritubutes. So, we can make a metaclass by subclassing type and overriding __new__:

class DecorateHttpVerbsMetaclass(type):

    def __new__(cls, name, bases, cls_attr):
        verbs = ['get', 'post', 'put', 'delete']
        for verb in verbs:
            if verb in cls_attr and isinstance(cls_attr[verb], function):
                cls_attr[verb] = deadline_decorator(cls_attr[verb])

        return super(DecorateHttpVerbsMetaclass, cls).__new__(cls, name,
                                                              bases, cls_attr)

In DecorateHttpVerbsMetaclass, we look for four (of the nine) HTTP verbs, because heck, only seven are supported in RequestHandler, and we're not that crazy. If the class has one of the verbs as an attribute and if the attribute is a function, we decorate it with deadline_decorator.

Now, we can rewrite our subclass of RequestHandler with one extra line:

class ExtendedHandler(RequestHandler):
    __metaclass__ = DecorateHttpVerbsMetaclass

    def handle_exception(self, exception, debug_mode):
        traceback_info = ''.join(format_exception(*sys.exc_info()))
        email_admins(traceback_info, defer_now=True)

        serve_500(self)

By doing this, when the class ExtendedHandler is built (as an object), all of its attributes and all of its parent classes (or bases) attributes are checked and possibly updated by our metaclass.

And now you and James Nekbehrd can feel like a boss when your app handles errors.

Imports:

from google.appengine.api import mail
from google.appengine.ext.deferred import defer
from google.appengine.ext.webapp import RequestHandler
from google.appengine.runtime import DeadlineExceededError
import sys
from traceback import format_exception
from SOME_APP_SPECIFIC_LIBRARY import serve_500
from LAST_POST import email_admins

*Pythonic:

An idea or piece of code which closely follows the most common idioms of the Python language, rather than implementing code using concepts common to other languages.

Notes:

• Using grep -r "Exception)" . | grep "class " I have convinced myself (for now) that the only errors AppEngine will throw that do not inherit from Exception are DeadlineExceededError, SystemExit, and KeyboardInterrupt so that is why I only catch the timeout.
• You can also use webapp2 to catch 500 errors, even when handle_exception fails to catch them.

Disclaimer: Just because you know what a metaclass is doesn't mean you should use one:

• "Don't do stuff like this though, what is your use case?" -Ali Afshar
• "Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don't (the people who actually need them know with certainty that they need them, and don't need an explanation about why)." -Python Guru Tim Peters
• "The main use case for a metaclass is creating an API." -Kevin Samuel

About Bossy Lobster