Wednesday, November 30, 2011

A Python Metaclass for "extra bad" errors in Google App Engine

So now here we are, having tried to handle errors in Google App Engine...and failed all because silly DeadlineExceededError jumps over Exception in the inheritance chain and goes right for BaseException. How can we catch these in our handlers while staying Pythonic*?

First and foremost, in the case of a timeout, we need to explicitly catch a DeadlineExceededError. To do so, we can use a decorator (hey, that's Pythonic) in each and every handler for each and every HTTP verb. (Again, prepare yourselves, a bunch of code is about to happen. See the necessary imports at the bottom of the post.)
def deadline_decorator(method):

    def wrapped_method(self, *args, **kwargs):
        try:
            method(self, *args, **kwargs)
        except DeadlineExceededError:
            traceback_info = ''.join(format_exception(*sys.exc_info()))
            email_admins(traceback_info, defer_now=True)

            serve_500(self)

    return wrapped_method
Unfortunately, having to manually
is not so Pythonic. At this point I was stuck and wanted to give up, but asked for some advice on G+ and actually got what I needed from the all knowing Ali Afshar. What did I need? Metaclasses.

Before showing the super simple metaclass I wrote, you need to know one thing from StackOverflow user Kevin Samuel:
The main purpose of a metaclass is to change the class automatically, when it's created.
With the __new__ method, the type object in Python actually constructs a class (which is also an object) by taking into account the name of the class, the parents (or bases) and the class attritubutes. So, we can make a metaclass by subclassing type and overriding __new__:
class DecorateHttpVerbsMetaclass(type):

    def __new__(cls, name, bases, cls_attr):
        verbs = ['get', 'post', 'put', 'delete']
        for verb in verbs:
            if verb in cls_attr and isinstance(cls_attr[verb], function):
                cls_attr[verb] = deadline_decorator(cls_attr[verb])

        return super(DecorateHttpVerbsMetaclass, cls).__new__(cls, name,
                                                              bases, cls_attr)
In DecorateHttpVerbsMetaclass, we look for four (of the nine) HTTP verbs, because heck, only seven are supported in RequestHandler, and we're not that crazy. If the class has one of the verbs as an attribute and if the attribute is a function, we decorate it with deadline_decorator.

Now, we can rewrite our subclass of RequestHandler with one extra line:
class ExtendedHandler(RequestHandler):
    __metaclass__ = DecorateHttpVerbsMetaclass

    def handle_exception(self, exception, debug_mode):
        traceback_info = ''.join(format_exception(*sys.exc_info()))
        email_admins(traceback_info, defer_now=True)

        serve_500(self)
By doing this, when the class ExtendedHandler is built (as an object), all of its attributes and all of its parent classes (or bases) attributes are checked and possibly updated by our metaclass.

And now you and James Nekbehrd can feel like a boss when your app handles errors.

Imports:
from google.appengine.api import mail
from google.appengine.ext.deferred import defer
from google.appengine.ext.webapp import RequestHandler
from google.appengine.runtime import DeadlineExceededError
import sys
from traceback import format_exception
from SOME_APP_SPECIFIC_LIBRARY import serve_500
from LAST_POST import email_admins
*Pythonic:
An idea or piece of code which closely follows the most common idioms of the Python language, rather than implementing code using concepts common to other languages.
Notes:
  • Using grep -r "Exception)" . | grep "class " I have convinced myself (for now) that the only errors AppEngine will throw that do not inherit from Exception are DeadlineExceededError, SystemExit, and KeyboardInterrupt so that is why I only catch the timeout.
  • You can also use webapp2 to catch 500 errors, even when handle_exception fails to catch them.

Disclaimer: Just because you know what a metaclass is doesn't mean you should use one:
  • "Don't do stuff like this though, what is your use case?" -Ali Afshar
  • "Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don't (the people who actually need them know with certainty that they need them, and don't need an explanation about why)." -Python Guru Tim Peters
  • "The main use case for a metaclass is creating an API." -Kevin Samuel

Sunday, November 27, 2011

Handling errors in Google App Engine...and failing

After spending a nontrivial amount of my nights and weekends working on an AppEngine app, I wanted a good way to monitor the logs without checking in on them every day. After a particularly frustrating weekend of updates that exposed unnoticed bugs that had yet to be triggered by the app, I set out to find such a way. I set out to find a Pythonic* way.

Since I knew the App Engine Mail API was super easy to configure,  I figured I would just email myself every time there was an exception, before serving my default 500 error page. To do so, I just needed to subclass the default RequestHandler with my own handle_exception method. (OK, prepare yourselves, a bunch of code is about to happen. See the necessary imports at the bottom of the post.)
class ExtendedHandler(RequestHandler):

    def handle_exception(self, exception, debug_mode):
        traceback_info = ''.join(format_exception(*sys.exc_info()))
        email_admins(traceback_info, defer_now=True)

        serve_500(self)
Awesome! By making all my handlers inherit from ExtendedHandler, I can use the native Python modules traceback and sys to get the traceback and my handy dandy
def email_admins(error_msg, defer_now=False):
    if defer_now:
        defer(email_admins, error_msg, defer_now=False)
        return

    sender = 'YOUR APP Errors <errors@your_app_id_here.appspotmail.com>'
    to = 'Robert Admin <bob@example.com>, James Nekbehrd <jim@example.com>'
    subject = 'YOUR APP Error: Admin Notify'
    body = '\n'.join(['Dearest Admin,',
                      '',
                      'An error has occurred in YOUR APP:',
                      error_msg,
                      ''])

    mail.send_mail(sender=sender, to=to,
                   subject=subject, body=body)
to send out the email in the deferred queue** so as not to hold up the handler serving the page. Mission accomplished, right? WRONG!

Unfortunately, handle_exception only handles the "right" kind of exceptions. That is, exceptions which inherit directly from Python's Exception. From the horse's mouth:
Exceptions should typically be derived from the Exception class, either directly or indirectly.
But. But! If the app fails because a request times out, a DeadlineExceededError is thrown and handle_exception falls on its face. Why? Because DeadlineExceededError inherits directly from Exception's parent class: BaseException.  (Gasp)

It's OK little ones, in my next post I explain how I did it while keeping my code Pythonic by using metaclasses.

Imports:
from google.appengine.api import mail
from google.appengine.ext.deferred import defer
from google.appengine.ext.webapp import RequestHandler
import sys
from traceback import format_exception
from SOME_APP_SPECIFIC_LIBRARY import serve_500
*Pythonic:
An idea or piece of code which closely follows the most common idioms of the Python language, rather than implementing code using concepts common to other languages.
**Deferred Queue: Make sure to enable the deferred library in your app.yaml by using deferred: on in your builtins.

Thursday, November 17, 2011

Quick and Dirty: Santa's Coming

I have been wanting to write a post for awhile, but was travelling for a work event and while on the road I decided to be lazy.

Since I just so happen to use a few GData APIs occasionally in my day to day work, most of the post ideas revolve around quirky things I have done or want to do with the APIs. Also, due to my obscene love for Python, all my mashups seem to end up using the Python Client for GData.

Back-story: As I was finalizing travel and gifts for my winter holiday back home, I called an old friend to let him know I'd be home in 40 days. After relaying this information to a few other people, I noted to my girlfriend that it would be nice if a computer would remind me of the count every day. This is where this quick and dirty pair of scripts come in to remind me when Santa is coming.

Pre-work — Account Settings: To allow an app to make requests on my behalf, I signed up to Manage my Domain for use with Google Apps, etc. For illustration purposes, let's say I used http://example.com (in reality, I used a pre-existing App of mine, I really just needed an OAuth token for one time use, no real safety concerns there). After adding this domain in the management page, I am able to get my "OAuth Consumer Key" and "OAuth Consumer Secret" which we'll say are EXAMPLE_KEY and EXAMPLE_SECRET in this example. Also in the management page, I set my "OAuth 2.0 Redirect URIs" and made sure my app can serve that page (even if it is a 404). Again for illustration, let's pretend I used http://example.com/verify.

After doing this settings pre-work, I have two scripts to do the work for me.

First script — get the OAuth Token:
import gdata.calendar.client
import gdata.gauth

gcal = gdata.calendar.client.CalendarClient()
oauth_callback = 'http://example.com/verify'
scopes = ['https://www.google.com/calendar/feeds/']
consumer_key = 'EXAMPLE_KEY'
consumer_secret = 'EXAMPLE_SECRET'
request_token = gcal.get_oauth_token(scopes, oauth_callback,
                                     consumer_key, consumer_secret)
out_str = ('Please visit https://www.google.com/accounts/OAuthAuthorize'
           'Token?hd=default&oauth_token=%s' % request_token.token)
print out_str
follow = raw_input('Please entry the follow link after authorizing:\n')
gdata.gauth.authorize_request_token(request_token, follow)
gcal.auth_token = gcal.get_access_token(request_token)
print 'TOKEN:', gcal.auth_token.token
print 'TOKEN_SECRET:', gcal.auth_token.token_secret
This script "spoofs" the OAuth handshake by asking the user (me) to go directly to the OAuth Authorize page. After doing so and authorizing the App, I am redirected to http://example.com/verify with query parameters for oauth_verifier and oauth_token. These are then used by the gauth section of the GData library to finish the OAuth handshake. Once the handshake is complete, the script prints out a necessary token and token secret to be used by the second script. I would advise piping the output to a file, augmenting the script to write them to a file, or writing these down (this is a joke, please don't write down 40 plus character goop that was produced by your computer). For the next script, let's pretend our token is FAKE_TOKEN and our token secret is FAKE_TOKEN_SECRET.

Second script — insert the events:
# General libraries
from datetime import date
from datetime import timedelta

# Third-party libraries
import atom
import gdata.gauth
import gdata.calendar.client
import gdata.calendar.data

gcal = gdata.calendar.client.CalendarClient()
auth_token = gdata.gauth.OAuthHmacToken(consumer_key='EXAMPLE_KEY',
                                        consumer_secret='EXAMPLE_SECRET',
                                        token='FAKE_TOKEN',
                                        token_secret='FAKE_TOKEN_SECRET',
                                        auth_state=3)
gcal.auth_token = auth_token

today = date.today()
days_left = (date(year=2011, month=12, day=23) - today).days

while days_left >= 0:
    event = gdata.calendar.data.CalendarEventEntry()
    if days_left > 1:
        msg = '%s Days Until Home for Christmas' % days_left
    elif days_left == 1:
        msg = '1 Day Until Home for Christmas'
    elif days_left == 0:
        msg = 'Going Home for Christmas'
    event.title = atom.data.Title(msg)

    # When
    start_time = '2011-%02d-%02dT08:00:00.000-08:00' % (today.month, today.day)
    end_time = '2011-%02d-%02dT09:00:00.000-08:00' % (today.month, today.day)
    event.when.append(gdata.calendar.data.When(
        start=start_time,
        end=end_time,
        reminder=[gdata.data.Reminder(hours='1')]))

    gcal.InsertEvent(event)

    today += timedelta(days=1)
    days_left -= 1
This script first authenticates by using the key/secret pair for the application (retrieved from the settings page) and the key/secret pair for the user token (that we obtained from the first script). To authenticate, we explicitly construct an HMAC-SHA1 signed token in the final auth state (3) of two-legged OAuth and then set the token on our calendar client (gcal).

After authenticating, we start with today and figure out the number of days in the countdown given my return date of December 23, 2011. With these in hand, we can loop through until there are no days left, creating a CalendarEventEntry with title as the number of days left in the countdown and occurring from 8am to 9am PST (UTC -8). Notice also I include a gdata.data.Reminder so I get an email at 7am every morning (60 minutes before the event) updating my brain on the length of the countdown!

Cleanup: Be sure to go to your issued tokens page and revoke access to the App (e.g. http://example.com) after doing this to avoid any unwanted security issues.

References: I have never read this, but I'm sure the documentation on Registration for Web-Based Applications is very helpful.

Notes:
  • You can annoy other people by inviting them to these events for them as well. To do so, simply add the following two lines before inserting the event
    who_add = gdata.calendar.data.EventWho(email='name@mail.com')
    event.who.append(who_add)
  • Sometimes inserting an item results in a RedirectError, so it may be safer to try the insert multiple times with a helper function such as the following:
    def try_insert(attempts, gcal, event):
        from time import sleep
        from gdata.client import RedirectError
    
        while attempts > 0:
          try:
              gcal.InsertEvent(event)
              break
          except RedirectError:
              attempts -= 1
              sleep(3)
              pass
    
        if attempts == 0:
            print 'Insert "%s" failed' % event.title.text
  • In what I swear was a complete coincidence, v3 of the Calendar API was announced today. I will try to use the new documentation to redo this quick and dirty example with v3.