Adam Lowry's Robotic Archive

'\n'.join([entry.text for entry in adam.blog])

Home | About

Introduction to gunicorn

written by Adam Lowry, on Jul 15, 2010 1:31:00 PM.

I did this month's Module of the Month at PDX Python on gunicorn. While it's technically not a module it is easy_installable and framework independent, so we made an exception to the rules. Here slides from the talk, with a bonus module at the end!:

Edit: Here are the raw slides:

Django Dose NoSQL panel

written by Adam Lowry, on Apr 21, 2010 10:47:00 AM.

This past weekend I participated in a panel discussion on integrating the newer non-relational ("NoSQL") databases in Django. We've been using MongoDB for a while at Urban Airship, so I was happy to lend some of our experiences. Unfortunately I didn't have much intelligent to add to the discussion, but thanks to Alex and Eric for inviting me along.

Testing My Patience

written by Adam Lowry, on Aug 12, 2009 9:05:00 AM.

Last night I gave a talk on automated testing for Python projects at the Portland Python Users Group. Here are the slides:

Sorry for the large filesize; perhaps next time I won't feel the need to pad it out with unnecessary pictures.

A few notes:

  • This is less a survey of all Python testing systems, and more of a survey of how I've done my testing over the years.
  • I've used Buildbot for continuous integration, but I have been recommended Hudson (despite being in Java).
  • When you're just getting started, make your tests as simple as possible, and refactor mercilessly. And mocking/stubbing libraries only when you need them, and try to limit their use until you feel very comfortable.
Thanks everyone for coming out!

Edit: Here it is on SlideShare:

Adam at iPhoneDevCamp

written by Adam Lowry, on Aug 1, 2009 11:22:00 AM.

Today and tomorrow I'm at iPhoneDevCamp (along with Scott and Steven), hacking with people on Urban Airship integration. I'm also doing a talk on Apple's Push Notification service (not about our product, but the general technical system). Slides from that talk are available, although it's possible they'll be edited after the talk.

Why I'm excited about Open Source Bridge

written by Adam Lowry, on May 14, 2009 10:17:00 PM.

Personally, I'm not terribly disappointed that OSCON's not in Portland this year. While the people it attracts are fantastic and there are great meetups concurrent to the event, I've had a hard time finding interesting things at the conference proper. But this year we've got Open Source Bridge, and I can't wait. That such a thing can come out of the community itself without corporate organization is mind boggling, and having seen the dedication and drive in the organizers and volunteers, I know it's going to be great.

The session lineup is incredible. There's a nice mix of some big-name names and new voices that I think will bring a fresh perspective. Three sessions I'm particularly looking forward to:

  • Assholes are killing your project -- Anyone that's spent any time on IRC knows this, but every day it makes me cringe a bit more. What is the actual effect of self-appointed "enforcers" yelling RTFM?
  • Configuration management panel -- a topic I really should know more about. Panels are sometimes hit and miss; you often don't get much depth. But since I know nothing but know I know nothing, I'm ready to absorb everything.
  • Web server shootout -- fellow PDX Python attendee Michael Schurter is going benchmark crazy to try to tease out practical recommendations on what web servers to use for what tasks.

And finally, the 24-hour hacker lounge is going to rock. I might have to find some new projects to hack on to get the most out of it.

Hg Is Not So Scary

written by Adam Lowry, on Apr 3, 2009 2:08:00 PM.

Half of the developers at work had been using Mercurial for the past few months, and now that we've joined a different project I'm planning on moving the entire dev team away from Subversion. One dev was a little apprehensive after a brief stint fixing up a demo stored in Hg, so I put together a presentation as part of our lunch brown bag series (Thursday Tech Talks, because we like alliteration).

I'm not sure if the presentation will be useful to anyone else, as I tailored it to the devs here and had a stint at the whiteboard drawing revisions, but here are the slides anyways: Hg Is Not So Scary.

Pictures in Temples

written by Adam Lowry, on Mar 1, 2009 9:15:00 PM.

It's already March, and I'm still piecing through pictures from our trip to Cambodia and uploading them bit by bit. Here are a few of my favorites—I believe Tara took the first three of these and I took the other two, but I don't know for sure.

Bayon

Preah Kahn

Ta Prohm

Angkor Wat

Flower and Buddha

OpenID for Zine

written by Adam Lowry, on Mar 1, 2009 8:01:00 PM.

Continuing my experiments with Zine I finished an initial go at an OpenID plugin. The plugin requires the python-openid library (2.2.1 was all I tested) and Zine 0.1.2.

There's certainly room for improvement with the experience, and it's currently only using the FileOpenIDStore; this will need to be replaced with a database-backed store.

If you'd like to take a look at the code, it's up on BitBucket. It's incredibly not done (it'll fail if you cancel the OpenID process, for example)—but it "works."

Form Processing with Flatland

written by Adam Lowry, on Feb 23, 2009 6:12:00 PM.

For the past three months at work we've been using Jason Kirtland's Flatland form processing library for HTML form definition and validation. We recently had a sprint (along with Michel Pelletier and Ben Stover) to head towards a release. One thing remaining is to spruce up the docs; I thought I'd write up a little introduction here as a part of that effort.

While code is the easiest way to explain using Flatland, it's important to start with understanding that Flatland has two main sets of objects that work together, schema objects and element objects. Schema objects represent the hierarchical data model your application uses, and element objects are a real instantiation of that schema with data.

If you've used Django's forms or WTForms the basic structure will be familiar.

import flatland

class CommentForm(flatland.Form):
    schema = [
        flatland.String('name', label='Your Name', default='Anonymous'),
        flatland.String('email', label='Your Email Address',
            validators=[flatland.valid.email.IsEmail()]),
        flatland.String('comment', label='Your comment'),
    ]

Using the form in a view/controller is straightforward (in this example, request is a werkzeug.Request object).

def new_comment(request):
    if request.method == 'POST':
        form = CommentForm.from_flat(request.form)
        if form.validate():
            name = form.el('name').u
            comment = form.el('comment').u
            # Do comment saving stuff
            ...
    else:
        form = CommentForm.from_defaults()

    # Template rendering, etc...
The classmethod from_flat() is a helper method that takes a set of key, value pairs in flat namespace (such as application/x-www-form-urlencoded data) and maps it to a Flatland schema, resulting in an element object. from_defaults() instantiates the elements with their defaults.

One note: the IsEmail validator was added in the spring, and hasn't made it in to the main branch yet. It will be soon, and will be in the first release.

Most form libraries have either methods to convert to HTML input elements or corresponding widget objects, but Flatland is display agnostic. Included in the library is a set of output handlers for the Genshi template language, and there's some initial support for Jinja2 soon. In any case rendering is a bit more explicit than with other libraries, which in practice has not been a problem at all for us.

Now this is all fine and dandy, but it doesn't really justify a different library, perhaps. Where Flatland really shines, however, is in hierarchical data models with multiple validation levels. For example:

class ProfileForm(flatland.Form):
    schema = [
        flatland.String('name', label='Your Name',
            validators=[flatland.valid.Present()]),
        flatland.List('emails',
            flatland.String('email', validators=[flatland.valid.email.IsEmail()]),
            label='Contact Email Addresses'),
    ]

form = ProfileForm.from_flat([
    (u'name', u'Me!'),
    (u'emails_0_email', u'bob@bob.bob'),
    (u'emails_1_email', u'foo@bar.baz'),
] 

form.validate() # True
form.el('emails').value # [u'bob@bob.bob', u'foo@bar.baz']
This example has 0 to N email addresses, each required to pass the IsEmail() test. Flatland uses a two pass validation system. Each schema object determines the validation behavior on the descending path and the ascending. This offers a lot of flexibility; containers can validate at their level only after the children have executed their validators. This way the containers can rely that their children's values were either properly converted to the native Python type or are still set to None.

The same schema could be used to validate a JSON request.

import simplejson

input = simplejson.loads(
    '{"name": "Me!", "emails": ["bob@bob.bob", "noone@example.com"]}')
form = ProfileForm.from_value(input)

form.validate() # True
form.el('emails').value # [u'bob@bob.bob', u'noone@example.com']

For one last, more complex example, here is a custom schema and element for using the VidoopCAPTCHA:

vs = vidoopsecure.VidoopSecure(settings.VS_API_USER, settings.VS_API_PASS,
    settings.VS_CUSTOMER_ID, settings.VS_SITE_ID)


class VSCaptchaElement(flatland.schema.containers._DictElement):
    """Flatland element for captchas. Knows how to fetch a new captcha."""
    url = None
    categories = None

    def set_default(self):
        """Get a new VS captcha and fill our needed data values."""

        captcha_id, captcha_url, captcha_categories = vs.create_captcha(
            order_matters=False, width=3, height=3, image_code_color='White')
        self.el('id').set(captcha_id)
        self.url = captcha_url
        self.categories = captcha_categories


class VSCaptchaField(flatland.schema.containers.Dict):
    """Field that starts and validates a Vidoop Secure CAPTCHA"""

    element_type = VSCaptchaElement

    def __init__(self, name='vscaptcha', **kw):
        fields = [
            flatland.String("code"),
            flatland.String("id"),
        ]
        super(VSCaptchaField, self).__init__(name, *fields, **kw)

    def validate_element(self, element, state, descending):
        if descending:
            return None

        if not element.el('code').value:
            element.errors.append("This CAPTCHA code may not be blank.")
            return False

        try:
            success = vs.submit_captcha(element.el('id').u, element.el('code').u)
            return success is True
        except vidoopsecure.CaptchaAlreadyTried:
            element.errors.append("This CAPTCHA has aready beeen attempted. "
                "Try clicking your browser's refresh button.")
        except vidoopsecure.CaptchaExpired:
            element.errors.append("This CAPTCHA expired. "
                "Try clicking your browser's refresh button.")
        except vidoopsecure.CaptchaFailed:
            element.errors.append('Sorry, you entered the wrong code. '
                'Please try again.')

        return False
When a form that includes this field is created with from_defaults(), the elements are instantiated and then set_default() is called on each one. When that happens, the CAPTCHA service is called to generate a new image. On validation the VSCaptchaField first validates its children (when descending == True) and then uses the id and code unicode values to validate the input.

I've been extremely impressed with Flatland so far. While there's still work to be done fleshing out the standard fields, validators, and docs, the code base is heavily tested and and a pleasure to work with. Hopefully there will be a release soon; in the meantime check out the code on Bitbucket.

Updated 2008-02-24 10:21: Fixed typos

Writing my first Zine plugin

written by Adam Lowry, on Feb 22, 2009 6:00:00 PM.

It's been quite a long time since I decided I wanted to get back in to blogging. Part of my hesitation was my existing multi-user ancient installation of Drupal, which made me immensely angry every time I looked at it. Since I couldn't ditch it while other people were using it, it stayed while I toyed around with the idea of writing my own blog in Django or with the Werkzeug/Jinja2/SQLAlchemy combination I've been digging at work. While I was considering it, Armin Ronacher of Werkzeug fame went ahead and released one, Zine. Digging through the code reveals a gold mine of Python gems, and setting it up was easy (with the caveat that I run it on a VPS and know my way around Python and Apache).

My first attempt at a plugin is quite simple--it's just a couple of config options to declare OpenID delegation URLs and display them in the HTML head. If you'd like to take a look, the OpenID delegation plugin is on BitBucket. I tested with Zine 1.2. Once I clean it up some more I'll bundle it for installation on other blogs. And later, I plan on working with Ben to see about creating an OpenID relying party plugin.