Hooking your apps into Amazon Web Services

Unsure how to get started with AWS? Some sample code will illustrate its relative simplicity

Connecting your application into the Amazon Web Services (AWS) isn't complicated, particularly if you've done Web service programming on other projects.

Before you begin, you'll need to obtain access credentials. If you already have an account with Amazon, you can easily extend it to include access to Amazon Web Services. You sign up for each service separately and, in the process, acquire access credentials. These credentials are used to create digital signatures required in requests made to any of the paid AWS components. Amazon services also accept X.509 certificates. Amazon can create a certificate for you when you subscribe to a service, or you can create your own and upload it to Amazon. Once you have your access credentials settled, you can begin working with the services themselves.

Amazon's online developer materials appear to slightly favor Java over other languages. But only slightly. While all the command-line tools for EC2 are written in Java, AWS tutorials exist for Ruby, PHP, and C#. In addition, several scratchpad applications written in HTML + JavaScript provide "live" interaction with individual Amazon Web services.

[ For an in-depth exploration of AWS, please read "Diving deep into Amazon Web Services." ]

For example, download the SQS JavaScript scratchpad from the SQS developer site, launch it in a browser, and you're presented with what amounts to a control console into the SQS APIs. Enter your AWS AccessKey ID and AWS Secret Access Key (the credentials I mentioned above) into the appropriate text boxes, and you can create queues, list queues, send messages, receive messages, and more or less dance through the entire SQS API. Not only are the scratchpads helpful for working out the protocols used in AWS transactions, but prowling the source is an ideal way to learn the APIs.

In addition, developers looking for a leg up in building AWS-aware applications should investigate the plentiful open source library offerings linked to from the AWS Developer Connection site. JetS3T is a popular Java interface to the S3 service. Another open source Java library, typica, provides programming interfaces to EC2, SQS, SimpleDB, and DevPay. So before embarking on a roll-your-own approach, you should spend some time looking into the open source libraries available to verify that a trailblazer hasn't already created what you need.

This is precisely what I did and so discovered an open source Python library called "boto," named after an Amazon River dolphin, that simplifies working with EC2, S3, SQS, SimpleDB, and Mechanical Turk. (You can download boto.)

Because it is written in Python, boto allows on-the-fly experimentation. There are a few tutorials on the boto site, some of which are in raw text, but boto is a moderately new project -- about a year old -- and the projects' authors have done a remarkable job in that time. For creating an SQS queue, writing and reading a message, then deleting the message and its queue -- the example I wanted to build -- boto made the job fall-off-a-log easy.

Following is the source for my example SQS program. This code uses the free boto library, and demonstrates how to create, read, write, and delete an SQS message queue.

# Simple message queue test using

# the boto library

# see: http://code.google.com/boto # Import needed objects

from boto.sqs.connection import SQSConnection

from boto.sqs.message import Message # Establish a connection with SQS

# Put your IDs here

aws_access_id = ''

aws_secret_key = ''

conn = SQSConnection(aws_access_id, aws_secret_key) # Create a message queue

mq = conn.create_queue('InfoQueue')

# Create a message object and put some content in it

msg = Message()

msg.set_body('This is our first message to SQS') # Send the message -- report results

rset = mq.write(msg)

print 'Result of write: ' + rset.StatusCode # Now loop until message(s) appear in the queue

while 1

 rset = mq.get_messages()

 if len(rset) != 0 : break

# Print out the messages

# Delete each message after printing

for msg in rset:

 print msg.get_body()

 mq.delete_message(msg) # All done ... delete the queue


print 'Done'

The code is about as straightforward as it gets. I only needed to import two classes from the boto library: SQSConnection and Message. The instantiation of an SQSConnection object required two arguments: an AWS access ID and an AWS secret key. If you don't want to put your ID and key directly in the code, a variant of the SQSConnection constructor (with no arguments) will read the information from a pair of environment variables.

Once the connection object is established, one can create a queue with a call to the method create_queue("<queue name>"). If the queue already exists, the object returned represents the existing queue. Next, I instantiated a Message object and wrote the message content via the set_body() method. A call to mq.write(msg) sends the message on its way.

The method get_messages() reads all messages on the queue and returns them in a list object. I can use a while: loop to wait for my sent message to show up, then step through the returned list, printing each message's body and deleting it from the queue. (Remember, reading a message from a queue does not delete it. The message is locked for a predetermined period of time.) Finally, I delete the queue itself.

The code is simple, in keeping with AWS's central tenet, but all the fundamentals of interacting with SQS are present. The boto library -- and the Python language -- make it a snap to test an idea with "live" code, then move the results into a project once you've confirmed that your routines are working correctly.