Archive for April, 2009

Using and Managing AWS – Part 2: Signup for AWS

April 30th, 2009 Comments off

Sign Up For AWS

First things first, if you don’t have an account, go get one. If you do have one, you can use the one you already have. Amazon offers personal and corporate accounts. A person may have both accounts and can choose which to use when purchasing items.

It also may make sense that all employees have a business only account that uses their work email to log into the service. That way you never have an issue where purchases or billing can go to the wrong place.

Or, you may do like I have done in the past, put all expenses on a personal card and expense them back to the company. It’s a little bit more paperwork, but it keeps it open to what already works for both you and your employees.

If you are a developer, the choice is a bit easier, just sign up using whatever card you like.

Once you have your Amazon account, you will need to sign up for AWS. Currently, the menu option to do this is in the menu structure at the left of the screen, below Business Services. It’s called For Developers.


Don’t get stuck on the particular menu item. It will change over time, I’m sure. You can get to the right page by navigating to AWS.AMAZON.COM. From this screen, choose Sign Up Now or Create an Account.

Once you have chosen to sign up, you will need to enter several pages on information. These are very common sense questions like: your name, your email address, your company name and web site, and some credit card information. Your credit card will be billed monthly for the amount of services used. If you don’t use any services, you shouldn’t get billed.

You will also have to agree to the Amazon AWS agreement. This is a multi-page agreement that you must agree to before you can use the services. I’m not a lawyer so I won’t council you on this. This is a legally binding contract though, so you may want your lawyer to take a look at it beforehand.

You can navigate around the AWS pages on via the menu on the right side of the screen (which is not available on all the screens) or via the top, drop down menu.


Once you are signed up, you can choose the various services that you want to use. At a minimum, you should sign up for EC2, S3 and SQS.

Technorati : , , , , ,

Amazon Web Services – Amazon DevPay

April 29th, 2009 2 comments

Amazon DevPay

Amazon DevPay is an easy to use billing system for AWS developers. Build your cloud application, allow users to sign up and use your application and let Amazon bill them for you.

DevPay is “AWS-Aware” in that it ties into the billing of AWS services. Instead of a user having to sign up for AWS and be billed separately, you can add in the AWS costs to your costs and just bill the users directly.

DevPay is web based and uses Amazon Payments. The web interface allows you to register your application and set your pricing. You can then link to your own web site and your customers will be unaware that they are paying for Amazon services (unless you want them to know).

DevPay allows you to bill a one time charge, a recurring charge (subscription) and metered usage (bill by the hour and/or storage).

One of the nicest things about it is that you are somewhat protected from deadbeats. You won’t get any money back for usage of your product, but Amazon does not try to collect their money from you unless they are able to collect from the customer.


Amazon charges 3.0% of the amount you charge minus any metered usage. That means that if the user uses $1.50 of metered CPU and you bill them $5.00, Amazon will only charge you 3% of $3.50.

Amazon also charges $0.30 for each bill collected from a customer.

Technorati : , , , , , , , ,

Using and Managing Amazon Web Services (AWS) – Part 1

April 28th, 2009 Comments off

Using and Managing Amazon Web Services (AWS)

I personally believe that AWS is perfect for any development and testing environment. Regardless of how sensitive your data is, you can build your applications and test them in a cloud environment using bogus data.

For production environments, the choice is much harder. Does the country(ies) you operate in have strict privacy, or data on-shoring, laws that would be impact your applications? If you can easily offshore your applications, you can easily use cloud computing.

Does the area where you work have reliable infrastructure? It doesn’t matter if Amazon has 99.99% uptime if your provider is down 50% of the time. You can easily use something like replication and keep a copy of your application’s data within your own data center but if you make that investment, do you really want to run anything in the cloud.

My suggestion to get started would be to use AWS to host a development effort first. Get comfortable with the quirks and gotchas of remote applications. Familiarize yourself with the additional security you will need when running in the cloud. Look at encrypting your data on disk. Amazon will encrypt the data as it travels over the wire.

The need for system administrators and DBAs does not go away by moving to the cloud. It really doesn’t change their jobs much at all. Most modern admins rarely touch the hardware directly anymore, anyway.

Once you’ve decided that it is for you and you have chosen your pilot project, you will need to take the actions described below.

A note to remember as you are working through this book. You only pay for what you use. When you run an instance, you pay for the CPU time that you use. When you use S3 or EBS, you pay for storage (and bandwidth in S3). You pay for Elastic IPs only if you allocate one and don’t attach it to a running instance.

Technorati : , , , , ,

Amazon Web Services – Mechanical Turk and Amazon FWS

April 24th, 2009 Comments off

Mechanical Turk

Mechanical Turk is an odd service. It’s called an “on-demand workforce” or peopleware. For large tasks that need to be automated but also require human intelligence, Mechanical Turk is the tool.

One of the examples Amazon uses is if you have 1,000,000 (one million) images that need to be tagged and categorized, you can use Mechanical Turk to “hire” 10,000 employees. You get to pick what you will pay and only those “turks” who want the work will sign up.

Amazon picks up 10% (additive) to whatever you pay someone.

Amazon FWS

FWS is the Amazon Fulfillment Service. This is another odd web service. Instead of using humans, this service has a warehousing component.

The short story is that Amazon has a world class fulfillment capability (i.e. ordering, packing and shipping products). FWS allows a business to tap into that by staging inventory in Amazon fulfillment centers.

Customers can order your products via or any other online or retail sales channel and your products will be packed and shipped from that center.

Actually storing your product at Amazon is called Fulfillment by Amazon and you pay to store your inventory. FWS is the web interface into the fulfillment center to allow you to programmatically send fulfillment requests from your web site or another merchant.

FWS (the API to make the requests) is free. FBA (the actual inventory part of it) is charged by the cubic foot of storage and how it is packed and shipped.

Technorati : , , , , , , ,

Amazon Web Services – SimpleDB Overview

April 22nd, 2009 1 comment


SimpleDB was Amazon’s first available (in beta) web service. It is a neat feature but it has its downsides. First, SimpleDB is not a relational database. It is a name/value key pair. For simple lookups, it is highly reliable and scalable. For anything more complicated, it falls apart.

SimpleDB is not ACID compliant and has not referential integrity. For that matter, it has not schemas, tables or relationships. Amazon says that it “eliminates the administrative burden of data modeling”. Some things make me say, “Hmmmmm.”

SimpleDB structures data somewhat like a spreadsheet. Think of columns across and values down. A particular column can have multiple values. I provide an example of SimpleDB data in Chapter 6.

Like everything else in AWS, SimpleDB is API based. There is no SQL access here. The APIs are very simple to use: CREATE creates a new domain (worksheet), you can GET, PUT and DELETE items (columns) and values (data), QUERY data or QUERYWITHATTRIBUTES (meta data).

Amazon does have a query language but it is strictly string based. You enter a key value (a key being the name of one of your key/value pairs) and then list possible values. There are simple operators that you can use.

SimpleDB is designed to store small volumes of data and is optimized for that. Amazon recommends that large files be stored in S3 and the pointer to those files stored in SimpleDB.


You pay for three things with SimpleDB: machine usage (executing queries), data transfer and persistent storage.

Machine usage is based on the requests made and the amount of time it takes to satisfy those requests. The CPU is based on the same criteria as an EC2 compute unit. It costs $0.14 per machine hour utilized. You start with 25 machine hours for free and start paying at the 26th hour.

Persistent storage was $1.50 per GB until Dec 2008. That was much more expensive than S3 or EBS. In late 2008, Amazon lowered the costs to a more reasonable $0.25 per GB. That is a significant change.

Data transfer is comparable to the other services: Data transfer in is $0.10 per GB, first 10TB out is $0.17, $0.13 for the next 40TB, $0.11 for the next 100TB and $0.10 for all data over 150TB.

For a limited time, at least until June 2009, the first 25 CPU hours and 1GB per month are free. This is designed to give people a chance to try out the service.

As a database guy, SimpleDB is a non-starter for me. It’s easy enough for me to install MySQL or Postgres (for free) or Oracle (if I want to pay for it) and scale those to almost ridiculous levels. SimpleDB does not provide the transactional consistency required for transaction processing (OLTP) not does it provide the access paths or any of the key features (except maybe partitioning) required in OLAP processing.

These prices are accurate as of the time of writing them. As always, verify before making a decision.

Technorati : , , , , ,