Cloud Computing Info

Mechanical Turk

Mechanical Turk is an odd service. It’s called an “on-demand workforce” or peopleware. For large tasks that need to be automated but also require human intelligence, Mechanical Turk is the tool.

One of the examples Amazon uses is if you have 1,000,000 (one million) images that need to be tagged and categorized, you can use Mechanical Turk to “hire” 10,000 employees. You get to pick what you will pay and only those “turks” who want the work will sign up.

Amazon picks up 10% (additive) to whatever you pay someone.

Amazon FWS

FWS is the Amazon Fulfillment Service.

SimpleDB

SimpleDB was Amazon’s first available (in beta) web service. It is a neat feature but it has its downsides. First, SimpleDB is not a relational database. It is a name/value key pair. For simple lookups, it is highly reliable and scalable. For anything more complicated, it falls apart.

SimpleDB is not ACID compliant and has not referential integrity. For that matter, it has not schemas, tables or relationships. Amazon says that it “eliminates the administrative burden of data modeling”. Some things make me say, “Hmmmmm.”

SimpleDB structures data somewhat like a spreadsheet. Think of columns across and values down.

CloudFront

Amazon CloudFront is Amazon’s Content Delivery Network (CDN). A CDN puts very large servers with high throughput at the edge of the network. That means that a CDN provider put cached data in multiple locations through out the network (internet). Requests for data are routed to a local server cache instead of the main server at a host. This improves performance, customer experience and possibly even costs (via lower bandwidth requirements).

An example would be a company that serves many pages to many users. Rather than have all of the pages stored in a central location and be accessed by many people all at once, the pages are distributed throughout the network and sit on many different servers.

Amazon Flexible Payments

Amazon Flexible Payments Service (FPS) is a set of web services that allow businesses or developers to bill users using the Amazon payment infrastructure (like a PayPal or Google Checkout). As a seller or a buyer, you can set limits on usage either globally or for specific senders and/or receivers. A gatekeeper component enforces the rules.

As a sender you can limit the number of transactions, transaction dates, dollar amounts, recipients and daily, weekly or monthly spending limits. Recipients can specify all of those and can specify allowable payment methods (credit card, bank transfer and amazon payments) and who pays the transaction fee.

One of the goals of FPS is to make micropayments effective and financially cost effective.

Simple Queue Service (SQS)

SQS is Amazon’s message queuing service. It works much like IBM’s MQ Series, JMS or Oracle AQ. Pop in a message and one or more recipients can pop it out. SQS is completely open so any internet connected computer can call a web service and add or remove a message.

Because SQS is API based, you can write an interface to it in the language of your choice. There are several free Java, Ruby and PHP interfaces available (that I know of) with more coming.

Elastic Compute Cloud (EC2)

Elastic Block Storage (EBS)

For most of its life in beta, EC2 offered only two kinds of storage, AMI based transient storage and S3. The transient storage was mounted as a filesystem and S3 was used for backup. To save data during downtime for instances, data had to first be saved off to S3 and the instance brought down. When the instance was brought back up, data was restored from S3. It was a painful process.

Enter EBS, the Elastic Block Store.

Simple Storage Service (S3)

Cost

Storage is cheaper in the US than in Europe. If you are based in Europe, you may want to decide which is more important when getting or adding data: price or latency.

Table 3: S3 Storage Costs

Table 4: S3 Data Transfer Costs

Table 5: S3 Request Costs

These prices are accurate as of the time of writing them.

I just got an email from Amazon Web Services.  In honor of their 3 year anniversary, they are offering 3 cents per GB data transfer (that’s external transfer) instead of the normal 10 cents per GB.  This is planned to last for 3 months.  If this was IN and OUT, this would be a significant savings for companies using S3 to serve up large files.  Still, while not as big as it could be, it does mean that this is the time to get all of your files loaded up.

Simple Storage Service (S3)

Security

Write and delete access to buckets and objects is controlled via Access Control Lists (ACL). You can assign read permissions to any object to specific users. You can also make an object public to grant access to anyone.

Transfer into and out of S3 can utilize SSH which will encrypt data. This prevents any “over the wire” interception of your data. Data at rest is not encrypted and Amazon recommends that users encrypt any sensitive data with their encryption tool of choice.

Simple Storage Service (S3)

The AWS S3 service is an API driven storage service. The API provides get, put and delete. Data is stored using a bucket concept that is not unlike directories and sub-directories. A bucket can hold one or more buckets and one or more objects (i.e. files). You can nest buckets as many levels deep as required by your application or other needs. Objects can be up to 5GB per and you can store an unlimited number of objects.

At the top level is a global bucket. All S3 accounts share the global bucket.

Sizing and Costs

EC2, like the other services in AWS are pay as you go, pay for what you use, services. As I mentioned above, you basically pay for the power you use which is a CPU per hour charge, bandwidth and storage. Linux and Windows guests have a different pricing menu. I am listing the prices current as of Dec 2008. I recommend you always check at aws.amazon.com to verify current pricing before making a commitment.

Instead of buying or leasing a specific type of hardware (that you would then be responsible for upgrading over time), AWS computing power is based on an EC2 compute unit.

Have you been wondering what the differences between S3 and EBS are? I recently gave a high level overview of S3 and I plan to do one on EBS. I also plan to follow with a detailed looked at both S3 and EBS.

In the meantime, Cloudiquity has posted an entry, Differences between S3 and EBS. This is a nice overview. It provides some excellent technical details as well as some pricing info. Well worth a read.

LewisC

Technorati : amazon, aws, ebs, s3, storage