Amazon Relational Database Service (Amazon RDS) is a web service that makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while managing time-consuming database administration tasks, freeing you up to focus on your applications and business.

Amazon RDS gives you access to the full capabilities of a familiar MySQL database. This means the code, applications, and tools you already use today with your existing MySQL databases work seamlessly with Amazon RDS. Amazon RDS automatically patches the database software and backs up your database, storing the backups for a user-defined retention period. You also benefit from the flexibility of being able to scale the compute resources or storage capacity associated with your relational database instance via a single API call. As with all Amazon Web Services, there are no up-front investments required, and you pay only for the resources you use.

This is pretty slick. I haven’t played with it yet as it was just announced but it seems to be an API driven mysql instance. For slightly more than a base instance, 0.11/hour RDS vs 0.10/hour base EC2 (this price is dropping 15% BTW) on a small server, you get a complete server with MySQL installed. You can create and manage your database instances via procedural call (the API) and you can scale to larger instances or additional storage fairly painlessly by also using those APIs. You also pay extra for your storage of course.

That’s about it from what I’ve read. I don’t see any automated replication (beyond the normal AWS safety features) and I don’t see any kind of clustering or sharding. This is not what most people would call a cloud database. It’s just an easy to configure, maintain and grow MySQL server. Not that that’s bad. For a small business with some technical savvy but not a lot of time, this is an awesome addition to AWS. I would be willing to bet that some kind of clustering will come, sooner or later.

Ooops, just stumbled across:

Coming Soon: High Availability Offering — For developers and business who want additional resilience beyond the automated backups provided by Amazon RDS at no additional charge. With the high availability offer, developers and business can easily and cost-effectively provision synchronously replicated DB Instances in multiple availability zones (AZ’s), to protect against failure within a single location.

One of the things I have always liked about AWS is that they really do make it simple. For the uses cases where SimpleDB is appropriate, using it is a no brainer, as is EC2 and S3. AWS even makes queuing simple. RDS keeps to that methodology.

Amazon RDS allows you to use a simple set of web services APIs to create, delete and modify relational database instances (DB Instances). You can also use the APIs to control access and security for your instance(s) and manage your database backups and snapshots. For a full list of the available Amazon RDS APIs, please see the Amazon RDS API Guide. Some of the most commonly used APIs and their functionality are listed below:

CreateDBInstance — Provision a new DB Instance, specifying DB Instance class, storage capacity and the backup retention policy you wish to use. This one API call is all that’s needed to give you access to a running MySQL database, with the software pre-installed and the available resource capacity you request.

ModifyDBInstance — Modify settings for a running DB Instance. This lets you use a single API call to scale the resources available to your DB Instance in response to the load on your database, or change how it is automatically backed up and maintained on your behalf.

DeleteDBInstance — Delete a running DB Instance. With Amazon RDS, you can terminate your DB Instance at any time and pay only for the resources you used.

CreateDBSnapshot — Generate a snapshot of your DB Instance. You can restore your DB Instance to these user-created snapshots at any point, even to reinstate a previously deleted DB Instance.

RestoreDBInstanceToPointInTIme — Create a new DB Instance from a point-in-time backup. You can restore to any point within the retention period you specified, usually up to the last five minutes of your database’s usage.

This is a very cool addition to AWS. I am looking forward to playing with it. It’s important to note that if you are capable of administering your own server and database, you can save money by running a base EC2 instance and DIY. If you want to run any database other than MySQL, you have to do that anyway.

LewisC

I personally believe that AWS is perfect for any development and testing environment. Regardless of how sensitive your data is, you can build your applications and test them in a cloud environment using bogus data.

For production environments, the choice is much harder. Does the country(ies) you operate in have strict privacy, or data on-shoring, laws that would be impact your applications? If you can easily offshore your applications, you can easily use cloud computing.

Does the area where you work have reliable infrastructure? It doesn’t matter if Amazon has 99.99% uptime if your provider is down 50% of the time. You can easily use something like replication and keep a copy of your application’s data within your own data center but if you make that investment, do you really want to run anything in the cloud.

My suggestion to get started would be to use AWS to host a development effort first. Get comfortable with the quirks and gotchas of remote applications. Familiarize yourself with the additional security you will need when running in the cloud. Look at encrypting your data on disk. Amazon will encrypt the data as it travels over the wire.

The need for system administrators and DBAs does not go away by moving to the cloud. It really doesn’t change their jobs much at all. Most modern admins rarely touch the hardware directly anymore, anyway.

Once you’ve decided that it is for you and you have chosen your pilot project, you will need to take the actions described below.

A note to remember as you are working through this book. You only pay for what you use. When you run an instance, you pay for the CPU time that you use. When you use S3 or EBS, you pay for storage (and bandwidth in S3). You pay for Elastic IPs only if you allocate one and don’t attach it to a running instance.

Technorati : amazon, aws, cdn, cloud computing, data, web service

SimpleDB was Amazon’s first available (in beta) web service. It is a neat feature but it has its downsides. First, SimpleDB is not a relational database. It is a name/value key pair. For simple lookups, it is highly reliable and scalable. For anything more complicated, it falls apart.

SimpleDB is not ACID compliant and has not referential integrity. For that matter, it has not schemas, tables or relationships. Amazon says that it “eliminates the administrative burden of data modeling”. Some things make me say, “Hmmmmm.”

SimpleDB structures data somewhat like a spreadsheet. Think of columns across and values down. A particular column can have multiple values. I provide an example of SimpleDB data in Chapter 6.

Like everything else in AWS, SimpleDB is API based. There is no SQL access here. The APIs are very simple to use: CREATE creates a new domain (worksheet), you can GET, PUT and DELETE items (columns) and values (data), QUERY data or QUERYWITHATTRIBUTES (meta data).

Amazon does have a query language but it is strictly string based. You enter a key value (a key being the name of one of your key/value pairs) and then list possible values. There are simple operators that you can use.

SimpleDB is designed to store small volumes of data and is optimized for that. Amazon recommends that large files be stored in S3 and the pointer to those files stored in SimpleDB.

Cost

You pay for three things with SimpleDB: machine usage (executing queries), data transfer and persistent storage.

Machine usage is based on the requests made and the amount of time it takes to satisfy those requests. The CPU is based on the same criteria as an EC2 compute unit. It costs $0.14 per machine hour utilized. You start with 25 machine hours for free and start paying at the 26th hour.

Persistent storage was $1.50 per GB until Dec 2008. That was much more expensive than S3 or EBS. In late 2008, Amazon lowered the costs to a more reasonable $0.25 per GB. That is a significant change.

Data transfer is comparable to the other services: Data transfer in is $0.10 per GB, first 10TB out is $0.17, $0.13 for the next 40TB, $0.11 for the next 100TB and $0.10 for all data over 150TB.

For a limited time, at least until June 2009, the first 25 CPU hours and 1GB per month are free. This is designed to give people a chance to try out the service.

As a database guy, SimpleDB is a non-starter for me. It’s easy enough for me to install MySQL or Postgres (for free) or Oracle (if I want to pay for it) and scale those to almost ridiculous levels. SimpleDB does not provide the transactional consistency required for transaction processing (OLTP) not does it provide the access paths or any of the key features (except maybe partitioning) required in OLAP processing.

These prices are accurate as of the time of writing them. As always, verify before making a decision.

Technorati : amazon, aws, cloud computing, database, simpledb, web service

What is CouchDB?

CouchDB is an Apache project. CouchDB is not a relational database. It seems that cloud computing has spawned, or at least made popular, a new breed of database. Rather than the hierarchical, network or relational databases of yore*, we have a new paradigm: key/value pairs. You declare a field and assign some values.

*I left object database and xml database off my database of yore list as they never really caught on.

SimpleDB is another key/value database that you may have heard of or used. SimpleDB is provided as part of Amazon Web Services (AWS).

What does CouchDB Offer?

CouchDB is accessible via JSON (which I like better than XML for tasks like these) and it uses JavaScript as a query language. CouchDB is document aware. That is, you create a new document and store related data wiithin that document. There is no schema, documents are the important classification of your data..

The really important thing is that CouchDB is highly distributed. It’s this feature that makes it desireable in situations where a relational database does not scale well. According to the Apache CouchDB Documentation:

CouchDB is a peer based distributed database system. Any number of CouchDB hosts (servers and offline-clients) can have independent “replica copies” of the same database, where applications have full database interactivity (query, add, edit, delete). When back online or on a schedule, database changes are replicated bi-directionally.

CouchDB has built-in conflict detection and management and the replication process is incremental and fast, copying only documents and individual fields changed since the previous replication. Most applications require no special planning to take advantage of distributed updates and replication.

Distributed from the ground up. Sweet.

An important note about where CouchDB is different from SimpleDB is that CouchDB is ACID. Rather than using logs for consitency, CouchDB uses redundants sets of data (much like Vertica). CouchDB, like the other key/value databases is “eventually consistent“. That means that it will take time for the replicas to be updated. CouchDB also uses MVCC and readers never block writers. Readers always see a consistent data set.

CouchDB is written in Erlang. That’s a down side to me in that it is not a very common language. If you do need a patch in a hurry, it may be difficult to find someone qualified to write it. CouchDB was originally written in C++ but the author chose to redo it in Erlang for scalability reasons. Hmmm.

That’s the short story on CouchDB. I plan to write more about actually using CouchDB in the near future.

LewisC

Technorati : apache, cloud computing, couchdb, database

An overview of cloud computing using Amazon Web Services (AWS). Also provides info on the business value of running in the cloud, what Oracle Corp provides for running in the cloud and a walk through of starting an EC2 instance and attaching an EBS volume.

Cloud Computing Using Oracle and Amazon Web Services