Generate Your Keys

Now that you have chosen your instance, but before starting you actually start your instance, you need to generate your key pairs. The keypairs are SSH keypairs. A later post will explain SSH in greater detail but the keys come in a pair because there is both public and private components.

SSH is a Secure SHell. This is a command prompt like a DOS box or a telnet connection. However, unlike DOS and Telnet, it is very secure. The private key is the local machine’s secret password. The public key is shared to any host that the local machine will connect to.

The host is able to create a query after seeing the public key that only someone with the private key could answer. The private key is never shared but the host is convinced that it is talking to the person (or machine) that is says it is.

This may sound confusing but it is actually very secure. It’s is much better than passwords that can be hacked or accidentally given away.

Amazon supports SSH and secure communications out of the box. If you choose to revert to simple protocols such as telnet and ftp and to password authentication, you may do so. However, your first connection to any instance started through AWS will have to be via SSH. Amazon makes it easy to be secure but gives you the option of making it less secure.

So at least one pair of keys needs to be generated. Each tool set that you choose will create the files in a different way. If you are running the command line tools, you will run the ec2-add-keypair program. If running ElasticFox or CloudStudio, you will have a button on the GUI. However you create the keypair, the end result is that you will end up with a file that tends in a .pem format.

When running SSH (and the tools) from a Windows client, you will need to convert the .pem file to a PuTTY formatted key file. PuTTY, like SSH will be documented in greater detail in a near future post. Review that post for tips on Converting SSH to PuTTY.

You choose an instance’s keypair when you start it and you cannot change that after it is running. Generate your key pair and getting working first.

Choose an AMI

Amazon, and Amazon clients, are providing a huge variation of machine images. The short story is that you can choose between MS-Windows, Linux and Sun Solaris for your OS. The real story is that it is a bit more complicated than that.

The real question is what applications do you plan to run and what expertise do you have on hand or plan to hire? A quick example is a database like MySQL. MySQL runs on various operating systems. If you have Windows expertise, you may want to stick with windows. On the other hand, you can run some Linux instances with MySQL pre-installed and configured.

This about the stack that you want to run. I generally run Linux instances. They are a few cents cheaper per CPU hour and I am good enough with Linux that it doesn’t cause me any issues. I can run Oracle, MySQL and Postgres side-by-side. I occasionally do run Windows instances though just to compare offerings.

If you run SQL Server, you will need to run Windows. Almost any other software stack offers an option of OS. If you do run Windows, you will be running Windows Server 2003, in either 32 or 64 bit. SQL Server can be the Express Edition or the full blown commercial edition (which costs extra for licensing).

If you want to run Solaris, you currently have to register with Sun to get access to the OpenSolaris instance. It’s free but it requires registration. With OpenSolaris you get DTrace and ZFS, two selling points for many people.

You get OpenSolaris 2008.05 or Solaris Community Edition and pricing is the same as a Linux install. You can AMIs with AMP preinstalled as well as stacks like Drupal and MySQL.

For Linux installs, the choices are almost limitless: Fedora, CentOS, Ubuntu, Oracle Unbreakable Linux, RedHat. You name it, it’s probably there. Many of these come with pre-installed software stacks. No download and configure, just run.

Plan to try many instance types. You may even end up with a RightScale instance.

This post will (attempt) to explain what SSH and PuTTY are so that as a user you understand the terminology of AWS and so that you can be productive in the environment. This post will not attempt to make you an expert in SSH. For best practices in implementing SSH, I strongly recommend a book dedicated to hardening *nix (Linux, Unix, Solaris, etc).

SSH

In the early days, not that long ago really, of networking, very simple tools were used to work with remote computers: telnet as a console, ftp for file copying, rsh for remote command execution and others. These were easy to configure and use tools. They were client server in that a software component needed to run on both the local machine (client) and the remote machine (server).

While easy to use, they were very insecure. They made no pretense at verifying that the calling host really was the calling host. Everything was username/password based and both the username and the password were passed around the network in cleartext. If you intercepted the little data packages that were being routed around the network (with a sniffer for example), you would be able to extract the login credentials. Even if you encrypted all of your data, your credentials were still in the clear.

SSH is an attempt (quite successful) to fix those insecurities without making things anymore complex than they need to be. SSH stands for Secure SHell. However, SSH is not really a command shell, it is rather a protocol that encrypts communications. That means that programs that use SSH can work like telnet or ftp but will be more secure.

Note: Technically, SSH is also a tool. There is a client terminal program called SSH. It’s a non-graphical command line tool that provides a window which executes a command shell on the remote system.

SSH offers multiple modes of connecting but for the purposes of AWS, we will talk about key based access. To make things more secure, EC2 uses a key based authentication. Before starting an instance, you need to create a key pair.

Note: The below explanation of SSH is a gross over simplification. I am just trying to give you a feel for what is going on. If you really want to understand the technical details, I really do recommend that you purchase a book. My personal recommendation is SSH, The Secure Shell: The Definitive Guide from O’Reilly.

When an instance starts up for the first time, EC2 copies the ssh key that you created to the proper directory on the remote server. The remote server will be running the SSH Server software.

You will then use an SSH client to connect to the server. The client will ask for some information proving that the server really is who it says it is. The first time you connect to a server, the client won’t have that information available so it will prompt you to vertify that the server is legitimate.

You verify that information by comparing a thumbprint. Verifying a host is a bit beyond this book but do an internet search for for “ssh host thumbprint”. You’ll find a variety of articles explaining it in detail.

Once the client accepts the host, the client will send secret information to the host. This is your key data. If the host is able to make a match, it will authenticate you and let you login in. If the host then asks for a password, you key did not work and something is not configured properly. In my experience, it will probably be that your client key file is not in the place your client is expecting it to be.

What happens next depends on the tool you are using. If you are using a terminal program, ssh for example, you will now have a command prompt. If you are using sftp or scp, you will be able to copy files.

In addition to command line tools, there are GUI tools that use the SSH protocol. WinSCP is an excellent SCP client for Windows.

Regardless of the tools you use, SSH is busy encrypting everything you send over the wire. The SSH protocol has evolved over the years, and will probably evolve even more in the future, but it is currently running a very secure form of encryption.

If you are running Linux, you are pretty much finished at this point. SSH ships with every Linux distribution that I am aware of. If you are using Windows, however, you either need to install CyWin (a unix environment that runs in windows), or you’ll want to get PuTTY.

PuTTY

You can download all of the programs discussed in this section at:

http://www.chiark.greenend.org.uk/~sgtatham/putty/

I honestly have no idea why PuTTY is spelled PuTTY. I can figure the TTY part of it is from the Unix command that output a display. I’m not sure bout the Pu though.

I do know what PuTTY is though. PuTTY is a very simple implementation of an MS-Windows SSH terminal client. When I say it is simple, I mean that as a complement. This is a tool that does not get in the way.

You tell PuTTY to connect to a remote server and, as long as your keys are configured, it will connect you. If are not using keys, you can connect with passwords (if the host allows that). As a best practice, keys are recommends over passwords.

PuTTY is the terminal client but you can get a couple of other tools from the same author. PSFTP and PSCP offer secure file transfers. These tools are as easy to use as PuTTY and work pretty much the same way.

For command line syntax and configuration, take a look at the documentation at the link above.

A note about SSH keys and PuTTY, they are not compatible. This same web site offers a utility called PuTTYgen. When you create a key pair for EC2, you download that file to your local machine. PuTTYgen converts that file (a .pem file) to a private key file (a .ppk file).

PuTTY Key Generator

The tool is named puttygen.exe. Run the executable and the above window pops up. To convert an amazon key to a PuTTY key, use the menu option Conversions ? Import Key. Load the .pem file that you downloaded and press the Save Private Key button.

It will warn you about leaving the passphrase blank. That’s ok.

Save the file to the location that PuTTY has been configured to look in for it’s keys.

Data Center Security

Amazon is a well known entity and works to provide an extremely secure environment for your applications ans your data. Amazon is pursuing Sabanes-Oxley certification (by an external auditing agency) and SAS-70 Type II certification.

Amazon does not broadcast the locations of their data centers and physical security is a top concern for them. They have military grade external protections. Physical access to Amazon data centers controlled by a two-factor authentication and only those Amazon employees with an actual need are ever given access.

Hardware access is provided only to those administrators who directly require it and they must use their own SSH keys to access bastion hosts (kind of like cloud overseers). They can then escalate access to gain access to individual client hosts. All administrator access is logged and audited.

The network is monitored by Amazon security services. Due to Amazon IP security, an EC2 instance cannot spoof an IP address. An instance is not allowed to send traffic with a spoofed address. Also, Amazon monitors for port scanning. If they find port scanning, they block the incoming address.

Because all clients are running in virtual servers with virtual storage, there is no way for one client to gain access to another clients data or traffic. For all intents and purposes, each client is running in their own data center.

Data Security

Your data is secured when traveling over the wire by SSL. You can chose less secure methods once you have an image up and running but by default, an AMI will be very secure. If you choose to open your firewall (security group) to any and all traffic, you will be open to hacking. If you chose to use password security instead of SSH keys, you take your own risks.

There are several additional steps you can take to protect your data.

• Only present web servers to the internet. You have the option of not having a public IP address on every instance. If you have amulti-tier application, you can choose to have a public IP address on your web server and have just an internal IP address on your database server. To access the database server, you would have to log into the web server and then ssh from there to the databaseserver.

• Another option is to encrypt all of your stored data (or at least the sensitive portions of it). Amazon offers Linux, Windows and Sunvirtual machines and all of these operating systems offer very robust (at least via third party tools) encryption. A very good, freeoption on windows servers is TrueCrypt.

Data being stored in AWS applications (S3, SimpleDB and EBS) is automatically, redundantly stored in multiple physical locations. You do not pay for this additional storage. Amazon does this to ensure the integrity of your data (and that they meet their SLAs).

Yet another option is to use the encryption capabilities offered by the various databases that you might be using. Oracle provides Transparent Data Encryption for data at rest and offers Oracle Secure Backup via RMAN. Using Oracle Secure backup with the Cloud Module extension will allow you to encrypt your back ups and store your data on S3.

Authentication

AWS allows two different methods of authentication. When you submit a request, be it to create a new instance in EC2 or to upload a file to S3, AWS needs to know that you are allowed to to submit the request that you are submitting.

AWS recognizes two different types of request identifiers: a secret key or an x.509 certificate. The x.509 certificate can only be used with SOAP transactions and can only be used with certain EC2 and SQS requests. The secret key method can be used with all of the services and for all of the request types. For that reason, I will assume you have chosen to use the secret key method and that is the method I will be using here on the blog.

Amazon allows you to regenerate your key any time you decide that you need to. Remember that your access keys are what you will use to access AWS via any third party software or external vendors (such as RightScale).

To get to the Access Identifiers, choose Your Account ? Access identifiers from the menu shown in Image 2 above. This screen will allow you to generate a new secret access key and a new x.509 security certificate.

Your access key does not change and is included on all requests to AWS. You can think of your access key as your username. Think of you secret key as your password. You can change your secret key at any time but your access key stays the same all the time.

Amazon notes on the page that you must protect your secret key and never email it to anyone. You will need to give it away under certain conditions though. When you use a third party tool like ElasticFox or Cloud Studio, you need to enter your credentials. You will also need to give your secret key to a third party vendor like RightScale who will issue requests on your behalf.

AWS Access identifiers

You can see your secret key by clicking the “+ Show” link. You will come to this screen to get your access keys. When entering the values into other tools, use cut and paste to do so. When you paste, paste it first into notepad or a VI session. For some reason, when cutting this data, it usually has several extra spaces at the end that will prevent it from working when you paste.

To generate a new key, just press the Generate button. Because this key is like a password, you may want to regenerate the key on aregular basis. If you are sharing this key at all, you will need to make sure you update anyone who has it.

]]> http://clouddb.info/2009/05/17/using-and-managing-aws-part-3-aws-security/feed/ 2 http://clouddb.info/2009/02/11/defining-cloud-computing-part-3-saas/ http://clouddb.info/2009/02/11/defining-cloud-computing-part-3-saas/#comments Wed, 11 Feb 2009 17:01:51 +0000 Lew http://clouddb.info/?p=69 Software as a Service (SaaS)

SaaS is currently the most popular type of cloud computing. Yahoo email, Google apps, zoho, and various other packages like CRM are all instances of SaaS. Application Service Providers (ASP) were the first SaaS providers. ASP was its own buzzword back in the late 1990s and early 2000s.

One of the aspects of SaaS is multi-tenancy or the ability for many customers to share the same service but maintain their own data securely. CRM is the predominant paid SaaS offering but email is, by far, the predominant free SaaS offering.

Any software that is offered over the internet, that runs remotely (where the location is unimportant and unrelated to the user), is a SaaS offering. With SaaS, a user has no need to worry about installation or upgrades. Of course, that also means the user has no control over versioning (or bug fixes).

The majority of SaaS offering are either free (think web mail) or per seat subscriptions (like online collaboration and CRM). SaaS may also be offered as a subscription for access and then charge fees for extras like additional storage.

SaaS vendors typically offer multiple levels of access (usually as subscriptions) and functionality. Many offer free trials or a free tier for trying out the software. If you are going to buy into a SaaS offering, first verify that the vendor actually does have paying customers and is stable (relatively to the rest of the internet companies).

It is not unknown that a SaaS vendor will go out of business, sometimes with little or no advance notice, and cause businesses a significant customer impact and/or data loss.

That’s not to scare you away from SaaS. The benefits far out weigh the downsides as long you perform due diligence when selecting your provider. In today’s market, for example, there is really no reason for an SMB to host their own CRM or email. You can outsource that to a SaaS provider for far less (when you subtract out the administrative overhead) of hosting it yourself.

LewisC

]]> http://clouddb.info/2009/02/11/defining-cloud-computing-part-3-saas/feed/ 0 http://clouddb.info/2009/02/09/defining-cloud-computing-part-2-distributed-computing/ http://clouddb.info/2009/02/09/defining-cloud-computing-part-2-distributed-computing/#comments Mon, 09 Feb 2009 17:01:15 +0000 Lew http://clouddb.info/?p=66 Distributed Computing

Distributed computing was one of the first real instances of cloud computing (albeit in reverse). Long before Google or Amazon, there was SETI@Home. Proposed in 1995 and launched in 1999, this program uses the spare capacity of internet connected machines to search for extraterrestrial intelligence. This is sort of the cloud in reverse.

A more recent example would be software like Hadoop. Written in Java, Hadoop is a scalable, efficient, distributed software platform designed to process enormous amounts of data. Hadoop can scale to thousands of computers across many clusters.

Distributed computing is nothing more than utilizing many networked computers to partition (split it into many smaller pieces) a question or problem and allow the network to solve the issue piecemeal.

Another instance of distributed computing, for storage instead of processing power, is bittorrent. A torrent is a file that is split into many pieces and stored on many computers around the internet. When a local machine wants to access that file, the small pieces are retrieved and rebuilt.

As the cloud computing buzzword has evolved, distributed computing has fallen out of that particular category of software. Even though distributed computing might take advantage of the internet, it doesn’t follow the other tenants of cloud computing, mainly the automatic and instant scalability of resources.

That’s not to say that a distributed system couldn’t be built to be a cloud environment. Bittorrent, or any P2P system, comes very close to a cloud storage. It would require some additional protections like file ownership and privacy across all nodes but it could probably be done. Privacy like that is not quite what P2P is all about though.

]]> http://clouddb.info/2009/02/09/defining-cloud-computing-part-2-distributed-computing/feed/ 0 http://clouddb.info/2009/02/05/defining-cloud-computing-part-1-2/ http://clouddb.info/2009/02/05/defining-cloud-computing-part-1-2/#comments Fri, 06 Feb 2009 00:58:50 +0000 Lew http://clouddb.info/2009/02/05/defining-cloud-computing-part-1-2/ So what is cloud computing? Defining the term “cloud computing” has become an industry unto itself. Is it utility computing? Is it an application service provider’s offering? Is it virtual machines in the sky? All of these are correct depending on who you ask.

Core features of cloud computing are scalable, centrally managed and accessible via the internet. Cloud computing boils down to running software on someone else’s robust hardware in a data center somewhere else. Let’s think about that for a minute: someone else’s software on someone else’s hardware in someone else’s data center. Nice.

Another key component has become a very low barrier to entry in both technical and financial terms. The service should be very easy to begin using and it should be very cheap and/or free (at least at a basic level). The general approach to pricing for most cloud platforms, so far, has been a per usage or subscription basis.

Per usage charges you only for what you actually use. If you use a virtual machine, you will pay a certain amount for every CPU hour consumed; for storage you will pay by the gigabyte or terabyte. You will usually have to pay some kind of bandwidth/networking costs. Think of per usage as metered.

Subscriptions are less common but are still offered. You may get a basic package for free and then start paying a subscription to upgrade. An example is the Mozy backup service from EMC. You get to backup 2GB of data for free. If you upgrade to the paid subscription, you can backup an unlimited amount of storage for $5.95 per month.

The final major pieces to the cloud computing puzzle are nearly instant scalability and nearly infinite scalability. If your web site grows from a few thousand hits per day to millions of hits per day, you don’t want your customers to get error messages until you can buy and provision new servers. Most platforms will allow you to automatically scale, or scale with just a couple of button presses.

I say “infinite” above with a somewhat tongue in cheek. Obviously, no one can offer infinite scalability. A more accurate expression might be that it allows exponential growth. Again, using a few thousand hits jumping to a few million, the scaling should be automatic and invisible. Customer demand should be load balanced to allow a pleasant customer experience (which means a pleasant business owner experience).

I will follow this post with some more definitions: Distributed Computing, SaaS, PaaS, IaaS and Cloud Desktops.

Let me know if you agree or disagree.

Thanks,

LewisC

Technorati : cloud computing, definition