Archive for the ‘cloud book’ Category

Using and Managing AWS – Configure the Security Group

January 25th, 2013 Comments off

Configure the Security Group

The next step is to plan out your security group. Remember that the security group is your instance’s firewall. You open (or close) ports, as needed, for the applications that will be running.

The first port to open is 22. Port 22 is used by SSH and is your primary means of connecting to the instance.

If you will be running a database, you will need to open the appropriate port. For example, Oracle uses a default port of 1521. It is a best practice not to use the default so you may want to open 15120 or something like that. The same applies for any other database using TCP connections.

If you are going to be running web applications, you will need to open ports 80 and, if running SSL, 43. I usually open alternative web ports such as 8000 and 8080 to allow me development and administrative access.

The ports that you open are determined by the applications that you will be running. You can open up all of the ports if you want to but that is inherently insecure. As a best practice, only open those ports that are required.

You cannot change the security group in a running instance but you can open and close ports while the instance is running. You can start with just port 22 and modify as needed.

Categories: cloud book, cloud data Tags:

Using and Managing AWS – Part 6: SSH Key Pairs

May 26th, 2009 Comments off

Generate Your Keys

Now that you have chosen your instance, but before starting you actually start your instance, you need to generate your key pairs. The keypairs are SSH keypairs. A later post will explain SSH in greater detail but the keys come in a pair because there is both public and private components.

SSH is a Secure SHell. This is a command prompt like a DOS box or a telnet connection. However, unlike DOS and Telnet, it is very secure. The private key is the local machine’s secret password. The public key is shared to any host that the local machine will connect to.

The host is able to create a query after seeing the public key that only someone with the private key could answer. The private key is never shared but the host is convinced that it is talking to the person (or machine) that is says it is.

This may sound confusing but it is actually very secure. It’s is much better than passwords that can be hacked or accidentally given away.

Amazon supports SSH and secure communications out of the box. If you choose to revert to simple protocols such as telnet and ftp and to password authentication, you may do so. However, your first connection to any instance started through AWS will have to be via SSH. Amazon makes it easy to be secure but gives you the option of making it less secure.

So at least one pair of keys needs to be generated. Each tool set that you choose will create the files in a different way. If you are running the command line tools, you will run the ec2-add-keypair program. If running ElasticFox or CloudStudio, you will have a button on the GUI. However you create the keypair, the end result is that you will end up with a file that tends in a .pem format.

When running SSH (and the tools) from a Windows client, you will need to convert the .pem file to a PuTTY formatted key file. PuTTY, like SSH will be documented in greater detail in a near future post. Review that post for tips on Converting SSH to PuTTY.

You choose an instance’s keypair when you start it and you cannot change that after it is running. Generate your key pair and getting working first.

Using and Managing AWS – Part 5: Choosing a Machine Image

May 21st, 2009 Comments off

Choose an AMI

Amazon, and Amazon clients, are providing a huge variation of machine images. The short story is that you can choose between MS-Windows, Linux and Sun Solaris for your OS. The real story is that it is a bit more complicated than that.

The real question is what applications do you plan to run and what expertise do you have on hand or plan to hire? A quick example is a database like MySQL. MySQL runs on various operating systems. If you have Windows expertise, you may want to stick with windows. On the other hand, you can run some Linux instances with MySQL pre-installed and configured.

This about the stack that you want to run. I generally run Linux instances. They are a few cents cheaper per CPU hour and I am good enough with Linux that it doesn’t cause me any issues. I can run Oracle, MySQL and Postgres side-by-side. I occasionally do run Windows instances though just to compare offerings.

If you run SQL Server, you will need to run Windows. Almost any other software stack offers an option of OS. If you do run Windows, you will be running Windows Server 2003, in either 32 or 64 bit. SQL Server can be the Express Edition or the full blown commercial edition (which costs extra for licensing).

If you want to run Solaris, you currently have to register with Sun to get access to the OpenSolaris instance. It’s free but it requires registration. With OpenSolaris you get DTrace and ZFS, two selling points for many people.

You get OpenSolaris 2008.05 or Solaris Community Edition and pricing is the same as a Linux install. You can AMIs with AMP preinstalled as well as stacks like Drupal and MySQL.

For Linux installs, the choices are almost limitless: Fedora, CentOS, Ubuntu, Oracle Unbreakable Linux, RedHat. You name it, it’s probably there. Many of these come with pre-installed software stacks. No download and configure, just run.

Plan to try many instance types. You may even end up with a RightScale instance.

A quick overview of PuTTY and SSH for AWS Newbies

May 17th, 2009 10 comments

Linux Access with SSH & PuTTY

This post will (attempt) to explain what SSH and PuTTY are so that as a user you understand the terminology of AWS and so that you can be productive in the environment. This post will not attempt to make you an expert in SSH. For best practices in implementing SSH, I strongly recommend a book dedicated to hardening *nix (Linux, Unix, Solaris, etc).


In the early days, not that long ago really, of networking, very simple tools were used to work with remote computers: telnet as a console, ftp for file copying, rsh for remote command execution and others. These were easy to configure and use tools. They were client server in that a software component needed to run on both the local machine (client) and the remote machine (server).

While easy to use, they were very insecure. They made no pretense at verifying that the calling host really was the calling host. Everything was username/password based and both the username and the password were passed around the network in cleartext. If you intercepted the little data packages that were being routed around the network (with a sniffer for example), you would be able to extract the login credentials. Even if you encrypted all of your data, your credentials were still in the clear.

SSH is an attempt (quite successful) to fix those insecurities without making things anymore complex than they need to be. SSH stands for Secure SHell. However, SSH is not really a command shell, it is rather a protocol that encrypts communications. That means that programs that use SSH can work like telnet or ftp but will be more secure.

Note: Technically, SSH is also a tool. There is a client terminal program called SSH. It’s a non-graphical command line tool that provides a window which executes a command shell on the remote system.

SSH offers multiple modes of connecting but for the purposes of AWS, we will talk about key based access. To make things more secure, EC2 uses a key based authentication. Before starting an instance, you need to create a key pair.

Note: The below explanation of SSH is a gross over simplification. I am just trying to give you a feel for what is going on. If you really want to understand the technical details, I really do recommend that you purchase a book. My personal recommendation is SSH, The Secure Shell: The Definitive Guide from O’Reilly.

When an instance starts up for the first time, EC2 copies the ssh key that you created to the proper directory on the remote server. The remote server will be running the SSH Server software.

You will then use an SSH client to connect to the server. The client will ask for some information proving that the server really is who it says it is. The first time you connect to a server, the client won’t have that information available so it will prompt you to vertify that the server is legitimate.

You verify that information by comparing a thumbprint. Verifying a host is a bit beyond this book but do an internet search for for “ssh host thumbprint”. You’ll find a variety of articles explaining it in detail.

Once the client accepts the host, the client will send secret information to the host. This is your key data. If the host is able to make a match, it will authenticate you and let you login in. If the host then asks for a password, you key did not work and something is not configured properly. In my experience, it will probably be that your client key file is not in the place your client is expecting it to be.

What happens next depends on the tool you are using. If you are using a terminal program, ssh for example, you will now have a command prompt. If you are using sftp or scp, you will be able to copy files.

In addition to command line tools, there are GUI tools that use the SSH protocol. WinSCP is an excellent SCP client for Windows.

Regardless of the tools you use, SSH is busy encrypting everything you send over the wire. The SSH protocol has evolved over the years, and will probably evolve even more in the future, but it is currently running a very secure form of encryption.

If you are running Linux, you are pretty much finished at this point. SSH ships with every Linux distribution that I am aware of. If you are using Windows, however, you either need to install CyWin (a unix environment that runs in windows), or you’ll want to get PuTTY.


You can download all of the programs discussed in this section at:

I honestly have no idea why PuTTY is spelled PuTTY. I can figure the TTY part of it is from the Unix command that output a display. I’m not sure bout the Pu though.

I do know what PuTTY is though. PuTTY is a very simple implementation of an MS-Windows SSH terminal client. When I say it is simple, I mean that as a complement. This is a tool that does not get in the way.

You tell PuTTY to connect to a remote server and, as long as your keys are configured, it will connect you. If are not using keys, you can connect with passwords (if the host allows that). As a best practice, keys are recommends over passwords.

PuTTY is the terminal client but you can get a couple of other tools from the same author. PSFTP and PSCP offer secure file transfers. These tools are as easy to use as PuTTY and work pretty much the same way.

For command line syntax and configuration, take a look at the documentation at the link above.

A note about SSH keys and PuTTY, they are not compatible. This same web site offers a utility called PuTTYgen. When you create a key pair for EC2, you download that file to your local machine. PuTTYgen converts that file (a .pem file) to a private key file (a .ppk file).

PuTTY Key Generator

PuTTY Key Generator

The tool is named puttygen.exe. Run the executable and the above window pops up. To convert an amazon key to a PuTTY key, use the menu option Conversions ? Import Key. Load the .pem file that you downloaded and press the Save Private Key button.

It will warn you about leaving the passphrase blank. That’s ok.

Save the file to the location that PuTTY has been configured to look in for it’s keys.

Using and Managing AWS – Part 3: AWS Security

May 17th, 2009 1 comment

AWS Security

Data Center Security

Amazon is a well known entity and works to provide an extremely secure environment for your applications ans your data. Amazon is pursuing Sabanes-Oxley certification (by an external auditing agency) and SAS-70 Type II certification.

Amazon does not broadcast the locations of their data centers and physical security is a top concern for them. They have military grade external protections. Physical access to Amazon data centers controlled by a two-factor authentication and only those Amazon employees with an actual need are ever given access.

Hardware access is provided only to those administrators who directly require it and they must use their own SSH keys to access bastion hosts (kind of like cloud overseers). They can then escalate access to gain access to individual client hosts. All administrator access is logged and audited.

The network is monitored by Amazon security services. Due to Amazon IP security, an EC2 instance cannot spoof an IP address. An instance is not allowed to send traffic with a spoofed address. Also, Amazon monitors for port scanning. If they find port scanning, they block the incoming address.

Because all clients are running in virtual servers with virtual storage, there is no way for one client to gain access to another clients data or traffic. For all intents and purposes, each client is running in their own data center.

Data Security

Your data is secured when traveling over the wire by SSL. You can chose less secure methods once you have an image up and running but by default, an AMI will be very secure. If you choose to open your firewall (security group) to any and all traffic, you will be open to hacking. If you chose to use password security instead of SSH keys, you take your own risks.

There are several additional steps you can take to protect your data.

  • Only present web servers to the internet. You have the option of not having a public IP address on every instance. If you have amulti-tier application, you can choose to have a public IP address on your web server and have just an internal IP address on your database server. To access the database server, you would have to log into the web server and then ssh from there to the databaseserver.
  • Another option is to encrypt all of your stored data (or at least the sensitive portions of it). Amazon offers Linux, Windows and Sunvirtual machines and all of these operating systems offer very robust (at least via third party tools) encryption. A very good, freeoption on windows servers is TrueCrypt.

Data being stored in AWS applications (S3, SimpleDB and EBS) is automatically, redundantly stored in multiple physical locations. You do not pay for this additional storage. Amazon does this to ensure the integrity of your data (and that they meet their SLAs).

Yet another option is to use the encryption capabilities offered by the various databases that you might be using. Oracle provides Transparent Data Encryption for data at rest and offers Oracle Secure Backup via RMAN. Using Oracle Secure backup with the Cloud Module extension will allow you to encrypt your back ups and store your data on S3.


AWS allows two different methods of authentication. When you submit a request, be it to create a new instance in EC2 or to upload a file to S3, AWS needs to know that you are allowed to to submit the request that you are submitting.

AWS recognizes two different types of request identifiers: a secret key or an x.509 certificate. The x.509 certificate can only be used with SOAP transactions and can only be used with certain EC2 and SQS requests. The secret key method can be used with all of the services and for all of the request types. For that reason, I will assume you have chosen to use the secret key method and that is the method I will be using here on the blog.

Amazon allows you to regenerate your key any time you decide that you need to. Remember that your access keys are what you will use to access AWS via any third party software or external vendors (such as RightScale).

To get to the Access Identifiers, choose Your Account ? Access identifiers from the menu shown in Image 2 above. This screen will allow you to generate a new secret access key and a new x.509 security certificate.

Your access key does not change and is included on all requests to AWS. You can think of your access key as your username. Think of you secret key as your password. You can change your secret key at any time but your access key stays the same all the time.

Amazon notes on the page that you must protect your secret key and never email it to anyone. You will need to give it away under certain conditions though. When you use a third party tool like ElasticFox or Cloud Studio, you need to enter your credentials. You will also need to give your secret key to a third party vendor like RightScale who will issue requests on your behalf.

AWS Access identifiers

AWS Access identifiers

You can see your secret key by clicking the “+ Show” link. You will come to this screen to get your access keys. When entering the values into other tools, use cut and paste to do so. When you paste, paste it first into notepad or a VI session. For some reason, when cutting this data, it usually has several extra spaces at the end that will prevent it from working when you paste.

To generate a new key, just press the Generate button. Because this key is like a password, you may want to regenerate the key on aregular basis. If you are sharing this key at all, you will need to make sure you update anyone who has it.