30 May 2011

Microsoft Web Farm Framework using AWS EC2

As can be the case with some products there can be a degree of tweaking required outside the advised setup to get them functioning, and the recent addition to IIS, the 'Web Farm Framework' (WFF), is no exception.
WFF is great in concept and there's no doubting it's usefulness for distributed web application and site replication across a multi-server platform. In this post I'm going to show you how to set up a basic two server WFF using EC2 instances across the Amazon AWS platform.

The first stage is to configure your AWS security group (Firewall) at the Amazon side in order for things to function smoothly during install. The following config is needed (don't worry, you can tie it down later) and I'll explain selective port allocation as we run through the installation.
In this example all source IP's are open however you can (and really should) tie things down to your own schema where possible (i.e for RDP etc).

You'll need a suitable Server 2008 AMI (Amazon Machine Image). For my WFF I used Amazon instance number ami-c3e40daa which is an i386 Server 2008 SP2 Data Center Edition base build. Onto this I added IIS, .NET4, all associated default dependancies + the most recent Microsoft updates, I've also (controversially) installed Microsoft Security Essentials. Yes, I know it's not officially recommended for a server environment but I'm not running a domain and it actually provides decent protection.

Ok, down to business.

For a two server setup we actually need three servers, one as the controller and the other two as primary and secondary instances. Fire up three AMI(s), selecting preferred region + zone, choose (or create) your key pair and select the security group you configured earlier. Should you wish to have all instances in the same zone you can start any number at once, however as this is a cloud platform resilience is recommended and multi-zoned instances are strongly advised. Due to the nature of Amazons DNS and IP allocation, and for obvious reasons, you'll also need to associate elastic (static) IP's to each server. This is quickly and easily achieved through the AWS console.

Carry out the following procedures on all three instances

  • Obtain the EC2 Admin passwords for each server (Right click over the instance in the AWS console and select 'Get Windows Password'). Generation can take up to 15 minutes but once you have it connect via RDP (Port 3389) to configure the build. Install base necessities as advised and update.
  • Set yourself up a WFF Admin account (with, of course, Admin rights) and disable (de-check) User Account Control (UAC)  - This is VERY IMPORTANT. WFF will throw a permissions error and won't connect to the chosen account unless you do, it took me a full day to discover this.
  • Next, Windows Firewall (Control Panel > Windows Firewall  > Allow a Program Though the Windows Firewall). Here you'll need to check Core Networking, File Sharing + Remote Administration. These entries correspond to the opened ports on the AWS security group but you will also need to add specific ports for the WFF itself (8173 & 8675), do so as single separate entries via the 'Add Port' tab.
  • The best, and most reliable way to install the WFF is by using the Web Platform Installer (WPI), this installs all dependancies and should have installed as part of IIS, if not download/install from here - http://www.iis.net/download/webfarmframework - Install, re-boot, and logon as the WFF Admin user you created earlier. *NOTE* - You only need to install the WFF on the Controller.
  • On the controller instance open IIS Manager and right click 'Server Farms', create/name your farm, check both the load balancer + the provisioning boxes and input the WFF admin account you created earlier, click next.
  • Make sure the 'Load Balancing' checkbox is ticked and add the elastic IP of the primary server. If all your security group settings are good it should drop in nicely with no errors.
  • Next add the elastic IP address of the secondary server and 'Finish'.
  • Back at the main IIS advisory screen under the server farm activity window you'll see the agents being installed to the primary and secondary boxes, it only takes a minute and once done you'll see load balancing status for both boxes change to 'Yes'. Try adding an application or website config to wwwroot and you'll see the changes being replicated. And that's it, WFF is installed and working. 

Supplemental :
  • From the Amazon side you can (and probably will) install your boxes behind a load balancer, there's no requirement to tie this in with the WFF balancer as the AWS balancer can be configured with its own range of comprehensive up/down schematics to route traffic accordingly, irrespective of what the WFF is up to.
  • Once installed you can remove the AWS security group entry for 49152-65535. This was only used by WFF for dynamic RPC during the initial setup to remotely install the agent.
  • *VERY IMPORTANT* - Once up and running you MUST tie down the security group ports between servers for respective IPs on at least ports 135-139 and 445. The last thing you want is an internet facing open SMB port.


Ranu said...

Thanks!! great!!! This experience and tutorial was what i been lookng for. Do you know if you could setup a web farm with only two ec2 instances and a ELB?
Mariano Vicario

RichBos said...

Hi Ranu, thank you for visiting

I did try initially with just two servers but it never seemed happy, however feel free to try things yourself, I'd be interested to see if you had success where I didn't. Everything worked instantly and cleanly with three instances.

For info I used 'localhost' as the address on the Controller when installing it as Primary, it was the only way it would 'see itself'.


Jonathan said...

Hi -- I keep getting errors when I start the web farm (mostly on the secondary machine).

Failed to run operation 'AutoRepair'. Failed to run method 'Microsoft.Web.Farm.GetInstalledProductsRemoteMethod' on server 'ip here'. Exception in response stream. The 'META' start tag on line 8 does not match the end tag of 'HEAD'. Line 10, position 3. Exception in response stream. The 'META' start tag on line 8 does not match the end tag of 'HEAD'. Line 10, position 3.

Any ideas? Thanks!

RichBos said...

Hi Jonathan

Sounds like a can of worms you have there, typical Microsoft, works fine, and then doesn't. I presume this was an empty/fresh installation to totally clean 2008 servers? I've found things don't work too well unless this is the case.

If it helps I have x2 AWS EC2 pre-built Server 2008 Data Center edition clean instances, one Primary WFF controller + the other secondary, happy to send you over the AMI numbers and login/admin account details if it helps. They're both tested and working.

Contact me here with your Amazon account details - http://mebbi.net/contact.php

Mitul said...


Great article, thank you. I've got the Controller & Primary Server setup, but it fails to add the Secondary server, I'm getting the issue as described here http://serverfault.com/questions/328414/issue-provisioning-iis-7-5-server-using-web-farm-framework-2/333698#333698.

I'm using new EC2 Large Instances.

Did you encounter this issue?


RichBos said...

Turns out the WFF is still incredibly flaky, however there is a much cleaner and simpler method to achieve the same result, here's how.....


Post a Comment