I have had a couple of questions via Skype about the architecture of the site due to it being based on Azure. It’s not a quick answer so I am going to address it here so I can save my self-repeating it over and over.
I used Visual Studio 2010 + Expression to build the Application ( it is .net based); Most of the core code is in ‘dll’s that is then referenced by the 3 applications that use it, the two Worker Roles and a Web Role.
Worker One
The first worker role which occupies two VMs, which span two Fault Domains and Two Upgrade domains are the most important in the system. Internally I reference them as the “downloader” however they now do much more than that.
The key things they do is download world data, then store it in to a Windows Azure Blob, being a cloud application and having two of these running we use the Windows Azure Queue to manage the workload between the two VM’s.
This workload is also scheduled by the same Worker which scans the apps settings for each world every 6 hours and then adds new items to the queue setting the invisibility of the message so they appear at the half hour intervals. The same system for scheduling downloads is also set for ‘processing’ of the data which is done by the other worker which I will address later.
This first worker role also scans for new worlds so every 6 hours it looks to see if the any new servers have come online once it has it then starts downloading data instantly.
The final job this worker role does is scan\downloads the world settings from Grepolis, they don’t make this available in a nice file like the rest of the data but with a touch of creativeness you can get the World Speed, Unit Speed, Gods, if it is HW and the World Name hence making the whole system of search, adding and processing a new world now all automatic.
Worker Two
This worker is currently running on 4 VMs (2 Fault Domains, and 4 Upgrade Domains) its key role is to process the data that the other worker role downloaded, this will keep based on the processing power we need.
The previous worker downloaded all the text files in to a Windows Azure Blob, this role downloads that data and also downloads the “current” data from the Azure Table Storage, it then processes each record looks for changes, merges the changes with the current data in memory then once it has done it then uploads the data back to the Azure Table Storage. This is done every 6 hours; it heavily uses the Parallel Task Library when communicating with the Azure Table Storage otherwise inserting a few million rows in a short time wouldn’t happen.
We keep the blob data for 2 weeks after it was downloaded, that means if something goes wrong and we have to reset a world we have two weeks data to start with, once the application is in production we will take that two 4 weeks.
A quick note about our choice to opt for a NoSql data store, a few people have gone “what?” when I told them we had used Azure Table Storage instead of SQL, but we are taking hourly data and tracking changes every hour that is a lot of data, and being a cloud application we intend for this to scale to every Grepolis Language that would mean we need a large data store.
The largest Azure SQL is 150GB at the moment, (which is a cost of £136pm) for Azure Table Storage while it has less Functionality 150gb of space is only £11.37 but more importantly still it has a limit of 100,000 gb per account which means we could scale this application with little work to span all ~400 Grepolis servers without many worries about hitting the limit in size.
There are a few other key reasons but I don’t care to share them at this time. (you have to keep some things secret after all)
Web Role
This takes up two VM’s and is what you all interactive with, we provided a Silverlight Application which using RIA Services with the web role gets the data you request.
The Silverlight Application uses the Navigation Framework which means it is possible for you to copy and paste links to your team makes and deep link straight in to the application. A few people have asked why Silverlight and not HTML 5. The quick answer is SL lends it self to the task much better than HTML5 does at this point. Our main target is to making a Windows 8 application which will “snap” to the right of your screen as you play Grepolis in the main window, and we aim to port this application over to WinRT in due course.
We are aware we have cut some iOS uses out of the loop a bit, but for the benefits it gave us and the speed of development Silverlight was the correct choice for the job. I also like that I can download the datasets in to Silverlight then let the use adjust them/change view with almost no code.
Lastly I would like to address the multi-language support if you have a none EN browser and visit the application it will attempt to change the language (if we support it) this is a tricky process because being an application more than a website you can’t use Google translate to use the site so we felt it was important to give provide that option ourselves.
So the first thing we are doing is trying to switch out as many words as possible and replace them with icons, secondly we are then translating each page one at a time it takes time so please be patient while we do it.
That just about wraps up the basics of how I built the application to its current state, using Windows Azure, Visual Studio, Expression Blende, Expression Design and some testing from a few friends, now I am going back to work.
