Server side
"Considering the workload async webserver and engine have advantage over threaded/ blocking solutions. I had a choice between Java/ Python/ Javascript.
Java and it's Play framework (with addition of Actors) could work, but Java and rapid development do not go together well, furthermore, quite a research would have been required to use Play.
Node.js(javascript) which is becoming industries standard when it comes to async applications and the spaghetti code that is often written because javascript (v8 version) does not provide additional language constructions for async code averted me from using it (plus, lack of experience with node).
What was left was Python. I had previously worked with Twisted, which turned out to be extreme heavy weight, although, it provided a lot of flexibility, but the overhead code for backwards compatibility slowed down whole process. Tornado, on the other hand, seems to be rather optimised, has large community and stable releases."[#1]
"Because Tornado is not thread-safe, and is running in single-thread, to utilise full capabilities of multi-core machines we will need to run multiple instances of applications, thus, we will need a load-balancer to handle requests and divide them to different instances. Nginx proved to be capable, stable. Has a lot of modules."
"For static data caching, precaching queries and using it as AMPQ for offloading long-running tasks from web instances, Redis is one of the best choices, it bring together caching engine (i.e. memcache) and AMPQ messaging service (i.e. RabbitMQ), so it can run celery as well as simple memory caching."
"For storing file/ code/ some configurations. Storing it in a filesystem would require to use distributed filesystem, rather then that, I find using NoSQL with master-to-master replication more favourable. Because using MapReduce faster data processing can be achieved."
"For storing relational data, i.e. user accounts, logs, statistics, which might require/ be used for aggregation"
[#1] I have looked into Cyclone, which is "Tornado" on top of twisted (read twisted), benchmarks show that it is much faster then twisted, and on par, or a little bit faster then tornado, but the community/ open source solutions/ gists is less then previous 2.
