The other day I sent a friend to this site and they said it wouldn’t load. Interestingly, twitter has been flakey lately. As it turns out Twitter is experiencing some catastrophic scaling issues so it would seem. Connection? Yep. We pull our rss feed from Twitter for the Twitterfeed in the sidebar and guess what, if you can’t ping twitter the code just waits for twitter to load eventually. Oops, right?

It’s an easy fix, just switched over to CURL, setup a timeout and we’re golden. However, that doesn’t solve the fact that twitter has outgrown its britches. It’s possible they could blame their Ruby on Rails setup which, in fact, might be legit. In fact, RoR is not the best choice for speed in production and the benchmarks I’ve seen seem to point to Python/Django as the better choice. As it turns out Google AppEngine uses that exact platform for speed and scalibility. Coincidence? I doubt it.

Yesterday I looked into the issue of Twitter’s scalibility issues and I guess they aren’t new at all. It’s a problem that grew a lot over time. My guess is they need a better DB design and better sharding to optimize for reads. Then again, twitter is a chat engine, so it’s probably more write oriented which makes the whole situation worse for performance. I wonder how hard it would be to redesign their system to run on a scalable archtecutre like Amazon EC2. Something to think about. At least Ideastone.com loads fast again!