Twitter moves from Rails to Java April 11th, 2011

A colorful feather up Rails’ cap is on the ground now. Twitter has decided to go away from RoR in favor of Java, this time for their entire search stack. Earlier in 2008-09, they decided to move their message queue back-end from ruby to Scala (a Java Platform) and now it is the time for their front-end to move to Java as well.

They have built a scalable platform called Blender that uses Java NIO based server (Netty) to be efficient in the face of heavy incoming traffic, replaced MySQL with a Java based Lucene search engine, created an engine that parallelizes execution of multiple backend services with dependency management and more. With this setup there is a 3X drop in search latencies and can scale to 10X more requests per machine.

Wow, that is quite an achievement. Could this mean that Java is a better platform than Rails for high scalability needs? Even if that is the case, for simpler scenarios, the beauty of RoR out-weighs Java’s performance.

They say that this change will enable them to rapidly iterate on search features in the coming months. That along with the news that Twitter has hired 25 more employees kinda tells that Java’s code base is practically more maintainable than equivalent Ruby code – at least when the code base is huge and the team size is large. Or that could mean that this time they really put a lot of thought into designing a maintainable system than when they started out. But for smaller team size and code base, RoR is still an unbeaten champion.

29 Responses to “Twitter moves from Rails to Java”

  1. 1. Ioannis Cherouvim on April 11th, 2011 at 2:23 pm

    Regarding performance I’d say that for most (99.9%) websites the platform doesn’t matter. Use whatever you like and wish for the day where it won’t be able to handle the load.

  2. 2. dave on April 11th, 2011 at 3:36 pm

    when i saw the title i was expecting to read about grails. at this point for twitter rolling their own solution might be better, but for anyone starting out i’d recommend looking into grails. many if not most of the benefits of rails but compiles to java bytecode and is able to call into existing java code or other backends seamlessly.

  3. 3. Ganeshji Marwaha on April 11th, 2011 at 7:05 pm

    @Ioannis Cherouvim – Well said. That is a nice problem to have…

    @dave – Although grails is based out of groovy and compiles to java code it is not efficient in the least because of the dynamism it offers… But static groovy initiative that is going on now should change the sorry state of affairs for groovy. Meantime, scala is the perfect balance between modern programming concepts and performance. And twitter made no mistake in choosing scala for their messaging back-end.

  4. 4. OtengiM on April 11th, 2011 at 7:39 pm

    Me I will suggest instead of Grails, Go or start with Apache Click(stateless component framework), Stripes(MVC) or SpringMVC and in the future you don’t need to move nothing and stay in the Java world with all the power of scalable, search engines and all that twitter is now replacing with. Those frameworks are super easy to use and also cause they have some stuff inspired by Rails etc. Take a look and try it. 2c

  5. 5. sam on April 11th, 2011 at 9:54 pm

    reading the engineering blog from twiiter seems to be very difficult. there is written that twitter only replaced the rails search stack yet with java. but hey we love java so we ignore the truth and spread our around the world own wishes as fact :)

    there is also written that the “old machines” are still serving rails to handle the ui for the users and render the results of search query. so i assume the installed blender on new machines with more power.

    so gaining 3 times faster search performance is not magic thing if you divide your system into different stacks. one stack handling ui, an other one searching and so on. but hey we love java so we ignore the truth and spread our around the world own wishes as fact :)

    there are so many bad news aroung java since oracle takes the ownership so we java lovers take every single mini victory and make him big and spread them around the whole world.

  6. 6. Once Again, Twitter Drops Ruby for Java | Tech Toinks! on April 11th, 2011 at 11:34 pm

    [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  7. [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  8. 8. Once Again, Twitter Drops Ruby for Java | ??????? on April 12th, 2011 at 12:05 am

    [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  9. 9. Once Again, Twitter Drops Ruby for Java | SEO College on April 12th, 2011 at 12:06 am

    [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  10. [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  11. [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  12. [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  13. 13. Once Again, Twitter Drops Ruby for Java | SEO Facts on April 12th, 2011 at 3:56 am

    [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  14. 14. Once Again, Twitter Drops Ruby for Java « Socialyze on April 12th, 2011 at 3:56 am

    [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  15. [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  16. 16. Chetan on April 12th, 2011 at 8:42 am

    grr… Twitter needs technical support here. :-)

  17. 17. essays uk on April 13th, 2011 at 8:06 am

    I tried in header.php in navigation simply insert a link to “portfolio” with the address of the page you want, put next to an elementary html link “subscribe to rss “, but it turned out that my link out of the menu. Nearby, but outside the frame.

    uh … I changed something?

  18. 18. Maximum Money Blueprints Review on April 15th, 2011 at 1:18 pm

    Tweeter moves it back. Thanks for this interesting post.

  19. 19. Twitter?Rails????Java | ??? on April 23rd, 2011 at 6:39 pm

    [...] Twitter moves from Rails to Java [...]

  20. [...] a big shift. Twitter moved its back end message queue from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  21. 21. Twitter moves from Rails to Java | iDuCK on April 28th, 2011 at 9:11 am

    [...] today=new Date(); var day; var date; var hello; hour=new Date().getHours() if(hour < 6)hello=' ???! ' else if(hour < 9)hello=' ???! ' else if(hour < 12)hello=' ???! ' else if(hour < 14)hello=' ???! ' else if(hour < 17)hello=' ???! ' else if(hour < 19)hello=' ???! ' else if(hour < 22)hello=' ???! ' else {hello='???! '} var webUrl = webUrl; document.write(' '+hello); function showLocale(objD) { var str,colorhead,colorfoot; var yy = objD.getYear(); if(yy ?????: ?? ???????? RSS ??? Twitter moves from Rails to Java ???1 ??? ⁄ ???? ⁄ ???? var jiathis_config = {"data_track_clickback":true}; var addthis_config = {"data_track_clickback":true}; it is form here [...]

  22. [...] article. You can get more information about  Extreme Niche Empires by checking out my profile here [...]

  23. 23. vincent.Chan on May 5th, 2011 at 2:27 pm

    twitter facebook are all wallled by our great fire wall.
    Hah.

  24. 24. Java rails | Kinolyar on July 23rd, 2011 at 6:14 am

    [...] Twitter moves from Rails to Java | GaneshApr 11, 2011 … A colorful feather up Rails’ cap is on the ground now. Twitter has decided to go away from RoR in favor of Java, this time for their entire … « Ryan brescia [...]

  25. [...] a big shift. Twitter moved its back end message que from Ruby to Scala, a Java platform in the 2008-2009 time frame. The move was attributed to issues [...]

  26. 26. Werner Keil on October 5th, 2011 at 12:06 am

    “Replacing MySQL with Java based Lucene” Sorry, but that makes you sound clueless! MySQL is a database, also owned by Oracle now btw, but has nothing to do with a search engine like Lucene.

  27. 27. Ganeshji Marwaha on November 4th, 2011 at 5:09 am

    @Werner Keil – Read both my article and the original article and don’t be surprised or shocked if you realize that you are the one who is clueless.

  28. 28. Twitter migrating from Rails to Java - Open News on November 7th, 2011 at 8:32 am

    [...] article is from Twitter moves from Rails to Java translation of this [...]

  29. 29. Twitter java | Alishiawebster on February 23rd, 2012 at 3:57 am

    [...] Twitter moves from Rails to Java | GaneshApr 11, 2011 … A colorful feather up Rails’ cap is on the ground now. Twitter has decided to go away from RoR in favor of Java, this time for their entire search … [...]