Twitter moves from Rails to Java

A colorful feather up Rails’ cap is on the ground now. Twitter has decided to go away from RoR in favor of Java, this time for their entire search stack. Earlier in 2008-09, they decided to move their message queue back-end from ruby to Scala (a Java Platform) and now it is the time for their front-end to move to Java as well.

They have built a scalable platform called Blender that uses Java NIO based server (Netty) to be efficient in the face of heavy incoming traffic, replaced MySQL with a Java based Lucene search engine, created an engine that parallelizes execution of multiple backend services with dependency management and more. With this setup there is a 3X drop in search latencies and can scale to 10X more requests per machine.

Wow, that is quite an achievement. Could this mean that Java is a better platform than Rails for high scalability needs? Even if that is the case, for simpler scenarios, the beauty of RoR out-weighs Java’s performance.

They say that this change will enable them to rapidly iterate on search features in the coming months. That along with the news that Twitter has hired 25 more employees kinda tells that Java’s code base is practically more maintainable than equivalent Ruby code – at least when the code base is huge and the team size is large. Or that could mean that this time they really put a lot of thought into designing a maintainable system than when they started out. But for smaller team size and code base, RoR is still an unbeaten champion.

Ganeshji Marwaha

I spend my days as the Director of Technology for Mobility practice and help my clients design enterprise and consumer mobile strategies. Mobile Payments, Digital Wallet and Tokenization technologies are my areas of specialization

29 Comments

  1. Ioannis Cherouvim   •  

    Regarding performance I’d say that for most (99.9%) websites the platform doesn’t matter. Use whatever you like and wish for the day where it won’t be able to handle the load.

  2. dave   •  

    when i saw the title i was expecting to read about grails. at this point for twitter rolling their own solution might be better, but for anyone starting out i’d recommend looking into grails. many if not most of the benefits of rails but compiles to java bytecode and is able to call into existing java code or other backends seamlessly.

  3. Ganeshji Marwaha   •     Author

    @Ioannis Cherouvim – Well said. That is a nice problem to have…

    @dave – Although grails is based out of groovy and compiles to java code it is not efficient in the least because of the dynamism it offers… But static groovy initiative that is going on now should change the sorry state of affairs for groovy. Meantime, scala is the perfect balance between modern programming concepts and performance. And twitter made no mistake in choosing scala for their messaging back-end.

  4. OtengiM   •  

    Me I will suggest instead of Grails, Go or start with Apache Click(stateless component framework), Stripes(MVC) or SpringMVC and in the future you don’t need to move nothing and stay in the Java world with all the power of scalable, search engines and all that twitter is now replacing with. Those frameworks are super easy to use and also cause they have some stuff inspired by Rails etc. Take a look and try it. 2c

  5. sam   •  

    reading the engineering blog from twiiter seems to be very difficult. there is written that twitter only replaced the rails search stack yet with java. but hey we love java so we ignore the truth and spread our around the world own wishes as fact :)

    there is also written that the “old machines” are still serving rails to handle the ui for the users and render the results of search query. so i assume the installed blender on new machines with more power.

    so gaining 3 times faster search performance is not magic thing if you divide your system into different stacks. one stack handling ui, an other one searching and so on. but hey we love java so we ignore the truth and spread our around the world own wishes as fact :)

    there are so many bad news aroung java since oracle takes the ownership so we java lovers take every single mini victory and make him big and spread them around the whole world.

  6. Pingback: Once Again, Twitter Drops Ruby for Java | Tech Toinks!

  7. Pingback: Once Again, Twitter Drops Ruby for Java | Technology Blog

  8. Pingback: Once Again, Twitter Drops Ruby for Java | ???????

  9. Pingback: Once Again, Twitter Drops Ruby for Java | SEO College

  10. Pingback: Once Again, Twitter Drops Ruby for Java | 365Online: E-Commerce en Online Marketing

  11. Pingback: Once Again, Twitter Drops Ruby for Java | thefinalcastle.com

  12. Pingback: Once Again, Twitter Drops Ruby for Java | Programming Blog

  13. Pingback: Once Again, Twitter Drops Ruby for Java | SEO Facts

  14. Pingback: Once Again, Twitter Drops Ruby for Java « Socialyze

  15. Pingback: Once Again, Twitter Drops Ruby for Java | Scripting4U Blog

  16. Chetan   •  

    grr… Twitter needs technical support here. :-)

  17. essays uk   •  

    I tried in header.php in navigation simply insert a link to “portfolio” with the address of the page you want, put next to an elementary html link “subscribe to rss “, but it turned out that my link out of the menu. Nearby, but outside the frame.

    uh … I changed something?

  18. Pingback: Twitter?Rails????Java | ???

  19. Pingback: Once Again, Twitter Drops Ruby for Java | Object Oriented Programing

  20. Pingback: Twitter moves from Rails to Java | iDuCK

  21. Pingback: Best Way To Become A Successful Article Marketer | I Do Articles

  22. vincent.Chan   •  

    twitter facebook are all wallled by our great fire wall.
    Hah.

  23. Pingback: Java rails | Kinolyar

  24. Pingback: Once Again, Twitter Drops Ruby for Java - 25 Popular Blogs - Popular Bloggers.com

  25. Werner Keil   •  

    “Replacing MySQL with Java based Lucene” Sorry, but that makes you sound clueless! MySQL is a database, also owned by Oracle now btw, but has nothing to do with a search engine like Lucene.

  26. Ganeshji Marwaha   •     Author

    @Werner Keil – Read both my article and the original article and don’t be surprised or shocked if you realize that you are the one who is clueless.

  27. Pingback: Twitter migrating from Rails to Java - Open News

  28. Pingback: Twitter java | Alishiawebster

Comments are closed.

If you have any questions/comments, the best way to reach me is via twitter @ganeshmax