Share via

Disadvantages of Ruby for Test Automation

  • Article

In the past few years there has been a push to use Ruby as a programming language for test automation. Ruby as a programming language has some benefits compared to other scripting languages. However, for test automation or any other serious application development the disadvantages of Ruby certainly outweigh any of the recent sensational hype promoting the language. I may be a bit biased, but I honestly can’t fathom why a tester wanting to learn a programming language would spend time learning Ruby, especially with the ease of use, availability of resources, and broad adoption of C#.

1. Ruby lacks informational resources. A search on Barnes & Noble, or Amazon reveals about 50 or so books on Ruby programming. But, that is barely a drop in the bucket compared to more than 400 books written about C#. These numbers certainly don’t inspire a lot of confidence in Ruby as a broadly accepted programming language in the industry at large. Web searches for available online resources also reflect this tremendous disparity. Sure, the Ruby zealots support the few websites and respond to requests for assistance. But, there are a greater number of C# forums with greater numbers of registered members who frequently participate and provide solutions to questions. Additionally, you won't find too many universities or community colleges offering courses in Ruby programming. If Ruby is so good then why are there such limited resources? The answer is because there simply isn't the business demand, or other compelling reasons for adoption.

2. Ruby is not a high demand skill among employers. Take a look at any of the technical job sites such as Dice, Monster, etc. and you will not find a plethora of jobs asking for Ruby programming skills. For example, of the approximately 16,500 software testing jobs on Dice only 65 contain the keyword Ruby as compared to 1,668 job listings containing the keyword C#. That means there are 25 times more employers desiring C# as compared to Ruby. IT Jobs Watch in the UK has an interesting site with lots of statistics relative to software testing positions in the UK. Looking at software testing jobs C# is listed in the top 5 desired programming languages. (Ruby doesn’t even make the top 20 list anywhere on this site.) The job trends on Indeed provides a visual perspective comparing jobs with C#, Ruby, Perl, and VB.NET keywords.

3. Ruby has performance problems. Scripting languages are notoriously slower than compiled languages, but it seems that Ruby is often slower (CPU time) and requires a larger memory footprint as compared to other scripting languages. The Computer Language Shootout site provides some very interesting benchmark results for various languages. One benefit of automation is the decreased time to execute a specific test. Now, you may be thinking that it only takes a few more seconds to execute a test in Ruby as compared to Java or C#. So, let’s arbitrarily say that each automated script in Ruby takes 5 seconds longer to execute as compared to the same automated test in C#. That may not seem like a big deal. But, instead of running one or two automated tests you want to run a test library of 200,000 tests. That’s about 12 hours of additional time needed to run the test automation written in Ruby. Now, I am sure Ruby advocates will discuss the reduced cost of development time, but time to develop an automated test is a one time expense (this does not include sustained maintenance which is a variable cost compounded over time for all automation libraries). Depending on your product’s shelf-life, you may need to rerun your test automation suites for the next 7 to 10 years for sustained engineering and product maintenance.

 

4. Ruby has not been widely adopted in application development. So, you may ask why this is a weakness for testers writing test automation? The simple fact is that if the development team is programming in say C/C++ or Java, and the test automation is in Ruby you probably won’t get a lot of support from the development team to help reviewing or debugging test automation. Also, it is very likely the developers may not want to install the Ruby interpreter on their Windows machine to use test automation to reproduce a defect, and instead ask for the manual steps. The test libraries the development team creates will require porting to Ruby which increases the cost and effort. Since many developers are familiar with at least the basic syntax of C/C++ and Java it is easier for them to pick up C# syntax and understand automated test code.

 

5. Ruby is just as cryptic as any other programming language. All programming languages use unique syntax, and users must learn the language’s syntax to code effectively. Now, I am no expert in Ruby but let’s compare a Ruby script to launch Windows calc.exe as compared to a C# program.

 

--------------------------------------------------------------

 

# Launch calc.exe in Ruby

require “win32/guitest”

include win32::GuiTest

 

run_cmd “calc.exe”

 

-----------------------------------------------------------------

 

// Launch calc.exe in C#

using System;

using System.Diagnostics;

 

namespace MyNameSpace

{

    class MyClass

    {

        static void main ()

        {

            Process.Start(“calc.exe”);

        }

    }

}

 

-----------------------------------------------------------------

 

Obviously there are more lines of code in the C# program as compared to the Ruby script. But, considering the fact the template in Visual Studio auto-generates the framework for a console application (the primary method of writing an automated test case) the only thing I need to add to the .cs file are the ‘using System.Diagnostics’ namespace declaration, and the ‘Process.Start(“calc.exe”);’ statement. Additionally, the IntelliSense feature of the Visual Studio IDE references language elements, and even inserts the selected element into the code. Also, perhaps it is a matter of personal taste, but Process.Start() seems a lot more ‘readable’ than run_cmd.

 

Ruby activists boast how quickly they can teach Ruby scripting to non-programming testers. I have been teaching C# to testers for more than 3 years. I have been very successful at teaching testers with no previous programming skills to write automated GUI tests in C# that will launch a Windows application, manipulate the application, generate and send data, verify and log results, and clean up the test environment within a day.

 

There may be some interesting features of Ruby, but don’t get sucked in by all the fanatical propaganda. Ruby has been around for more than 10 years and hasn't replaced any language or garnered a significant following! The simple fact is that Ruby simply doesn't offer anything revolutionary, and thus hasn't compelled the development community to rush to adopt or support it. All programming languages have strengths and weaknesses depending on the application. But, for test automation Ruby is not the best choice as a programming language.

 

In my (biased) opinion, I think C# is a much better choice, and in a later post I will outline the benefits of C# as a programming language for test automation.

Comments

  • Anonymous
    October 10, 2008
    I can't find anything to match Ruby's testing speed on water. Try Ruby with Watir on Browsers.

  • Anonymous
    October 10, 2008
    Hmm...something is telling me that you need to do some real performance measures. All perf studies on Ruby clearly indicate it is slower than many other scripted languages, and scripted languages are certainly slower compared to most compiled languages. Your statement would carry more weight if you actually provided empirical evidence of your claim (eg. Ruby compared to what?, what specific tasks did you measure, how was it measured? where is the repeatable data?). But, that being said, if you are satisfied with the performance you are getting from Ruby that's great.

  • Anonymous
    June 17, 2009
    PingBack from http://pooltoysite.info/story.php?id=11202

  • Anonymous
    November 16, 2010
    Old article, so your points may not stand as much.

  1. The documentation around ruby is great. The forums too. I primarily work with C# but when using Ruby the base and lib documentation, and forums is so much better than the MSDN equivalent, or asp.net forums etc... C# may have plenty of books, but I honestly can't remember the last time I needed to use a book. Google, pair programming, alt.net groups etc... provide excellent ways to learn.
  2. Ruby is more so, at least from my experience. Even if .Net is a primary language we've used ruby for automation or my preference, for build and deploy scripts.
  3. I'm not sure performance is at all an issue with automated tests. Maybe if you were performing calculations, but when controlling a browser ruby won't be the bottleneck. I used selenium at my last place of employment with C#, and currently use Ruby/Watir now. Where the speed is lost is the controlling framework, responses and the browser, not the language. And I know you only used it as an example, but if you were running 200,000 tests then I'd be questioning your CI setup rather than the language you used.
  4. This point is slowly changing over time. Given rake becoming more popular, and again people using watir, most developers do know some ruby, or at least have it installed on their machine. And the days of a developer knowing only one language are a long way back. You often need to know multiple languages, or use whatever tool best suits your needs.
  5. Ruby is typically more readable, and you can do the same using a lot less code. There are various IDE's with Intellisense (RubyMine), or you can happily use (I do) a text editor. It's the libraries that make ruby so effective too. Cucumber is a stand out tool, and mature. .Net has Specflow, which isn't as mature and in many ways doesn't work as well. You're right it's not the be all and end all. But for writing web based automation scripts it's excellent. That and build scripts. And in my opinion preferable to C#.

  • Anonymous
    October 31, 2012
    Hi, Like in point 2, can you add a stats for how many automation tools use each of these languages? That way it will be possible to say which one is in great demand. Thanks, Pradeep

  • Anonymous
    March 04, 2014
    sf ssdgfg fdng dsfg; fgsigonfisoghishgisdfg s dfgdfgsdng gsoidfng sifgnsinginfg sihfdgiohgoisfgh soidhfgiofhdsg soihdfgidosh isodhfgsiofhgsd g sdhgs ghsdoighsdoigs dogfdghsohg oihdsfgoihsoghoshdgsoghsoigighdighoighgiohgidsoighd gsgidh goishfgsdifgskjdnffjnvfhsdiuogsfhgsfsgoihaioghn gihsioghdfsogihdsfioghsdgifhiohgoidghodfighdoifghdfoigdhgiohdgiodhgiodfghdoighdfogihdoighdfoighdghdoighdoghdgodgodi dfighdiofghodifghdfiog dfoighdfiog dfoighdfoig dfioghdfoighdfo dfogihio gdiohgdoigidoshg dg digd gdifg dfigsdf gs'fdgj fdigj fdigfdgj dfigjdsf gjsdfgggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg

  • Anonymous
    May 14, 2015
    Ruby lacks informational resources. I agree. That's why I never use this. I used it once, and I regret it many many years later. <a href="www.ideaconsulting.co.id/">Lowongan Kerja Gratis Jakarta</a>

  • Anonymous
    May 14, 2015
    Ruby lacks informational resources. I agree. That's why I never use this. I used it once, and I regret it many many years later. www.ideaconsulting.co.id