Why I chose Selenium WebDriver instead of Telerik's free testing framework

Telerik Testing Framework vs Selenium WebDriver


In this post I will discuss why I chose Selenium WebDriver instead of Telerik's Free Testing Framework.

We are a Microsoft shop using C# and usually the latest version of Visual Studio.  I'm using nUnit because our build tool Bamboo easily intergrates with nUnit.

Telerik Testing Framework Information

First I want to say I love Telerik components.  My organization owns Telerik's Dev Craft suite and we use components from almost each toolkit.   Their components and support are wonderful.  I would recommend you check them out if you are in the market for high quality .NET components.

The one component I am not using from the Telerik suite is their free testing framework.    Their testing framework allows you to automate web and wpf user interface testing.    The API is very rich as it has went through several iterations.   When you install the testing framework it gives you nice Visual Studio templates that help you write your unit tests.  

Here is sample code using Telerik's API that tests logging into one of our sites.  I have omitted initialization code.

       

       [Test(Description="Attempts to login to Site")]
        public void Login()
        {
            Manager.LaunchNewBrowser(BrowserType.InternetExplorer,true);
            ActiveBrowser.NavigateTo("http://yoursite");
            ActiveBrowser.WaitUntilReady();
 
            Element userNameInput = ActiveBrowser.Find.ByName("UserName");
            Assert.IsNotNull(userNameInput, "unable to find the name element");
            userNameInput.SetValue("value","Name");
 
            Element passwordInput = ActiveBrowser.Find.ByName("Password");
            Assert.IsNotNull(passwordInput, "unable to find the password element");
            passwordInput.SetValue("value","SomePassowrd");
 
            Element loginButon = ActiveBrowser.Find.ById("login");
            Assert.IsNotNull(loginButon, "Unable to find the login button");
            ActiveBrowser.Actions.Click(loginButon);
            ActiveBrowser.WaitForUrl("http:/yoursite/index", true, 10000);
 
            ActiveBrowser.RefreshDomTree();
            Assert.AreEqual(ActiveBrowser.Url,"http://YourSite/index");
        }


Telerik's framework requires you to install their browser extensions into Firefox and Chrome so they can be automated.     I had issues installing the browser extension for Chrome becauseI use multiple profiles in Chrome.

Selenium WebDriver Information

The Selenium WebDriver is an interesting beast.   It has went through several iterations and is now at the Selenium WebDriver release.    The WebDriver is a proposed W3C standard.  

To me the API for the WebDriver and Telerik feel very similar.

The WebDriver requires drivers for each specific browser you want to automate.   The drivers need to be part of your PATH so they can be instantiated for browser automation.   Here is a link to the ChromeDriver.

Think of the drivers needed by the WebDriver as similar to Telerik needing their extensions installed into the browser.

An interesting feature about the WebDriver is the fact that it is using a proposed W3C standard to automate and interact with the browser.  Telerik's framework requires custom extensions to control browsers.  These extensions are not implementing the WebDriver protocol.

Here is the same code to log into one of our sites using Selenium's WebDriver.  I've omitted initialization code.

       

           
        protected void Login(string UserName, string Password)
        {
            driver.Navigate().GoToUrl(ServerUrl);

            var userNameInput = driver.FindElement(By.Name("UserName"));
            Assert.IsNotNull(userNameInput, "unable to find the name element");
            userNameInput.SendKeys(UserName);

            var passwordInput = driver.FindElement(By.Name("Password"));
            Assert.IsNotNull(passwordInput, "unable to find the password element");
            passwordInput.SendKeys(Password);

            IWebElement loginButon = driver.FindElement(By.Id("login"));
            Assert.IsNotNull(loginButon, "Unable to find the login button");
            loginButon.Click();

            WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(25));

            IWebElement headerElement = wait.Until(d =>
            {
                return d.FindElement(By.ClassName("HEADERpersonal"));
            });

            Assert.IsNotNull(headerElement, "Unable to load the personalization page.");
        }

You can see the APIs for accessing the DOM are similar.     The Telerik API to wait for the browser is more intuitive.   You don't need to create a separate object to wait.  

THE MAJOR DIFFERENCE

 So why did I decide to use Seleniums Web Driver instead of Telerik's testing framework?   We need to run automated tests on our build server.      Our build server is using Bamboo which runs as a windows service.

Anyway,  Bamboo runs on a server and we don't interact with bamboo by logging onto the server.  We access and manage our builds via Bamboo's cool web interface.

This is where Telerik's testing framework falls short.     To run unit tests that automate and control the browser you need an active desktop session for Teleriks testing framework.  This means someone has to log onto the server and run the tests.       This is not acceptable in our case.   We are not logging into the server.

We want our tests to be automated without an active desktop session.  I could not meet our business needs using Telerik's free testing framework.

Telerik states this is a limitation of windows and I do agree but the WebDriver does not have this limitation.

The way Telerik's testing framework automates the browser is what causes this limitation.    Here is a link to Telerik discussing workarounds.

In Telerik's defense they did purchase the company, ArtOfTest (I think that is the name), to have a testing suite product.  

Maybe in the future their testing framework will implement the WebDriver protocol and this won't be an issue.












Comments

  1. UnknownAugust 30, 2015 at 2:24 PM

    This is actually wrong. The described limitation only applies to "desktop" actions which you shouldn't be using anyway. The idea is this: normally when you automate the browser (with Telerik or Selenium) you're using JavaScript injection to execute code right in the browser. So it's like manually adding JavaScript code in the page to say "Click on this button". The mouse never moves when this happens - it's very different from a 'normal' user click. The code for that is something like this:
    HtmlControl e = ActiveBrowser.Find.ByName("buttonName");
    e.Click();

    But the Telerik Testing Framework also allows you to simulate 'real' clicks (the kind that move the mouse and trigger real click events) like this:
    HtmlControl e = ActiveBrowser.Find.ByName("buttonName");
    e.MouseClick();

    In the same way you have 'real' keyboard presses etc. And all of those require an unlocked desktop like this article says but the thing you don't really need to use them and you should NOT use them. They're clunky and if you just stick to JavaScript injected actions you'll be fine.

    I believe Selenium also exposes this type of 'simulated' actions that would failed without an active session in their Actions class.

    With that out of the way the Telerik Testing Framework doesn't really do anything differently from Selenium. I'm quite confident in what I'm saying because it was me that wrote that workaround article in the Test Studio documentation some years ago. Also, I recently worked on a big automation project with the Testing Framework and there were no major issues.

    But I'm not saying that Selenium isn't the better choice. Here's my own recent post on the subject:
    slashstars.com/telerikvselenium

    ReplyDelete
    Replies
    1. Rob McElroySeptember 29, 2015 at 9:34 AM

      Hi Stoil, thank for your feedback.

      I didn't know that desktop actions were frowned upon. I don't see that mentioned in any documentation. Is that documented on Telerik's site somewhere? We have Telerik Test Studio and i'm pretty sure it uses "desktop" actions to capture many of the users key strokes.

      If I remember correctly, I did try to find elements by DOM name and initiate a click and it never worked for me using the Telerik Testing Framework when the desktop was locked. When I have time I will circle back around and test it again.

      I do like that Selenium uses a different mechanism overall to communicate w/ the browser, via the WebDriver protocol which is a proposed W3C standard. Having to install an add-in for each browser is what I remember the Telerik testing framework required. That experience was more painful especially on QA devices.

      Thanks again!

      Our QA team has Telerik Test Studio and we all love it.

      Delete
    2. UnknownOctober 5, 2015 at 4:25 AM

      The Test Studio documentation doesn't state that desktop actions are bad. In practice desktop actions are easier to use in certain situations where you need to trigger javascript events. The desktop actions trigger all that stuff like a regular user. You don't need to worry too much about what's going on under the hood. This is inline with Test Studio's philosophy of abstracting away the test recording as much as possible and just letting you churn out tests quickly.

      But IMO too much is sacrificed in terms of flexibility and stability and in the end you will get more bang for your buck if you code out your own test framework. The skill set of the QA team will also be a factor and if you don't have coders you don't really have much choice.

      You can definitely execute non-desktop action on a locked screen. You're probably right that Selenium communicates with the browser in a better way but the Testing Framework is not really hard to setup either.

      Delete
  2. akhilapriya404October 25, 2018 at 5:16 AM

    Thanks for posting such a great article.you done a great job selenium Online Training

    ReplyDelete

Post a Comment

Popular posts from this blog

Displaying shared content across multiple Sites

Sitecore does a great job of separating content from the presentation of the content. This is the main selling point of any CMS tool, but Sitecore does an excellent job of allowing this to happen. What I found not intuitive in Sitecore is the ability to display shared content. PROBLEM For instance, assume you have stories which live outside of the context of any site site. The stories can be tagged with specific categories, where each category associates to a different web site. (ok, this is a simple example, but follow me here). So you have the following structure in your sitecore tree /sitecore/content/site 1 .. [all of the nodes that make up the site] /sitecore/content/site 2 ..[all of the nodes that make up the site] /sitecore/content/Stories 2010 January Story Item 1 [category = site 1] Story Item 2 [category = site 1 and site 2] February Story Item 1 [category = site 1 and site 2] ...etc [you get the picture] POSSIBLE SOLUTIONS Wh
Read more

Handling the situation where scItemPath does not exist in Sitecore MVC

What happens if the scItemPath specified in your route does not exist in Sitecore? In this post i'm going to discuss how I handle this situation. Here is a sample route from Sitecore's MVC documentation .   (see Section 4.1) // matches the item '/home/blogPosts/{blogNo}' routes.MapRoute("example2", "blog/{blogNo}", new { scItemPath = "/home/blogPosts/{blogNo}" }); But what happens when the blog number doesn't exist?   What page will Sitecore show?  What if the user has a link to an old blog post that you have archived or someone links to a blog post using the wrong number by accident? What I wanted was a new route value that specified where to go when the path resolved by scItemPath is not found.  So I created the scNoItemFound value. // matches the item '/home/blogPosts/{blogNo}' routes.MapRoute("example2", "blog/{blogNo}", new { scItemPath = "/
Read more