icon-search
end-to-end testing

End-to-end testing Angular 2 applications with Protractor

Jakub Majerz 20.10.2016

Testing is indispensable in writing large-scale, complex applications. It may sound like a cliché but we believe it is worth emphasizing. This is so because as deadlines loom, it is tempting to postpone or completely abandon writing tests. And as the pile of to-do test suites gets larger and larger, it becomes even harder to get round to it. Giving up strict testing regimen quickly turns out to be a double-edged sword: initial gains in time evaporate as changing one part of the app produces errors in other areas. Eventually one becomes afraid to introduce any modifications and development grinds to a halt. Because of the importance of testing, we dedicate this article entirely to one of its flavors, namely end-to-end testing (e2e). Get ready to dive deep into the world of end-to-end testing with Protractor – a ready-to-use framework that Angular 2 comes with!

What is end-to-end testing?

End-to-end testing (system testing) consists in testing complete scenarios in the full system. This is different from other very popular flavor of testing, namely unit testing, where we test only tiny, isolated bits of software. This requires us to set up the whole application, whereas in unit tests we can mock out every dependency of the functionality that we’re testing. So a sample end-to-end test might comprise logging in, navigating to the users section, creating a new user and verifying that a new one has indeed been created. A unit test could, on the other hand, encompass only verifying that we use the proper endpoint when we create a new user. Or that we process the response correctly.

What is Protractor?

Protractor is a framework dedicated to end-to-end testing AngularJS applications. It allows running tests against a wide range of browsers. Since it is tailor-made for AngularJS, it can seep into the application life cycle. And so it knows when a particular asynchronous operation has ended, thus liberating you from the need to explicitly force the browser to sleep or wait for a given event. It uses many technologies and we encourage you to visit the project’s webpage to find out how Protractor works internally. What’s really important for us, is that we need to choose a behavior-driven development test framework in order to write our tests. Protractor supports two right away: Mocha and Jasmine, the latter being the default one. It is also possible to use custom frameworks.

Sample test

We’re not getting into details of how to configure Protractor in this article, but exhaustive information on this subject can be found on the project’s website. Instead, we’d like to focus on how Protractor tests look like. We’ll use Jasmine (as we use it in our own projects) to demonstrate. Assume there’s a website http://sample-ng2-page.com, that has a div with id “welcome-message” and text value of “Welcome Angular 2 Developer!” The snippet below tests whether this is indeed the case:

describe("sample test suite", () => {
  it("should display proper welcome message", () => {
    browser.get("http://sample-ng2-page.com")

    const welcomeMessageLocator = by.id("welcome-message");
    const welcomeMessageElementFinder = element(welcomeMessageLocator);

    expect(welcomeMessageElementFinder.getText()).toBe("Welcome Angular 2 Developer!");
  });
});

Let’s go through this code step by step. describe defines a test suite, i.e. a set of individual tests (specs) while it contains a single spec. Both take two parameters: first is a string with suite/test description and second is a function. In case of describe the function contains definitions of individual tests while it‘s function argument contains individual spec’s code. There can be (and usually are) multiple specs within a describe block.

Now, let’s move on to the it‘s contents. browser.get(url) loads the page with specified address. by.id(idString) defines a locator. Locator is a strategy for searching for a particular element on the page. In our case we’re telling Protractor to look for an HTML element with id equal to “welcome-message”. There are many more ways to locate elements ranging from CSS classes through XPATH expressions to elements peculiar to AngularJS itself such as models or bindings. Having such a strategy, we can use it to get the element we’re after with element(elementLocator). We could have also written it in a more concise way:

const welcomeMessageElementFinder = element(by.id("welcome-message"));

But we chose a more verbose one for clarity’s sake. Now that we have our element we can run some assertions on it. Here we only make sure it contains the expected text. But we can test a great variety of things: particular values, presence of given CSS class, proper display of elements and so on.

Protractor vs Jasmine Elements

In our test code, there are both standard Jasmine elements as well as the ones introduced by Protractor. describe, it and expect all belong to the first group and can be used in any test written with Jasmine (e.g. unit tests). browser, by and element are bread and butter Protractor elements used in virtually any end-to-end test written with the aid of this framework.

Protractor is a wrapper around WebDriverJS and the three main elements mentioned above are actually wrappers around WebDriverJS elements. So by is Protractor’s equivalent of webdriver.By and element is equivalent to browser.findElement. Both Protractor and WebDriverJS use browser to mean the same thing (you can access the vanilla WebDriver browser object via browser.driver).

Control Flow

WebDriverJS (and so, Protractor) APIs are completely asynchronous which means that all functions return promises. But there were no thens in our example spec above. Why is it that we could write:

expect(welcomeMessageElementFinder.getText()).toBe("Welcome Angular 2 Developer!");

and didn’t need to use the “traditional” promise approach? I.e.:

welcomeMessageElementFinder.getText().then((welcomeMessage: string) => {
  expect(welcomeMessage).toBe("Welcome Angular 2 Developer!");
});

It is because WebDriverJS internally maintains a queue of pending promises and makes sure they are resolved properly. This promise queue (called ControlFlow) takes care of synchronizing asynchronous actions. Thanks to ControlFlow you only have to register a promise callback when you need error handling or the promise’s return value. In other situations we can enjoy the cleaner look of code that resembles synchronous flow. We can still use the conventional then approach whenever we want.

Our experience with end-to-end testing with Protractor

We use Protractor to end-to-end test our applications and in this section, we’d like to share our experience. We also outline their pros and cons, mostly comparing them with unit tests, in terms of development, performance and usefulness.

First off, end-to-end tests are slow in comparison with unit tests. While running a single unit test is a matter of milliseconds, completing a single end-to-end scenario can sometimes take several dozen seconds. It is not that surprising: we need to bootstrap the whole application and then simulate real-user interaction. But when we have 100 e2e tests, it makes it impossible to run the whole suite locally before each commit, as is the case with unit tests. This is why we run end-to-end tests on our Continuous Integration server after a build has completed successfully. Only after tests have passed, do we proceed to code review. The chart below shows data from a few historical builds in one of our projects. The difference in execution times is enormous.

Comparison of execution times for unit and end-to-end testing in one of our projects.
Comparison of execution times for unit and end-to-end tests in one of our projects.

 

Secondly, end-to-end tests are costly to develop as they are usually more complex than very focused unit tests. They are also harder to debug as we operate on the whole application and its individual pieces are usually hard to isolate. Moreover, since e2e tests cover more code, they will need to be updated more frequently as there’s a bigger chance that changes in code will break an end-to-end test.

Why bother with end-to-end tests if we have unit tests, then?  Unit tests are great at pinpointing bugs in single parts of the app. But e2e tests give us insight into the way that the complete application functions and allow to simulate real-user interaction. We have also found them to be helpful in uncovering software regressions. There were numerous situations where we were able to catch bugs introduced after a change in some seemingly distant and unrelated part of the application.

Conclusion

Let’s face it: end-to-end tests are not perfect. They are slow, take time to develop and can be a pain to debug. But our experience shows that they are also indispensable in medium- and large-size projects as not everything can be unit-tested. With end-to-end tests we can test arbitrarily complex scenarios of users interacting with our application. This helps us spot bugs in our application and provides at least some degree of confidence that it works as intended (one can never get too complacent in software development!). Fortunately, AngularJS comes with a dedicated framework that understands its quirks and idiosyncrasies and facilitates end-to-end testing.

Komentarze: 0


Notice: Korzystanie z Motyw nieposiadający comments.php uznawane jest za przestarzałe od wersji 3.0.0! Nie istnieje żadna alternatywa. Proszę zawrzeć w motywie szablon comments.php. in /var/www/html/www_pl/wp-includes/functions.php on line 4592

2 odpowiedzi na “End-to-end testing Angular 2 applications with Protractor”

  1. Hi Jakub, thanks for the post, got one question – in case I would use Selenium for testing AngularJS2 application, what kind of difficulties can I have?

    • Hi Andrey, thanks for your question. Since Protractor is built on Selenium, I understand you’re referring to using “pure” Selenium without Protractor. I haven’t tried that myself but I imagine it would be quite troublesome as you would need to deal with Angular’s intricacies (change detction, bindings, etc.) yourself. This is why I prefer to rely on frameworks in such cases. So to sum up: I believe that the main difficulty would be to solve the problems that have already been solved 😉 But this is just an opinion of mine, as I don’t have any hands-on experience with using Selenium alone.

Dodaj komentarz

Twój adres email nie zostanie opublikowany. Pola, których wypełnienie jest wymagane, są oznaczone symbolem *