code coverage, Robotium, JaCoCo, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA

Code coverage for Android UI Automated Test

Yana Gusti 04/01/2017

Let us talk about Android Automated test. The profession of QA Engineer used to be associated with the people, who are “not good enough” to become a developer. I’ve heard for several times that “the easiest way to get into IT sphere is to become a tester”.

But in fact, it is required to study new technologies and constantly improve yourself to become a professional tester.

1. 2017’s trends in testing:
integration, automation and more

The notion “Quality Assurance” as the separate branch of IT-industry has appeared not so long ago. Full-scale testing  departments existed only within large software/hardware developement companies, where the mistakes were inadmissible.

Revolution in creating the standards has become the next step. It became clear, that certain rules must be obeyed for the correct functioning of IT products. So the support departments appeared to receive and customers’ feedbacks and solve problems.

top qa
QA priorities

Nowadays most CEOs and managers want their products to meet all the requirements.

Companies started to invest more and more in the process of testing, therefore the profession of QA Engineer is just on the start of its development.

code coverage, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
QA Budget

The most popular testing trends in 2016 are the following:

1. integrated testing – to involve testers at the very beginning of development process and throughout the whole lifecycle of a product. As it is said: It is much cheaper to fix bug in a documentation than during the release.

code coverage, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Bug vs Feature

2. automation testing: helps to substitute regression testing, e.g the product is developed using the agile method and so on.

code coverage, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Automation testing

3. cloud sevices, which provide virtual testing tools;
4. crowdsourced testing – testing, which involves a great number of users (effective for beta-testing).

Using these tools is the key to providing better mobile testing services.Future of testing belongs to automation

2. Future of testing belongs to automation

Nowadays automation of processes becomes popular in all spheres of human life.

Society is striving to delegate all the routine responsibilities to machines. Things that appeared to be impossible few years ago, in our time have turned into ordinary processes and testing is not an exception.

Though, according to the data of “STATE OF TESTING 2016”, the percentage of QA Automation Engineers on the market equals only 8 %, more and more clients demand to run the automated tests for checking the programs.

code coverage, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
get from qablog.practitest.com

Unit, UI automation, Load tests, as well as usage of automated testing tools, gain popularity for checking the compatibility on different OS and devices.

But, as it has appeared, that’s not enough. Consequently, if you don’t use the tools for calculating code coverage, automated testing won’t guarantee the testing of all functions, test-cases and scenarios.

Clover, JaCoCo, Code Coverage (the standard tools of code coverage IntellijIDEA) can be used for this purpose. However, most of them were designed for calculation of Unit test code coverage (for example, when starting test via Clover, the message “Clover Test Optimization supports JUnit configurations only” is shown by IDE).

code coverage, Clover, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Clover test coverage for unit tests

Otherwise, there is no ready-made and full-scale solution for presenting the code coverage of UI automated tests on the market.

3. How to present the results of android automated test for UI

We started to investigate these issues on clients demand to present the visual result of our automated tests for mobile apps testing.

After investigating some articles, blogs and forums we found the great project by Oleksandr Kucherenko called Android Test + Espresso + JaCoCo. He has created AndroidJacocoTestRunner, which we have successfully used to get the code coverage of our automated tests.

Unfortunately, Espresso has the range of problems while writing code. For example, this library doesn’t provide the ready-made scroll methods and so on.

This is the reason our team have decided to elaborate Oleksandr’s project for it to be able to interact with the Robotium library. Thanks to AndroidJacocoTestRunner’s source code simplicity and clearness, just a few lines had to be changed:

  • change the path to AndroidJacocoTestRunner in build.gradle

defaultConfig { 
    applicationId "com.example.android"
    minSdkVersion 15
    targetSdkVersion 21
    versionCode 1
    versionName "${ getBuildName() }"
    testInstrumentationRunner 
    "com.example.android.AndroidJacocoTestRunner"
}
  • change dependency espresso to robotium and synchronize build.gradle

androidTestCompile'com.jayway.android.robotium:robotium-solo:5.4.1'

After project has been cleaned and rebuilt, IDE notified about successfully passed configuration 🙂

The team started to write code. Code of the tests that are written with Robotium, doesn’t differ from a usual code and gives us a chance to get a graphic result and there’s no need to write any additional conditions.

For example, the test for authorization in the app has been created:


public class TestLogin extends 
ActivityInstrumentationTestCase2<SplashActivity> {
   private Solo solo;
   public TestLogin() {
      super(SplashActivity.class);
   }
   @Override
   public void setUp() throws Exception {
      super.setUp();
      solo = new Solo(this.getInstrumentation(), 
      getActivity());
   }
   @SmallTest
   public void testProfile() throws Exception{
   //check Logout or not
      if (solo.waitForView(R.id.search_button)) {
         solo.clickOnText("Staff");
         solo.scrollToBottom();
         solo.clickOnText("Logout");
      }
      solo.typeText(0, "username");
      solo.clickOnView(solo.getView(R.id.etPassword_ASM));
      solo.typeText(1, "123456");
      solo.clickOnText("LOG IN");
   }
   @Override
   public void tearDown() throws Exception {
      solo.finishOpenedActivities();
   }
}

To get a demonstrative result, test may be started in two ways:
1. from the command line: gradlew :app:connectedAndroidTest

code coverage, Robotium, JaCoCo, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Run Robotium tests

2. from IDE – create Android Test in Edit configuration, choose AndroidJacocoTestRunner and press “Run”. Start gradlew :app:jacocoTestReport with a command line to get a graphic result.

code coverage, Robotium, JaCoCo, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Run Robotium tests

Result in the form of table (file “index.html”) may be found in <Project name>appbuildreportscoveragedebug directory:

code coverage, Robotium, JaCoCo, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Result of automation testing with code coverage

 Also there is a possibility to get Test Summary:

code coverage, Robotium, JaCoCo, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Result of automation testing with code coverage

and see which test hasn’t passed and why:

code coverage, Robotium, JaCoCo, trends in testing, integration, automation, test coverage, UI, automation tests, Android, EasyQA
Result of automation testing with code coverage

Conclusions

Usage of the tools for calculating code coverage of automated tests increases the speed and efficiency of QA engineer’s work.

There is no need to create manual reports and to demonstrate the effectivness of your work. Customer receives the clear result, tester resigns from the obligation of manual reporting. So both sides are satisfied:).

Though automized testing is much more complicated than manual. So learn it, as it is interesting, extraordinary and promising. And we will continue investigating this topic in our next articles!

You can also use our EasyQA test management tool to use several useful features to improve software quality, such as:

  • build sharing
  • catch crashes
  • get build directly from GitHub or GitLab
  • report bug from app and others.

Register new account for free!