Effective testing is one of the biggest challenges in the process of software engineering. Especially testing Android applications can be tricky because the code often calls Android specific methods which a normal unit-test cannot handle.
In this article I want to describe the different types of Android tests and how to measure the combined test coverage.
Android Counter
Android Counter is a very simple Android application that I developed to demonstrate different testing strategies.
It allows to increment and reset a counter displayed on the app's screen. The counter's value is persistent across multiple sessions.
The project can be found here: https://github.com/matthinc/AndroidTestDemo
Android counter consists of only three classes:
Counter.kt contains the counter logic itself:
class Counter(private var value: Int) {
fun increment() {
++value
}
fun reset() {
value = 0
}
val counterValue
get() = value
}
The CounterPersistenceManager.kt stores the value of the counter using SharedPreferences:
class CounterPersistenceManager(private val context: Context) {
private val preferences: SharedPreferences by lazy {
context.getSharedPreferences("counter", Context.MODE_PRIVATE)
}
fun persistContent(counter: Counter) {
preferences.edit().putInt(COUNTER_KEY, counter.counterValue).apply()
}
fun loadCounter(): Counter {
return Counter(preferences.getInt(COUNTER_KEY, 0))
}
companion object {
private const val COUNTER_KEY = "counter_value"
}
}
MainActivity.kt connects the the counter logic to the app's user interface:
class MainActivity : AppCompatActivity() {
private val counterPersistenceManager by lazy { CounterPersistenceManager(this) }
private val counter by lazy { counterPersistenceManager.loadCounter() }
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
findViewById<Button>(R.id.increment_button).setOnClickListener {
counter.increment()
updateCounter()
}
findViewById<Button>(R.id.reset_button).setOnClickListener {
counter.reset()
updateCounter()
}
updateCounter()
}
private fun updateCounter() {
findViewById<TextView>(R.id.counter_value).text = "${counter.counterValue}"
counterPersistenceManager.persistContent(counter)
}
}
Testing the Counter
The class Counter does not contain any Android specific code. That means that it can be tested using a "traditional" unit-test.
Unit-Tests are much faster than Android tests so they should be used when possible.
class CounterTest {
@Test
fun testNewCounter() {
val counter = Counter(0)
Assert.assertEquals(0, counter.counterValue)
}
@Test
fun testIncrement() {
val counter = Counter(0)
counter.increment()
Assert.assertEquals(1, counter.counterValue)
}
@Test
fun testIncrementInitialValue() {
val counter = Counter(5)
counter.increment()
Assert.assertEquals(6, counter.counterValue)
}
@Test
fun testReset() {
val counter = Counter(5)
counter.reset()
Assert.assertEquals(0, counter.counterValue)
}
}
The coverage of this test can simply be measured by using Android Studio's "Run Test with Coverage" feature. The generated coverage report shows that the class Counter now has 100% test coverage. Great!
Testing CounterPersistenceManager
The CounterPersistenceManager uses Android's SharedPreferences class which requires a Context. That means that a normal unit-test can't be used to test this class without mocking a lot of stuff.
That means that we have to use Android tests. Android tests work like normal unit-tests except that they have to be executed on a real device or an Android emulator. A valid context object can be optained by using
val appContext = InstrumentationRegistry.getInstrumentation().targetContext
The final test looks like that:
@RunWith(AndroidJUnit4::class)
class PersistenceTest {
@Before
fun resetCounter() {
val appContext = InstrumentationRegistry.getInstrumentation().targetContext
CounterPersistenceManager(appContext).persistContent(Counter(0))
}
@Test
fun testLoadCounter() {
val appContext = InstrumentationRegistry.getInstrumentation().targetContext
val manager = CounterPersistenceManager(appContext)
val counter = manager.loadCounter()
assertEquals(0, counter.counterValue)
}
@Test
fun testStoreCounter() {
val appContext = InstrumentationRegistry.getInstrumentation().targetContext
val counter = Counter(5)
val manager = CounterPersistenceManager(appContext)
manager.persistContent(counter)
val counter2 = manager.loadCounter()
assertEquals(5, counter2.counterValue)
}
}
Testing the User Interface
Now, MainActivity remains the last untested class. The best way of testing MainActivity is an UI-test. UI-tests are a special kind of Android tests. Instead of testing the code directly, UI-tests simulate user interaction to the app's UI and check, whether the results match the app's expected behavior.
The most popular library for writing UI-tests is espresso.
It should be integrated into every Android application by default.
Android Counter could be UI-tested like that:
@RunWith(AndroidJUnit4::class)
class UITest {
@get:Rule
val activityRule = ActivityScenarioRule(MainActivity::class.java)
@Before
fun resetCounter() {
onView(withId(R.id.reset_button)).perform(click())
}
@Test
fun testIncrement() {
onView(withId(R.id.counter_value)).check(matches(withText("0")))
onView(withId(R.id.increment_button)).perform(click())
onView(withId(R.id.counter_value)).check(matches(withText("1")))
}
@Test
fun testReset() {
onView(withId(R.id.increment_button)).perform(click())
onView(withId(R.id.counter_value)).check(matches(withText("1")))
onView(withId(R.id.reset_button)).perform(click())
onView(withId(R.id.counter_value)).check(matches(withText("0")))
}
}
Measuring the Coverage of Android Tests
You might have noticed that the "Run with Coverage" option is not available for Android tests.
The coverage of those tests can be measured by the library jacoco.
Jacoco can be installed by adding the following lines to the project's build.gradle file:
The coverage also has to be enabled in the module's build variants:
buildTypes {
...
debug {
testCoverageEnabled true
}
}
This adds a new gradle task called createDebugCoverageReport.
Executing this task creates the new file:
build/reports/coverage/debug/index.htmlThis html file contains the coverage report for the Android tests. It looks like that:
Combining Unit-Test and Android-Test Coverage
Now we have the coverage of our unit-tests and Android-tests in two different places. It would be much more useful to have one unified coverage report. Luckily, this is also possible using jacoco.
It's archieved using a new Gradle task that runs all unit- and Android-tests and collects the coverage information from both test results.
The task has to be defined in the module's build.gradle file:
plugins {
...
id 'jacoco'
}
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {
reports {
xml.enabled = true
html.enabled = true
}
def fileFilter = [ '**/R.class', '**/R$*.class', '**/BuildConfig.*', '**/Manifest*.*', '**/*Test*.*', 'android/**/*.*' ]
def debugTree = fileTree(dir: "$project.buildDir/tmp/kotlin-classes/debug", excludes: fileFilter)
def mainSrc = "$project.projectDir/src/main/kotlin"
sourceDirectories.from files([mainSrc])
classDirectories.from files([debugTree])
executionData.from fileTree(dir: project.buildDir, includes: [
'jacoco/testDebugUnitTest.exec',
'outputs/code_coverage/debugAndroidTest/connected/*coverage.ec'
])
}
Running this task creates another test report in
build/reports/jacoco/jacocoTestReport/html
which contains the combined coverage of all tests:
Tests and Coverage using GitHub Actions
For automated testing and coverage calculation we can use GitHub's integrated CI/CD service.
To active GitHub actions for the project we add the following file .github/workflows/android.yml to our project:
name: Android CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: set up JDK 11
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
- name: Grant execute permission for gradlew
run: chmod +x gradlew
- name: Build with Gradle
run: ./gradlew build
This triggers the Android build on every push to the project's main branch.
GitHub itself can't measure test coverage.To do so we need an external coverage service. There are various options available but for this demo, I decided to use Sonarcloud. In addition to test coverage, Sonarcloud can also measure different code-quality metrics which is quite handy.
I created the project android_test in sonarcloud for this demo.
I also created an access token so that our GitHub Action can access this project later.
Now, I added the following build job to android.yml:
android-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: reactivecircus/android-emulator-runner@v2
name: Run tests with coverage
with:
api-level: 23
target: default
arch: x86
profile: Nexus 6
script: ./gradlew jacocoTestReport
- name: Remove build.gradle
run: rm build.gradle
- name: SonarCloud Scan
uses: sonarsource/sonarcloud-github-action@master
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
args: >
-Dsonar.coverage.jacoco.xmlReportPaths=app/build/reports/jacoco/jacocoTestReport/jacocoTestReport.xml
-Dsonar.organization=matthinc
-Dsonar.projectKey=android_test
-Dsonar.sources=app/src/main/java
You have to change the parameters sonar.organization and sonar.projectKey to match your Sonarcloud project.
After adding the SONAR_TOKEN as a secret in GitHub's repository settings, the build of the project succeeded.
The coverage for this buid can be checked in Sonarcloud:
