Robust UI tests with Espresso and Idling Resources

A few years back, when I first dived in automated UI tests for Android using Espresso I very quickly noticed the framework lacked any reliable way to wait for an UI element to be displayed.

So I did what all good juniors would do, I added Thread.sleep() everywhere 😅

Fast forward to the present, new project, we want to add integration tests to the app and the Android team now has a recommended approach in Idling Resources.

A quick disclaimer on our setup first, since it heavily influenced how we chose to write the tests.
Early in the process, we decided against using mocks for the server. Our back office has a dev environnent with fixtures that mimic a fairly complex business system and tests users for authentication. It seemed more interesting from a testing point of view to directly access the server API but by doing so, we obviously introduced timing issues due to response time from the server.

This article is going to be our team's feedback of using espresso with Idling Resources and will assume you are familiar with writing Espresso tests.

The idling resource

The first thing to understand with idling resources is that there is going to be some impact in the "non-test" code.
I already see the eyebrows raising and I have to admit I had the same reaction too. Fortunately, there are ways to make that impact as minimal as possible.

The two classes we use are :

class SimpleCountingIdlingResource(
    private val resourceName: String
) : IdlingResource {

    private val counter = AtomicInteger(0)

    @Volatile
    private var resourceCallback:
        IdlingResource.ResourceCallback? = null

    override fun getName() = resourceName

    override fun isIdleNow() = counter.get() == 0

    override fun registerIdleTransitionCallback(
        resourceCallback: IdlingResource.ResourceCallback
    ) {
        this.resourceCallback = resourceCallback
    }

    fun increment() {
        counter.getAndIncrement()
    }

    fun decrement() {
        val counterVal = counter.decrementAndGet()
        if (counterVal == 0) {
            resourceCallback?.onTransitionToIdle()
        } else if (counterVal < 0) {
            throw IllegalStateException(
                "Counter has been corrupted!"
            )
        }
    }
}

and :

object TestIdlingResource {

    private const val RESOURCE = "GLOBAL"

    @JvmField
    val countingIdlingResource
        = SimpleCountingIdlingResource(RESOURCE)

    fun increment() {
        countingIdlingResource.increment()
    }

    fun decrement() {
        if (!countingIdlingResource.isIdleNow) {
            countingIdlingResource.decrement()
        }
    }
}

The second one serving as the entry point to notify Espresso the app is currently busy in a background process and should wait before carrying on with the test.

To make sure none of this is called in production, we used Gradle's product flavors. We already had them setup for other uses so we just had to add a dummy implementation of TestIdlingResource for non-dev flavors:

object TestIdlingResource {

    fun increment() {
        // no-op
    }

    fun decrement() {
        // no-op
    }
}

Updating your app code

Warning, here's the tricky part.
We need to use the idling resource at key moments in the app when it's busy.
I personally try to keep usage of them in fragment classes as to not hinder our unit tests suite.
To use the example of an authentication system, you could have something like:

fabValidate.setOnClickListener {
    TestIdlingResource.increment()
    viewModel.startLogin(user, pass)
}

and

private fun navigateToHomeAfterSuccessAuth() {
    TestIdlingResource.decrement()
    // Call to whatever abstraction
    // of the navigation you're using
}

It's hard to give clear advices here, you'll have to try, test and see what works best for you depending on your architecture.

Setting up the test

Registering the idling resource in the test is done as such:

private var idlingResource: IdlingResource? = null

@Before
fun setUp() {
    idlingResource = TestIdlingResource.countingIdlingResource
    IdlingRegistry.getInstance().register(idlingResource)
}

@After
fun tearDown() {
    idlingResource?.let {
        IdlingRegistry.getInstance().unregister(it)
    }
}

The espresso tests then become fairly straightforward.
Here, it properly waits the end of the authentication process before checking the home screen is displayed:

onView(withId(R.id.login)).perform(replaceText("user")))
onView(withId(R.id.pass)).perform(typeText("123456"))
onView(withId(R.id.fabValidate)).perform(click())
// No need to sleep the thread for seconds here!!!
onView(withId(R.id.home)).check(matches(isDisplayed()))

Hopefully, this article has shown you that you can write reliable UI tests with Espresso and TestIdlingResources 😎

As always, huge thanks to my team for review and feedback :)