Tutorial: Kotlin Coroutines, Retrofit and fore

This is part of a series on android fore

Tutorials in Series
1) Tutorial: Spot the deliberate bug
2) Tutorial: Android fore basics
3) Tutorial: Android architecture, full todo app (MVO edition)
4) Tutorial: Android state v. event
5) Tutorial: Kotlin Coroutines, Retrofit and fore

Thread based networking

If you've been using fore to wrap your Retrofit networking calls, you'll be familiar with the Java / Thread based CallProcessor:

callProcessor.processCall(
  service.getThingsFromTheInternet(),
  workMode,
  things -> handleSuccess(things),
  failureMessage -> handleFailure(failureMessage)
);

You would call this from a model (or repository, or usecase etc) and what it does is to insulate the code that we are in, from any specific considerations to do with HTTP and network connections.

At this level

• we don't want to care about HTTP 200s : we care about the things we get back
• we don't care about 403s : we care about the VIDEO_ACCOUNT_SUSPENDED failureMessage (for instance)
• we don't care about IOExceptions : we care about the NO_CONNECTION failureMessage.

(failureMessage is your own enum or other class that needs to implement a certain interface, and you decide exactly what the values are and what they represent, you just don't do it here, but as part of the api layer).

We can also easily test this code by mocking callProcessor to return whatever things, or whatever failureMessage we want.

(If you're not familiar with this, you can take a look at the retrofit sample app with tests that's in the fore repo.)

It all works quite nicely, but you can see the potential for nested callbacks if you want to chain more than one network request together. So we'll take a look at fore's Coroutine version of this class next.

---

Coroutine based networking

The Kotlin / Coroutine based CallProcessor has a slimmer API:

val deferredResult = callProcessor.processCallAsync {
    service.getThingsFromTheInternet()
}

when (val result = deferredResult.await()) {
    is Left -> handleFailure(result.a)
    is Right -> handleSuccess(result.b)
}

The main difference is that the callbacks have gone. Instead we return a Deferred<Either<F, S>> from the processCallAsync() functions, or a straight Either<F, S> from the processCallAwait() functions. (The Retrofit service is also now defined as suspended).

Obviously this version isn't driven by threads, the network calls are run with Dispatcher.IO and the results will be returned to you on Dispatchers.Main, so like the Java CallProcessor you won't need to do anything special to get back to the UI thread.

---

Quick refresher on "callback hell"

Let's back up a little and remind ourselves of one of the reasons why Coroutines are cool: they help us to write asynchronous code as if it was synchronous (similar ideas exist in lots of languages of course).

The idea is to avoid what's commonly known as "callback hell" which can sometimes happen if you need to write a lot of nested callbacks and you're not careful about it. So with Coroutines we can go from writing code like this:

callService1("input") { response1 ->
    callService2(response1) { response2 ->
        callService3(response2) { response3 ->
            doSomethingElse(response3)
        }
    }
}

(I realise this nested code doesn't look too bad! - but imagine those nested callbacks spread out over a few pages, with lots of code in between, then it can start to become a problem.)

to writing code like this

val deferredResult1 = CoroutineScope(Dispatchers.IO).async {
    callServiceA("input")
}

val deferredResult2 = CoroutineScope(Dispatchers.IO).async {
    callServiceA(deferredResult1.await())
}

val deferredResult3 = CoroutineScope(Dispatchers.IO).async {
    callServiceA(deferredResult2.await())
}

CoroutineScope(Dispatchers.Main).launch {
    doSomethingElse(deferredResult3.await())
}

---

The CallProcessor and carryOn

But here's a little snag: because network connections sometimes fail, you'll notice there is still a place where we branch for success or failure (and therefore the potential for nesting when we chain more than one network request together - even though we're using coroutines).

when (val result = deferredResult.await()) {
    is Left -> handleFailure(result.a)
    is Right -> handleSuccess(result.b)
}

Does that mean we're back to where we started and could end up in call-back hell again? Well no actually, especially when we look at what we can do with Kotlin.

A good example is the carryOn extension function that we have in the fore-network-kt package.

TLDR;

The CallProcessor API takes a functional type, we can write whatever we want inside that function as long as it ends up returning the type that the function is expecting. And carryOn will let you write code like this:

val deferredResult = callProcessor.processCallAsync {
    service.doSomething0().carryOn {
      service.doSomething1(it)
    }.carryOn {
      service.doSomething2(it)
    }.carryOn {
      service.doSomething3(it)
    }

}

And even though it doesn't look like it at first glance, carryOn completely covers you for any error cases you might encounter from any of the network calls.

The full explanation

Let's suppose our service looks like this:

interface MyService {
    suspend fun doSomething0(): Response<Int>
    suspend fun doSomething1(number: Int): Response<String>
    suspend fun doSomething2(text: String): Response<Double>
    suspend fun doSomething3(dNumber: Double): Response<Float>
}

If we needed to chain those calls together it might look like this:

suspend fun attempt1(svc: MyService) {

  var response: Any? = func {

    val error: Any?
    val resp0 = svc.doSomething0() //returns Response<Int>

    if (resp0.isSuccessful) {
      val resp1 = svc.doSomething1(resp0.body()!!) //Response<String>
      if (resp1.isSuccessful) {
        val resp2 = svc.doSomething2(resp1.body()!!) //Response<Double>
        if (resp2.isSuccessful) {
          return@func svc.doSomething3(resp2.body()!!) //Response<Float>
        } else {
          error = resp2
        }
      } else {
        error = resp1
      }
    } else {
      error = resp0
    }

    error
  }
}

The first call to doSomething0() is going to return an Int wrapped in a Retrofit Response object: Response<Int> (which we assign to resp0).

If that call fails, then we return the error and stop, if the call is successful, then we use response Int to call doSomething1(). This time we get a Response<String> back and the pattern continues until we finally return the Response<Float> object we get from doSomething3 (which may or may not have succeeded).

Apart from the nesting, we loose type information here, func expects to return a Response<Float> (whether it succeeded or failed), but further up the chain we may only have a Response<Int> (for example, if doSomething0() failed).

suspend fun attempt2(svc: MyService) {

  var response: Response<Float> = func<Float> {

    val resp0 = svc.doSomething0() //returns Response<Int>

    val resp1 = resp0.body()?.let {
      svc.doSomething1(it) //Response<String>
    } ?: run {
      return@func Response.error(resp0.errorBody()!!, resp0.raw())
    }

    val resp2 = resp1.body()?.let {
      svc.doSomething2(it) //Response<Double>
    } ?: run {
      return@func Response.error(resp1.errorBody()!!, resp1.raw())
    }

    val resp3 = resp2.body()?.let {
      svc.doSomething3(it) //Response<Float>
    } ?: run {
      return@func Response.error(resp2.errorBody()!!, resp2.raw())
    }

    resp3
  }
}

This is a little better, we've used Kotlin to lose the nesting and we also keep the Type information - remember that the generic of the Retrofit Response objects only matters for the success case where you return a String or an Int etc, the errors are all the same and aren't concerned about the generic. This lets us create a synthetic Response<Float> error, using the data we got from our failed Response<[whatever]> objects.

But with a little extension function, we can take care of making this synthetic Response error. Hence the carryOn function:

suspend fun attempt3(svc: MyService) {

  var response: Response<Float> = func<Float> {

    val resp0 = svc.doSomething0()

    val resp1 = resp0.carryOn {
      svc.doSomething1(it)
    }

    val resp2 = resp1.carryOn {
      svc.doSomething2(it)
    }

    val resp3 = resp2.carryOn {
      svc.doSomething3(it)
    }

    resp3
  }
}

And to tidy it up a little further:

suspend fun attempt4(svc: MyService) {

  var response: Response<Float> = func<Float> {

    svc.doSomething0().carryOn {
      svc.doSomething1(it)
    }.carryOn {
      svc.doSomething2(it)
    }.carryOn {
      svc.doSomething3(it)
    }
  }
}

Here is a more complicated example, as part of fore's CallProcessor

callProcessor.processCallAsync {

    var ticketRef = ""
    ticketSvc.createUser() //Response<UserPojo>
    .carryOn {
      ticketSvc.createTicket(it.userId) //Response<TicketPojo>
    }
    .carryOn {
      ticketRef = it.ticketRef
      ticketSvc.getEstWaitingTime(it.ticketRef) //Response<TimePojo>
    }
    .carryOn {
      if (it.minutesWait > 10) {
        ticketSvc.cancelTicket(ticketRef) //Response<ResultPojo>
      } else {
        ticketSvc.confirmTicket(ticketRef) //Response<ResultPojo>
      }
   }
}

So the carryOn extension function you'll find in fore is just one example of how to use Kotlin and Coroutines to prevent getting nested callbacks - the same techniques can be used in your own code to banish callback hell.

Just to be clear here, carryOn is not a magic bullet - there are restrictions when you use it: you're still required to return a single object type wrapped in a Response object at the end of the function (in this case Response). But it handles moderate chain complexity pretty well, and together with the CallProcessor it does cover you completely for any error cases (which is surprisingly not the case for a lot of commercial networking code I've encountered over the years).

If your chains are more complicated than this, it could be time to roll up your developer sleeves and write some software to coordinate the process explicitly.

Update: fore now also supports Ktor and Apollo using the same CallProcessor and carryOn techniques discussed here, see the fore repo sample apps for each of those libraries.

---

As always, there is a full example app in the fore repo which includes a comprehensive set of unit tests and instrumentation tests.

Thanks for reading!