dry-rb is a fascinating set of tools and libraries, but their usage may not be readily apparent. Why might one add these libraries when there are perhaps much simpler techniques that will suffice? What is the reasoning behind the abstractions, and what gains might they afford us that could make a case for their inclusion?
Well that's what this series is. We're going to take a look at a few more non-trivial examples and explain the reasoning behind why one might want these tools, because there are most certainly cases where the incurred complexity becomes very cost-positive indeed.
In order to build the intuition there though, we first need to explore a bit of theory.
Shapes and HTTP Responses
Perhaps one of the most powerful single concepts in programming, and one we so often take for granted, is the HTTP Response and HTTP response codes. With REST/JSON endpoints we can be (reasonably) sure we're going to get back a response which looks something like this:
HTTP::Response(body: "<json content>", code: 200)
Name 100 different APIs that use REST and JSON and you can name 100 that very closely follow a similar convention to this. These responses come with a very clear "shape" that allows us to make reasonable assumptions on how we might act on them, such as with a Rack Response:
response = get_data # Rack::Response returned
response_body = JSON.parse(response.body)
if response.successful? # Rack::Response::Helpers
process(response_body)
else
handle_errors(response_body)
end
This is exceptionally powerful, underlies so much of our modern web, yet is so frequently taken for granted. Encapsulated in this single idea of a response we have both data, as well as the context for the data such as whether the request was a success, an error, maybe a bad form, who knows but it's consistent (as long as we play by its rules.)
Rule Breaker
Now there is indeed a danger, alluded to above, that this presumes reason and playing by a set of rules that allow us to make those assumptions. There are some APIs which might do something like this:
HTTP::Response(body: "ERROR! IT BLEW UP!", code: 200)
By doing that we've violated reasonable expectations and have lost trust in the interface, so it's up to us to be vigilant in compliance to those rules as the power of that interface is well worth the extra overhead of adopting it.
Without Context
Let's say that we did away with codes for the moment, and a response only contained a body. Take a moment, before continuing, to muse on why that may make things substantially more complicated to use.
Got it? Well let's take a look into a few implications of removing that additional context to the data.
How do you define success? An error? Is it somewhere in the body? What if the response is empty, nil, falsy, or any other number of shapes? The JSON could very well take any number of these shapes:
# Inline status
{ content: "<data>", status: "success" }
# Empty response
{ }
# No response
""
# String error
"Error: Something went boom"
# Number code
123456
Now take that potential inconsistency of how we define success and stratify it across every API in the world that plays (mostly) nicely by REST and JSON standards (oh yes, JSON has rules too) and you can see how this quickly becomes a headache.
Every single API from here on out that you consume plays by its own unique rules, and as a result inflicts the cost of implementing error and success handling on the user rather than on the service. What a painful world that would be to work in, especially if you find yourself consuming several APIs at once and creating wrappers around each to make them play nicely in your own domain.
Oh, and those wrappers that you'd have to write? You'll very likely end up with some vaguely standardized response type all your own, perhaps or probably still inconsistently, within your own application.
Cross-Company Coupling
The more systems are coupled together, and the more they require intimate knowledge of the underlying technologies to work with them, the more difficult you will likely find it to interface.
If a company were to create a new pseudo-standard on top of HTTP like that above 200 Error bit they have now directly coupled all of your code with their own system for processing errors.
You're now required to write a wrapper with knowledge of their own strange implementation to get it to reasonably work within your application, as mentioned above, which heavily couples you to their definition of success.
I'll give you a hint to the next major section: Imagine that internally in your own application / service mesh and how pesky that might be.
Consistency
While there is indeed a cost associated with adopting the HTTP Response standard there is undeniably a benefit we derive from playing by its rules. That's why web servers, Rack included, use it.
It doesn't matter what the underlying data looks like, but it does matter that we can expect a consistent response from any reasonable API adhering to the standard with the same shape and agreeing to the same language of what it means when we say success.
You go to any company, work on any API, consume any API, or otherwise in this arena and you'll very likely find things very familiar. That's powerful, so incredibly so, and yet we take that for granted.
APIs aren't Only External
That's all well and good, but how does that apply to our applications? Why is it relevant and why should we care about this consistency of shapes and HTTP responses?
Quite simply because API means public interface, and that's not necessarily talking about external interfaces either. Every public method is an API to your application, class, library, or what have you.
Consider the above for a moment, and think of all the ways you might reflect failure inside of your application, then continue along.
What Does it Mean to Fail?
What did you come up with? Probably several different ways, each with a myriad of assumptions and requirements for working with them. Let's look at a few real quick:
# Exceptions - String based
raise "It failed!"
# Exceptions - Class based
raise MyCustomSpecificUsefulError, "Something went wrong"
# Returning false or nil (what if falsy is valid? That's fun)
false
nil
# Reasonable empty defaults
""
[]
{}
# ...and probably many more
Even worse is when you realize this same headache applies to what it means to succeed, but we'll skip that one for now.
You can see how each of these likely has very interesting implications on code that relies on that interface. Now you have to know about all the underlying exceptions, or perhaps whether or not falsy returns are valid, or in the more useful case like with Enumerable methods you get back a reasonable empty result like so:
[2, 4, 6].select { |v| v.odd? } # => []
That last one is particularly interesting as we can continue chaining things on the end of that select, and if it actually had data it'd continue down that pipeline. Sounds useful, no? We'll get back to why that's critically important here in a moment, but first...
Exception Handling
What if select instead raised an exception like this:
# Please don't actually do this to `select`:
module Enumerable
def select(&block)
found_elements = []
self.each do |element|
# If calling the block on the element is truthy, add it
found_elements.push(element) if block.call(element)
end
raise NoResultsHere, "No data!" if found_elements.empty?
found_elements
end
end
Can you imagine working with that? You'd probably have to do something like this:
begin
[1, 2, 3, 4].select { |v| v > 5 }
rescue NoResultsHere
[]
end
Now make every Enumerable method do that instead, and we've inflicted that error handling on consumers making it a much more unpleasant interface to work with.
The Power of Enumerable
The fact that select and other Enumerable methods continue to return Enumerable shaped objects is really something quite useful. It even lets us do something like this:
(1..100)
.select { |v| v.even? }
.map { |v| v * 5 }
.group_by { |v| v > 50 }
.transform_values { |vs| vs.sum }
# => {false=>150, true=>12600}
You get all of that because Enumerable agrees on what shape of object to return, allowing you to chain as you will, and allowing you to even route around failure in a way to achieve results.
This entire concept of reasonable defaults has a name, and that name is "Identity". It's even a law, fancy that, of something much more interesting when joined with two other laws.
If you haven't yet, stop and read Introducing Patterns for Parallelism for Ruby. I promise you will find it quite interesting with where your intuition might currently be going with these shapes I've been on about.
Those shapes have a name, and that name is "Monoid". Honestly though the name doesn't matter as much as the intuition that these like-typed things can be combined and chained together however we wish. Once you have that, well there are some very interesting possibilities which open up.
And that? That my friends is where dry-rb enters the stage with the Result type.
Introducing Our Friend Result
I've spent a good bit of time leading into this, and I do apologize for that, but as a friend once said it's far easier to show the value of painkillers rather than vitamins. In a way you have to experience the absence of something to appreciate what value it might bring, rather than being immediately extolled on its inherit virtues.
Setting the Stage
Let's set the stage right quick. You happen to be in a large Rails application that follows along with something like Packwerk to clearly delineate different packages in your Rails monolith. Let's say you have 100 packs, which is not particularly unusual with larger applications.
Now take all 100 of those packs, and give them public interfaces with minimal coupling. We'll say those happen in the pack in a file like public_api.rb or something similar, go wild really. What matters is there's a single entry point to each package.
Opening one of those public APIs we might find something like this:
# packs/app/public/service_name.rb
class ServiceName
class << self
def offering_a; end
def offering_b; end
def offering_c; end
def offering_d; end
def offering_e; end
end
end
All of which reach into the internals of the pack to do very important businessy things, probably some ActiveRecord, and return back a result.
Remember that bit about defining failure, and the hint on defining success as well? Oh yes:
# packs/app/public/service_name.rb
class ServiceName
class << self
# Exception
def offering_a(id)
ServiceModel.find(id)
end
# Nil
def offering_b(id)
ServiceModel.find_by(id: id)
end
# False
def offering_c(id)
ServiceModel.find_by(id: id) || false
end
# Empty Collection
def offering_d(*ids)
ServiceModel.where(id: ids)
end
# Invalid Object
def offering_e(**model_info)
ServiceModel.create!(**model_info)
end
end
end
Every single one of them has both an idea of what constitutes success and failure. Now multiply that by 100, where every service takes their own unique and probably very valid approach to this problem and you can imagine it becoming very difficult to reason about. Doubly so when you rely on multiple services input to produce your own output, that means you get all of those services downstream too.
Now every bit of that error handling or failure handling has now been imposed on the consumer, and you're right back to square one with coupling despite having packages and very firm lines between them. Even returning direct ActiveRecord objects or collections introduces coupling, but that's another matter we won't touch for the moment.
What if, instead, they agreed on a singular definition and shape of what it means to succeed or fail? Well that's where Result comes in.
The Result Type
dry-rb presents a very interesting solution called Result to this problem, along with very lovely documentation I suggest you wait a moment before reading that you can find here. Ah, and don't mind that "Monad" word, it's not important for the moment.
It gives us the idea of Success and Failure, the two parts that comprise the larger Result type:
require "dry/monads"
extend Dry::Monads[:result]
result = if foo > bar
Success(10)
else
Failure("wrong")
end
Think of Success and Failure almost like that additional context provided by HTTP Responses status code. It's a wrapper that clearly indicates to us whether something is considered successful, or if it had failed somewhere.
Like Enumerable, we can chain Result types together because their reasonable default is still a Result type:
require "dry/monads"
extend Dry::Monads[:result]
# Pretend it's a DB of some sort
IDS = { "a" => "Red", "b" => "Blue" }
def offering_a(id)
return Failure("ID not found: #{id}") unless IDS.key?(id)
Success(IDS[id])
end
offering_a("a")
# => Success("Red")
offering_a("nope")
# => Failure("ID not found: nope")
# Remember Enumerable? What happens if we chain each of these?
offering_a("a").fmap { |name| "We found #{name}!" }
# => Success("We found Red!")
offering_a("nope").fmap { |name| "We found #{name}!" }
# => Failure("ID not found: nope")
For Result types we can call a special method called fmap on them to do things to the value inside, but if it's a failure? It does nothing, the failure persists through and we get it out at the end.
It can be far more interesting, though, when applied to pattern matching in Ruby 2.7+:
result = offering_a("a")
case result
in Success("Red" | "Blue") then "Found who we were looking for"
in Success then "Not who we expected, but still ok"
in Failure(/_why/) then "He's still in our hearts though"
in Failure then "I give, you win"
end
We can reach inside each of those values based on the fact it's a Result type, or really we don't even need to reach at all:
result = offering_a("a")
result.value_or("Yellow")
Though perhaps you want the more familiar if branch:
if offering_a.success?
There's quite a bit you can do with this type of concept, but as with an HTTP Response because we've agreed on that shape and if all of our services play nice with it all of the consumer code can make those same assumptions and handle them the same way.
Hey, if the shapes all line up, that even means we could combine them or do all types of other interesting tricks like say how Promise or Future works with async in Javascript, or how Task could have a transactional execution list, or maybe Validated combining multiple errors when trying to create something that's not quite right.
The collective mental burden that alleviates when navigating a codebase pays dividends over time, just as you very likely never spend much time thinking about HTTP Responses either.
Good code is useful, but great code is transparent. HTTP Responses are transparent, and I would say that Result types very may well be too.
Yeah, but the Underlying Data
Record screech
Yeah, about that one. You have the same shape, sure, but that doesn't necessarily absolve you from worrying about the data within it. The container may be great and make sense, but that doesn't mean the content is too.
If you're trying to keep clear boundaries across packs it's probably not wise to directly hand them an ActiveRecord object that can still directly access the database, or that cannot be serialized, or maybe even validated beyond the context of the database. That's only the success branch.
The failure branch is even more fun. Are we doing String errors now? What about the context, the backtrace, what params it was called with, or any of that other lovely meta information? Granted Result comes with Tracing Failures, but not the point.
Standards
The larger the application the more that standards start to become force multipliers rather than annoying rules that someone in infrastructure told you to follow. The part left unsaid there is that defining and creating those standards, especially for something like errors, is really really danged hard.
There are hundreds if not thousands of possible approaches to these problems, but overall what I've found? Pick one, build on it, and as long as you can agree on what the edges look like you can work from there.
For me things like Promise, HTTP Response (and even Request), Result, and other such types make great edges to start from.
Wrapping Up
Rails wasn't built in a day, and neither will your application. It's a continual and evolving process where we learn and grow, and with our understanding so to changes our opinions. If anything the best advice is to make it easy to change, easy to recover when you fail, and minimize how much systems know about each other.
Perhaps my opinions will change one day too, they have several times already, but that's about where I'm currently at.
dry-rb, to me, presents a set of tools which define edges capable of bearing the weight of larger applications and reducing coupling and complexity. Are they free? No, but nothing ever is. Are they worth it? Well that'll be for you to decide, but I would say they are.
I'll be taking a look into the dry-rb libraries more in this series and taking a similar approach of building a case rather than telling you why things are so great or amazing. We're not here for shiny, we're here to solve problems, and perhaps this will solve yours.
Top comments (2)
Yes! I recently built an non-web, CLI application which needs to get data from a db, get data in and out of Redis, use several AWS services, and use results from another locally developed application. And I wanted it to be able to take advantage of parallelism where possible. Nothing near the scale of what you are describing, but maybe the most ambitious project I've taken on. Oh, and the shape/requirements of the project kept shifting as I built it.
I ended up using dry-monads and basically having every callable class in the thing return a Result. I'm sure I used it clunkily and in some cases it was overkill. However, it was the least frustrating development experience I've ever had, I think. When requirements changed or were added I could just combine existing classes with new ones in different ways without worrying about implicit assumptions built in.
I'm sure there are plenty of other ways I could have built the things but adhering to the assumptions of dry-monads made it easy
Wow.. you had me even with the long intro. I bet applying necessary dry.rb libraries to a personal project will reap benefits over the lifetime of the app.
dry-rails packages like nine of the dry.rb libraries and feels like a great place to enter the ecosystem. Thanks!