Is it possible that you've undervalued or overlooked stories in your backlog that will return thousands of dollars on every hour of your effort? Are you sure you're working on the most important thing?
That's exactly what happened to me. I'd been ignoring a bunch of "low priority" stories until I discovered that a similar story returned well over $15K per hour of my time.
In this post, I'm going to tell you what happened, how you can find those kind of stories in your own backlog, and how you can make sure you're working on the most important thing.
I made a landing page that effectively prints money
About 5 years ago I got interested in keyword analysis. So, I spent a little time investigating it between work assignments. I soon discovered a really great keyword that didn't appear anywhere on our site. So, I quickly built a landing page, made an ad group for it on AdWords, and started pushing traffic to it.
My landing page was ugly. It was just a bunch of benefits I threw together and a couple of calls to action. Because it was an experiment, I didn't spend much time on it. Yet, despite its amateurish look, we soon learned that it converted like crazy--way beyond anything else we had in AdWords. I was proud of myself and thought about taking it further. But I had lots of other work to do so I moved on to other things.
The years passed and I continued to be proud of my landing page. It was always our best ad group--by far--and it was doing well in organic search too. But I was busy so I left it alone.
I waited 5 years to assess the performance of my landing page
An unrelated work assignment prompted me to finally assess my landing page. And I was floored when I learned that my ugly landing page has made us in excess of $15K per hour of effort (so far). And that its annual revenue is increasing every year. So, there's no telling how much money that one landing page will make my employer over its lifetime.
Now, my goal is to always be working on the most important thing. And I thought I was doing pretty well at it. But given the rate of return on this one landing page, I'd have to say that I haven't been working on the most important thing because I don't do anything on a regular basis that brings in five figures per hour.
So, I spent some time looking at new keywords (I'd be crazy not to, right?). And after a couple of hours I found a bunch more that look like they have the potential to be in the same league as our best keyword.
I'd been ignoring a huge opportunity for years.
How to ensure you're working on the most important thing
We knew that my landing page was good. We talked about it every time we talked about AdWords or Google Analytics. I even built another landing page on different keyword that my boss and I brainstormed one day. But we never actually sat down and calculated the value of my landing page. Other tasks always seemed to be a higher priority.
It's difficult to identify the most important thing because the solution space is huge and it's difficult to compare stories. But I've found The Theory of Constraints to be a helpful way to quickly zero in on high value areas without looking at the whole business in excruciating detail. And last year I learned about cost of delay and CD3, which combine very nicely with the Theory of Constraints. I wrote about these concepts here, here, and here. And yet I still failed to find the most important thing for at least 5 years. What gives?
You need to close the loop
I recently read The Lean Startup and I think Eric Ries has the answer. He spends a good portion of the book talking how important it is for startups to gain something he calls "validated learning" by engaging in a "build-measure-learn" cycle. Well, I did the build. But I didn't measure or learn for five years (author comically applies palm to forehead).
So, my solution is simple. Use the Theory of Constraints, Cost of Delay, and CD3 to prioritize your work. But also take the time apply a build-measure-learn feedback loop whenever you can.
You must convert your results to dollars
CD3 forces you to convert your stories to dollars and that's the key to this whole thing.
We were able to turn inaction into action when we calculated an actual dollar value of my landing page. We knew it was good but I do lots of things that have good results so it didn't really stand out. But when we attached a dollar value to how "good" my landing page is, both my boss and I were instantly convinced that we needed to make more landing pages if we can find more similarly attractive keywords.
Without a dollar value attached to your outcomes, its impossible to accurately calculate the relative "goodness" of different stories. For example, should you spend 5 hours squashing bugs or 5 hours adding a new feature to your software? See the problem? Your whole backlog is filled with things that are difficult to compare. That's why you need to prioritize your backlog by CD3.
Update your estimates based on the results of your validated learning
Finally, I think Ries' build-measure-learn feedback loop hints at something more subtle. The value of each story in your backlog relies on two estimates.
You must estimate the value of the story to your customers and you must estimate the cost of implementing that story (possibly adding some maintenance costs as well). You use these two estimates to determine the ROI of your stories. But you won't know how accurate either of those estimates are until you complete the story, get it in front of your customers, and analyze your results.
So, here's the important bit. If you quantify both the value of your stories and your actual effort in dollars, you can calculate two things:
- how close to optimal you ordered your stories in the past
- if you need to update your value and/or cost estimates for similar future stories
With this data in hand, you can prioritize your backlog more accurately and ensure you are--to the extent possible--always working on the most important thing.
Resist the urge to use your gut to prioritize your backlog
I know the cowboys software developers out there will want to ignore all this analysis stuff and just get down to coding the next thing. That's a mistake that will lead you to develop needlessly expensive software in the best case, worthless software in the average case, and no finished software at all in the worst case.
The value of your stories likely follows a power law distribution, which has significant consequences.
You need a sound method of consistently identifying those few truly valuable stories at the far left side of your distribution. Without such a method you'll waste the majority of your resources working on low priority stories. You need CD3 if you don't want to be mediocre.
Takeaways
The more I learn about project management, the more I realize the extent to which managers are needlessly making decisions based on gut feelings and delivering poor results. In some cases, following your gut is absolutely the right thing to do. But for the vast majority of business decisions, we are just engaging in sub-optimal behavior when we don't take the time to do the 'measure-learn' phases of the 'build-measure-learn' feedback loop.
It literally took me less than five minutes quantify the value of my ugly landing page. I don't really have any excuse for not doing it sooner other than that my focus was divided among the other three hundred things in our backlog. But now that I've looked, we've moved landing pages way up in our backlog.
So, dear reader, are you working on the most important thing? If so, how do you know? If not, tell me why in the comments.
Top comments (8)
I love your posts Blaine. Great as always.
Thanks for the positive feedback, Ben.
Very sound advice. As programmers, we tend to go with what we think is the most important thing, without any benchmark or measure. Need to be making educated decisions to make the decisions effective.
Yep, a friend once told me that "we have to develop things for people (our target people), and not for us at all ..." as developers we sometimes forget what our target market is or what they really want; That usually happens more when we work in a personal project.
Good point, Juan.
If you'll allow me to paraphrase from the book Code Simplicity (by Max Kanat-Alexander): "The purpose of software is to help people.
Yes. You're exactly right.
Our intuition is terrible for this stuff.
For example, my boss and I often have different perceptions about how well a marketing initiative is performing or how profitable something is. He sees things from his side and I see them from mine. But, in the cases where we have data to analyze, we often find out we were both wrong.
It's exciting and frustrating at the same time but I'm sure glad I took those stats courses in university.
I've never worked anywhere I could just throw up an experimental page on a production site to see what would happen, and I've never worked anywhere where suggesting to the people in charge, "hey, it might work better if we did X" was particularly effective.
As programmers, we tend to go with what we're told to work on. If you're in a role where you do that kind of analysis as well as development, then is it because it's a very small company where while you have some autonomy, you're too overworked to find time for side projects?
I guess I'm fortunate then to be at a smaller company, where developers do get a chance to voice suggestions and to try some things out.
This is a problem that the business tries to solve with DevOps or other cross-department techniques. Us devs, left to our own devices, will gravitate towards things that make sense to us, which isn't always the most effective use of our time, in terms of business value.