DEV Community

Cover image for Why I don't like story-point-driven estimates

Why I don't like story-point-driven estimates

Tomasz Łakomy on October 05, 2020

Before we start - I'm working on https://cloudash.dev, a brand new way of monitoring serverless apps 🚀. Check it our if you're tired of switching b...
Collapse
 
190245 profile image
Dave

It seems to me that your issue isn't story points themselves, but the usage of them.

Story Points (in combination with Team Velocity) - allows PMO to forecast delivery, to know if the set of requirements is achievable by the deadline or not.

Story Points isn't, and shouldn't ever be, a way to measure developer productivity (either within a Team or between Teams). There's better tools for that job (such as Git Quick Stats).

Collapse
 
anortef profile image
Adrián Norte

dev.to/anortef/explaining-scrum-st...

Story points are very useful to predict how much complexity can a team handle and for that you apply a rule of thumb, in my workplace it goes about this:
1 point - almost done, there is no unknown complexity or significant amount of grind to do.
2 points - A story with very little complexity.
3 points - A story with little complexity.
5 points - A story that has few uncertainity but still some investigation has to be done.
8 points - A story that has a lot of uncertainity.
13 points - Break that story because it has way too much complexity.

Now, the part about the discussions, that tells me two things: Your team needs to work way more about sharing domain knowledge and that you do not have someone in the role of Scrum master to say "enough pointless discussion, we go with X".

And the final part about story points is that it allows upper management to have some sort of quick feedback about the team performance and when something will be done because a team that cannot give feedback or realistic estimations is a useless one for management.

Collapse
 
bloodgain profile image
Cliff

This is pretty similar to how my team(s) has/have bid, though we haven't written it out as such. And we almost always split a story that requires investigation before implementation into 2 stories. And if a story requires multiple people, that'll add 1.5x to 2x the points, depending on how involved they must be.

That said, I think if you didn't need to put stories "on sprint" based on a nebulous velocity number, it would be easier to just use words to describe a story's size or complexity: simple todo/reminder, trivial, small, medium, large, jumbo (aka SYNS: see you next sprint), all hands, or split

Collapse
 
aminmansuri profile image
hidden_dude

I like a power of 2 point system: 1, 2, 4, 8
If it's larger than 4 it must be split.

But we've done fine by asking people to estimate features in days, and if it takes more than 5 days split it.

Then once you have everyone's estimates, multiply by 2.

I can seem pessimistic, but I've found that it is a very good estimate of ship dates.

We recently had a very high pressure project where we were able to measure the actual multiplier for the project. And because the team was very good. It was 1.51.

So this works well.

I do like the idea of being able to measure if we are speeding up or slowing down with velocity. And velocity helps normalize time to make up for different speeds from different team members. But working off days works well as well.

BUT ONE THING IS CLEAR: this is something that should be taught in Universities.

Collapse
 
bloodgain profile image
Cliff

This approaches Joel Spolsky's estimation adjustment technique, though his involves measuring it for every developer over time:
joelonsoftware.com/2007/10/26/evid...

I like the estimate in days approach for some teams. However, I always insist on the rule that no story can be less than 1/2 day unless it's an extremely trivial reminder (e.g. log into production system so your account doesn't expire), and after that it's whole days only. It's not that no story will take less than half a day, it's that by the time you consider interruptions and other aspects of reality, that tends to give you the right amount of padding on average.

Collapse
 
aminmansuri profile image
hidden_dude

I agree. I always count it as 0.5 days no matter what minimum they put.

An important principle in software estimation is that it's better to overestimate than underestimate.

Steve McConnell's book on the Art of Software Estimation (amazon.com/Software-Estimation-Dem...) cover's this point well.

All work needs to be estimated, not just coding time. Even talking about it before hand is part of the work. But generally many things get overestimated and many things get underestimated. But if you're in about the middle of that estimation error you can get a pretty good estimate.

Lately we've been able to do work in LESS time than estimated. But it's taken many years of polish to get there.

Thread Thread
 
bloodgain profile image
Cliff • Edited

I'm convinced that the real drivers of success, whether a team is using Scrum or any other system -- or no "system" to speak of -- are trust and communication. If you have those things on a team that is at least competent on the average, which means you can even handle some incompetence if you have some rock stars, you'll find a way to execute that is effective for your team.

I'd bet if we looked at how your team got to the point that they can finish under their estimates, we'd find that not only does your team trust each other and communicate well, your team has gained the trust of the rest of the organization/customer and communicates your progress effectively to your external stakeholders. This results in fewer interruptions from outside.

Thread Thread
 
aminmansuri profile image
hidden_dude

Yeah.. I can agree with that.

Also, I don't really believe in "rock stars".. I believe teams are a mix of talents that complement each other. When a team does this well.. the results are good. Even if some are inevitably faster than others.

Thread Thread
 
aminmansuri profile image
hidden_dude

But I like that FogBugz has the estimation process incorporated. Maybe we should have adopted it instead of JIRA.

Thread Thread
 
bloodgain profile image
Cliff

Yeah, I said "rock stars", but I really just meant programmers who are more than 1x, either because they are just crazy good or more often because they amplify the abilities of the team by being excellent library writers and support for other devs. A close friend of mine is one of the crazy good types, both skillful and fast. I'm a good programmer with a solid theoretical background, but I try to be much more in the second category of providing good support, because I'm just never going to be that fast. Sure, I can find a bug and get a patch out quickly, but I take forever to develop good, strong abstractions and libraries. But when I'm done, they'll be well-made.

Collapse
 
cescquintero profile image
Francisco Quintero 🇨🇴

I have to disagree that I agree(?)

While I was reading I was just crossing my fingers hoping you weren't going to favor estimating by hours.

I'm relieved.

I agree that point estimation might be bad (most probably used very bad as everything in software development life cycle) but hour estimation is worst.

I agree that point estimation and velocity will be misused by PMs, POs, and everyone who is managing the project.

I agree that setting a deadline and scoping work to meet the deadline is ideal. This is the whole stuff Basecamp sells in their books: Getting Real, It Doesn't Have to Be Crazy at Work, Shape Up.

Define work for a given amount of time/weeks/date and set in stone the things the team could be able to work on AND remove stuff that isn't important or that won't be ship.

Chop that scope!

In the other hand, the thing with estimations is that people do not really understand what it is. According to "Software Estimation" book, there's a misconception between "Estimate" and "Commitment".

When PMs ask for an estimation of when something will be done, they do not want to know the effort but when it's going to be delivered.

Estimation: this will take 3 months to be done.
Commitment: but if we only work in these two features(nothing more), we can release it in two months.

In the end, story points are a way to measure effort which will never say when something will be done but how complicated something might be.

Collapse
 
aurelio profile image
Aurelio • Edited

This is funny. I swear i fell asleep yesterday night right after reading this article titled "Stop using story points".

And now i wake up, open dev.to without even thinking right before breakfast and this is the first article in my feed.

Looks like someone out there really thinks i have to question story points (I agree tbh).
By the way, the gist of the article i linked is

We have been counting items done. Each week we just choose the most important items and sign up for them up to the number from last week. It turns out that we get about the same number of them done regardless of estimated effort. We have 1 week iterations so we tend to break things down a bit at the iteration planning meeting.

Perhaps the effect is that we have learned how to break things down to the right size. I don’t know yet, but the point is we get about 8 things done each week, no estimation required.

Collapse
 
codemouse92 profile image
Jason C. McDonald
Collapse
 
fjones profile image
FJones

I have to very strongly disagree on cutting features to maintain the deadline.
We're service providers - to stakeholders first, downstream customers second. Our job is to fulfill the business dream. Discussions about non-mandatory aspects are fine - and I would encourage everyone to question the why and what that's being asked of a development team, but by the time a ticket is considered for estimation, these discussions should already have been had. What ends up anywhere near the sprint is a promise to deliver. Setting a hard deadline is a good means of giving stability to the business expectation, but features should not be cut to meet that deadline - unless that is roadmapped precisely. Delaying the bells and whistles for a later product iteration is good, but estimation is not when this should happen.

Instead - and this is where many teams use pre-refinement and story point caps - tickets need to be refined to a workable state early-on. Call it a 100 if you think this isn't a shippable iteration within a certain timeframe. Tell business to resize the iteration - and help them sort through the bells and whistles, providing product value in a staircase.

Collapse
 
cescquintero profile image
Francisco Quintero 🇨🇴

I see your point. The thing with chopping things off is that software projects tend to always add more work to the already established time.

With this "chop it off" mindset, what we want instead is to stop adding unnecessary BS to the scope and keep removing unnecessary BS that might leaked when estimating.

I agree that work to do have to be well defined but barely happens in many teams.

Collapse
 
peaonunes profile image
Rafael Nunes

Great post! I've actually been thinking a lot about story points, I was hoping to write down my thoughts as well hahaha. I do relate with a lot of points you highlighted.

I do like the idea of throw away everything that is not absolutely necessary, but the team needs to have a strong maturity to be able to follow up on these "nice to have" things that ultimately can make the difference to the end user. Otherwise, that's will always produce "half ready" projects.

One thing I've been thinking a lot is that we do story points to estimate scope and predict velocity, but ultimately the constraints are under time. The product, management, or any client theam is not interested in how many points that feature is. They are interested in when that thing will be ready.

We were thought that estimating using time/hours is bad, the good is to estimate complexity. But my impression is that this fixation about complexity just adds one more layer to the problem. That layer being to translate the "complexity points" to "time estimatives" so you can communicate to other stakeholders the when. And that is yet another place that we increase the error margin.

I've worked with various models and teams, the conclusion for me is the same: Software engineering estimates are ridiculously hard.

Collapse
 
190245 profile image
Dave

Or as I like to repeat to our Project Management Office - estimates are always wrong. When was the last time you got a taxi (not Uber etc), and the estimated fair is what you paid? When was the last time you called a plumber, and the quoted estimate on the phone is what you paid?

They're called estimates, because we're guessing.

To me, a developer measuring complexity is a good thing. Project Management usually aren't technical, so wouldn't have a clue how complex a bug was to fix. Planning poker solves that.

Story Points then allow a Team to build a Velocity (number of points burnt down, across a number of sprints, gives average velocity per sprint).

Knowing the Team velocity, and having a fully estimated backlog, allows a Project Manager to simply drag/drop stories into Sprints 6 months in advance if they like. Of course, the difficulty then is how to manage the fact that estimates can, and should change (as we know more about the problem later down the line).

Our software engineering problem is: How do we define Done? (I'd love it to be, it's in the Production environment, and users can hit it with a hammer... but honestly, we're not there yet for a number of reasons.)

Collapse
 
bloodgain profile image
Cliff • Edited

Your 3 factors are sort of a rephrasing of the old refrain: "Fast, good, and cheap. Pick 2."

The issue, however, is that the "size of the team" factor ignores Brook's law: adding manpower to a late software project makes it later. I think it's even worse than that, that in a real-world scenario, adding people can never pull forward a delivery date but has a high chance of pushing it back, even if the project is on time or ahead of schedule. There are obviously caveats to this, mostly for long-term planning.

I've seen others say recently that Scrum in general is guilty of the same mistake that so many others make in forgetting the lessons laid out in The Mythical Man-Month, and I'm starting to thing they might be right. I have a lot of criticisms of Scrum, but I think this might be the source of many of them.

Collapse
 
togakangaroo profile image
George Mauer • Edited

The thing is that story points are a tool for helping to make exactly the sorts of determinations that you propose. They get abused to high-heaven, but tell me a single project management thing that doesn't.

The main thing about agile is that you're supposed to do the retrospective and in the retrospective you're supposed to evaluate your process and come up with a plan for how to improve it. Which means that we need to understand why we do the things that we do so we can reason on first principles.

So first of, I feel like no one actually covers why the hell story points recommend Fibonacci Sequence to begin with. Understanding this has some implications. The premise is that human beings are pretty bad at estimating fixed points, but pretty good at estimating relations. And - because it is everywhere in nature - there is something about human physiology that makes us particularly good at estimating relations on the golden ratio of 1.618. Which is the ratio that Fibonacci happens to have between its numbers.

Yes, this means that Jira's description of the concept is full of shit - who would have guessed? It also means that starting at 1 is dumb because that is the part of Fibonacci Sequence that doesn't follow that ratio. Have a small task be an 8 or a 13 and you're at least sticking with the intent of the technique and can evaluate its effectiveness on fair grounds.

Also note that looking at first principles, velocity makes no sense if you don't have a good sprint cadence. If that is the case and sprints are all over the place and no one has a good feel for what unit of work a sprint is, then sure, story points are only going to be useful in terms of being a framework by which you can talk about complexity. That in itself can be useful, but isn't always, it's absolutely something to be considered in your retrospectives. Also, for many projects I run more Kanban style - it's not terribly useful there at all.

The bit about comparing teams - it's a fair point, but I don't know how often that realistically happens. Velocity is meant to be used within the project planning process as administered by the team. No one who has oversight over multiple teams needs to see it. I have no clue why they would want to. But even if they did - let's say they wanted to have a dashboard of whether teams are slowing down or not. Understanding the underlying theory lets us propose an alternative. Don't show the higher ups your velocity, show them what they actually want to see - show them the sprint-over-sprint percentage of change. Now they're comparing apples to apples and problem solved.

As for fixing deadline, etc. That's all well and good. It's a fine way to view structure definition of done. But so long as you work in sprints, you still are going to need a way to know what work you should schedule in your sprint planning. This still means making estimates, viewing dependencies, and priorities, and organizing things properly. Story points are a technique for one part of that. Throwing them out and replacing them with nothing might not harm you but certainly doesn't leave you any better off.

Also, don't just start friggin work. People should plan their projects. "Oh we're so bad at estimating and nothing ever gets done on time". Well how much time did you spend out planning your project? Drawing out dependencies? Did you revisit your assumptions often and address issues? Hardly anyone ever does that part.

Collapse
 
michaelphipps profile image
Phippsy

Under-estimating the complexity of a task that seems simple is my constant source of suffering.

Collapse
 
thevikas profile image
thevikas #BanDigitalElections

7 years later,... and I thought...OMG....2 weeks becomes 7 years...

Collapse
 
jonrandy profile image
Jon Randy 🎖️

Estimation is a total waste of time and effort. Avoid

Collapse
 
aminmansuri profile image
hidden_dude

I'm sure we all wish that were possible.. but projects often start like this:

CLIENT: We'd like to build XYZ
DEV: Ok, sure, we can do that
CLIENT: How much will it cost? When will it be ready?

If you don't answer those questions they won't commit.

Collapse
 
jacobherrington profile image
Jacob Herrington (he/him)

I don't like pointing stories because I'm a difficult person.

All the things you said are valid, but for me it's because I don't want to. ;)