A sales consultant, a project manager and a developer walk into a bar.
What sounds like the beginning of a bad joke actually has potential for a lot of drama and mutual misunderstandings galore. Often the drama starts with the topic of this text, estimating the effort or budget of a software project.
I would like to highlight what I consider to be the biggest problems with estimates, present estimation methods and approaches to solving them. In the best case, the text serves for challenging and improving one's own approach to estimation.
What problems do we have with estimates in software development?
Summary:
Estimates based on a thin factual base, or more precisely in the presence of extreme uncertainty, are necessarily inaccurate
Estimates are sometimes communicated based on a problem description; however, reliable estimates can only be based on solution approaches, which in turn requires a proper analysis of the problem
The authority to propose and communicate to the customer is often held by people who do not have sufficient experience in software development
The expectation of the customer is not adjusted to artificially low estimations
Implementation and estimation methods do not match the complexity of the project
Holding on to management and planning methods that do not fit into a complex and ever-changing digital world
A team has to implement features that they have not estimated themselves
Raise your hands: who has ever been asked for a price for a project for which only rudimentary information was available, and a rough estimate was intended to serve "only as an approximate pointer"?
The simple and often quoted example gets to the point: If you ask a car salesman what a car costs, he will not immediately want to give you a fixed price, on which you can even commit him in the worst case. He will first ask you about the brand, basic features and additional configuration features and whether it can also be a pre-owned model. And only once he has all the information he needs, he will give you a price and make you a binding offer. Logical? Of course!
After all: only known and understood(!) requirements and boundary conditions for a project can be estimated.
Problem No. 1 and for me, the most serious, in direct words: Estimates made on a thin factual base, or more precisely in the presence of extreme uncertainty, are necessarily inaccurate.
Unknown information and not articulated requirements cannot be estimated! Do not be tempted to make binding commitments if you have not been provided with sufficient information. The difficulty here is to clearly define "enough information". The amount of information needed depends on a person's experience and skill set directly.
I consider this problem to be always present and extremely critical. We need to keep in mind that we often don't know that we don't know something yet.
-
We know that we don't have information on certain topics\
→ we request this information from stakeholders, we can analyze it, and we can estimate it
-
We don't know that we don't have information on certain topics\
→ we stumble upon new information during requirements analysis, in the worst case during implementation, and plan for new features, for instance, that increase the scope of the project/product, putting schedule and budget in risk
-
We don't know that we don't have information on certain topics and which also influence or challenge our existing knowledge\
→ we stumble upon information that requires re-planning of already specified features, in the worst case making already implemented features obsolete or influencing them and so on, thereby torpedoing the overall project planning
There are surely other manifestations of the problem. However, let's note that with increasing complexity, the one described under number 3 becomes more and more likely. A first hint that we should iterate estimates, whether at the milestone level or by constantly adjusting and refining an existing estimate.
An example from my practical experience. Speaking with the sales department during a conversation about what was predictably a very complex and lengthy project, I requested a workshop with the customer to bring light to the project and learn essential things needed for an estimate. I warned about communicating an approximate budget range before we were provided with key information. After all, we as contractors also wanted budget predictability in the implementation of the project. This request was wiped away with a quick "We can't annoy the customer with questions right from the start." "I see" I thought to myself, "have fun then." And it happened as it had to happen. An estimate based on the available information was "nicely calculated" by the Sales department (= "corrected" downward to be able to stand up to the competition) and manifested in a fixed offer. The order was placed, the sales staff raised their glasses with commission champagne and halfway through the implementation of the project the budget was exhausted. The nerves were on fire -- with our developers, our project manager, our management and, of course, with the customer. Congratulations. If only someone had warned us.
And unfortunately, it's not that unusual for contracts to come about like this. After all, in sales you have to be fast. Otherwise, the competition will run out of the room with the contract.
My example brings two other critical problems to light.
Problem No. 2: The authority to make offers and communicate with the customer often belongs to people who do not have sufficient experience in software development.
A shift in focus from quick promises to secure contract agreements for all parties must be a high priority for the company's own health. Often, a discussion between an experienced IT consultant or software architect and the customer right at the beginning can help. If the persons responsible for contract signing have little or no experience in software development, people who have such experience must be involved in the process for the parties' own safety.
Problem No. 3: The customer's expectation is not adjusted to artificially low estimates.
An estimate of effort always refers to the known pool of information. A change in only one of the two parameters must also result in an adjustment of the other parameter. Development teams are not expected to do the same work in less time. Quality losses and budget overruns are therefore usually unavoidable.
To be clear, if development teams estimate a 15-day effort and a 10-day offer is made to stay competitive, the estimators cannot be expected to complete the implementation in that reduced time. One is thereby shifting all the responsibility regarding the success of the project. An obvious failure and, unfortunately, a not so uncommon phenomenon in agencies.
Problem No. 4: The implementation and estimation methods do not match the complexity of the project.
I have always argued that as the complexity of a software project grows, iterative and agile development must be accompanied by iterative estimation of effort, or at least by refinement of an initial estimate. Whether we divide projects into milestones or whether we actually use an agile framework is secondary. We can only estimate manageable and thus imaginable work packages.\
For small projects, classic approaches like waterfall may make more sense.\
What does not make sense, on the other hand, is a fixed and complete prediction at the beginning of an iteratively developed project. Agile frameworks therefore address these problems with their estimation methods, which should be used.
The fact that customers or management want to make a budget planning that is safe in advance is understandable. However, often this leads to the problems described earlier. And therefore the customer or management is also a problem in this equation.
Problem No. 5: Holding on to management and planning methods that do not fit into an increasingly complex and constantly changing digital world.
It must stop that the conditions of a complex software development adapt to the higher-level management! Instead, management must face reality and adapt, refine, and change its approaches. Only if this happens, problems with estimates of effort can be solved and thereby with the development of complex software as a whole.
A "you must learn to estimate more reliable" often comes easily from those who cause these very problems due to their inflexibility. "Then provide me with further information and adjust your planning" should be the answer here.
One last point I would like to address is the responsibility for estimation.
Problem No. 6: A team has to implement features they themselves have not estimated.
A person who has 10 years of experience in developing complex software will come to a different estimate of the expected effort than a person who has only been "at it" for 2 years. It is the same with theorists and practitioners. If estimation certainty is top priority, then the people who are to implement the project must work out the estimate of the features. The primary focus is on what effort this team is likely to have based on their expertise and specialization. So, it estimates itself -- this should not be done by anyone else. Unfortunately, the situation is often quite different, especially in the classic project approach. Agile frameworks thankfully offer tools to solve this problem.
My conclusion and thesis: People are bad at making reliable estimates of effort. But they are even worse at communicating and processing these estimates. And this lack of ability increases significantly as software solutions become more and more complex.
There are ways and methods to address these problems.
So, how can we minimize problems with estimates in software development?
Summary:
Knowledge is power -- requirements engineering helps to create a foundation for estimation
Project planning must be adapted to the complexity of the solution to be delivered and its environment
Estimates should be regularly reviewed during the entire development of the project and adjusted to the current state of knowledge.
The right estimation and planning method is key to increase reliability and predictability
Estimates from the implementing teams are more valuable than estimates from consultants or planners.
Including estimators in customer communications prevents misunderstandings
Taking responsibility in the context of your roles and knowing your limits are essential and important characteristics for a realistic estimation of a project.
Let's turn away from good and bad practices in project management at this point, let's take a step back and focus on how we can make estimates as realistic as possible.
Knowledge is power -- requirements engineering helps
Foremost, let's keep in mind that an estimate is based on knowledge. In project requests and after the first discussions with the customer, we usually only know "headlines" or requirements that are stated in very general terms and are not specific enough. The more we challenge such "headlines" and go into conversation with the customer or with the stakeholders, the more we learn about the small building blocks, about the "small headlines" if you will.
It is worth mentioning that the idea of "everything being clear" and every required piece of information being available at the start of implementation is pure utopia. It will always be necessary, even during implementation, to analyze new requirements, new information, and so on, and thereby adding them to the knowledge that has been gained.\
Therefore, always plan enough time for requirements management and analysis in your projects. Even if you have ideally planned a requirements analysis phase before implementation in classic implementation projects like waterfall.
And yes, consistent requirements management is also immensely important in agile projects. On the one hand, this is a task of a product owner, on the other hand, the requirements analysis in agile frameworks is also mapped with refinements or similar events. In an iterative approach, this work must be planned and executed as a constant and parallel task to the development.
In the last few years, I have heard the most absurd reasons (most commonly time/money) why people would like to skip a properly performed requirements analysis. And I've seen the projects fail just about every time. Mostly by significantly exceeding the agreed budget and timeframe. And this was almost without exception due to countless reworking cycles because drum roll requirements were not understood or not formulated and specified concretely enough.
Adapt project planning to a complex reality
Split complex and expected lengthy projects into small and estimable packages and plan them as sequential milestones, or plan the project as an iterative implementation from the beginning. The more complex a project, the more agile frameworks help -- even with estimating effort!
Smaller packages are easier to plan, reduce the risk of all parties involved, and simplify budget planning. Estimates are greatly simplified and can consider new conditions that have changed or emerged since the start of the project.
A challenge can be to convince your client of these benefits if they have to calculate with fixed budgets from the beginning. Avoid this inflexibility becoming a disadvantage for you, and take this aspect into account when evaluating your risk as a contractor! Occasionally, it may be better to reject a project.
Check and adjust estimates regularly
If an estimate for a complete project or for individual packages was necessary long beforehand, it is worthwhile to look at it periodically. In these cases, the project partners should agree on a regular review of the estimates and adjust them as necessary to reflect any new conditions and requirements. Both the client and the contractor must be aware that this may result in budget or scope changes. It is therefore important that this fact is communicated clearly and transparently.
Use the right estimation and planning method
Depending on the nature of the project, depending on the requirements of the customer and depending on the implementation methodology, certain estimation methods are more suitable than others. In the following, I try to give an overview of these methods. This overview is definitely not complete. It covers methods I have come into contact with in the last few years. Furthermore, you will find methods that are more for planning and not for the actual estimation.
Top-Down method
Classic planning and estimation method
Appropriate, if a fixed project duration or the date of the project end is given, and the implementation is planned in a classic approach
This estimation method assumes that requirements have been analyzed and understood beforehand
The granularity of the estimate determines the time required for the estimate, which also has to be considered
This classic estimation method is primarily suitable for project scenarios with demanding time or budget constraints. This requirement is the starting point for all further steps towards an estimate. The time span can be divided into phases, i.e., smaller time spans, which give us the possibility of control and readjustment in the planning. Requirements/features can now be planned across these phases. This may even be done thematically, i.e., technically related requirements are planned together in a milestone or phase. To make a realistic estimate possible, the requirements must be still divided into tasks and estimated individually. The result is a "top-down" planning and, in the best case, a detailed "bottom-up" estimation. Often, the creation of tasks is omitted and the requirements themselves are estimated, which saves time but brings some inaccuracy. Moreover, in this method, the estimations are often not done by the developers themselves, which again increases the inaccuracy.
If the estimated project duration is longer than specified, adjustments must be made. This means that requirements with the lowest business value are not implemented in order not to exceed the time limit. The rule is therefore: implement as many features as possible in the specified time.
More information: https://en.wikipedia.org/wiki/Top-down_and_bottom-up_design
Bottom-Up Method
Classic planning and estimation method
Appropriate if requirements are known, the project duration is not specified and the implementation is planned in a classical approach
This estimation method assumes that requirements have been analyzed and understood beforehand
The granularity of the estimate determines the time required for the estimate, which also have to be considered
Also known in the traditional context, this method relies on an analysis of the requirements beforehand. In contrast to top-down estimation, the starting point here is the pool of known requirements. These can be divided into individual tasks and then estimated. However, for time reasons this is often omitted -- even if this results in more inaccurate estimates.\
The estimates are summed up to give an anticipated duration of the project.
This method is often more accurate than the top-down method. However, it tends to take more time. And as mentioned earlier, the accuracy also depends very much on the knowledge of the estimators.
More information: https://en.wikipedia.org/wiki/Top-down_and_bottom-up_design
Three point/time estimation / Program evaluation and review technique (PERT)
Appropriate for estimating entire projects and individual features
Suitable for classic and iterative process approaches
This estimation method assumes that requirements have been analyzed and understood beforehand
The granularity of the estimate determines the time required for the estimate, which also needs to be considered
A technique that aims to take uncertainties into account is three-point estimation, three-time estimation, or simply "PERT". This technique was developed as early as around 1958 and was first applied in a project which had to take extreme uncertainties into consideration and the estimates therefore had to be based on probabilities.
The smaller the packages to be analyzed, the better they can be estimated. Once this has been done, each package is estimated optimistically (ideal case), realistically (possible value from experience) and pessimistically (worst case). Using the formula shown in the illustration, one now calculates an estimated value that "probably" applies to this package. Teams may adjust the formula slightly if the calculated estimated values differ notably, often to the positive or negative side relative to the time actually worked on.
The method is appropriate in both classic and iterative project procedures for estimating the expected effort. It can be applied to large milestones as well as, for example, only for estimates of individual features.
Note: We will discuss this technique in more detail in an upcoming blog article.
More information: https://en.wikipedia.org/wiki/Program_evaluation_and_review_technique
Two point/time estimation
From-to statements are probably the most common form of estimates when it comes to an initial budget size for a feature or project. Contrary to the PERT technique, no realistic value is used here. Only the optimistic and pessimistic estimates are considered. My experience is that in most cases the average of these two values is used for a communication towards the client, although from-to statements would be more honest and would reflect the uncertainty in the estimate. While this technique is suitable for a quick estimate, I believe it is inappropriate for making definite claims or even contractual agreements. I strongly recommend refining such estimates.
Analogue estimate
If you have already implemented similar projects in the past, you can use this information about implementation duration, difficulties, etc. for the estimate. The more accurate this information is, the more accurate the estimate will be.
If you are tasked with implementing software that may use the same components from project to project, this technique is well suited.
Parametric estimation
This technique also uses information from past projects. It attempts to identify differences between the two projects, estimate them, and thereby determine the total effort.
As with analog estimation, this technique is suitable in an environment where the projects are very similar, but there may be differences. This is the case, for example, with customizations of white-label solutions that need to be adapted to different needs.
Story Points, shirt sizes and so on
In my opinion, an estimation statement in hours, days or similar based on Story Points is not applicable. Since Story Points are primarily intended to reflect the expected complexity of a task/requirement, this must first be converted into a time-based statement. Often, Story Points are multiplied by the time spent on a past 1-Story Point requirement to make a statement about time and budget. So if a 1-story point task took 1.5 hours to implement, an 8-story point task will take 12 hours to implement in sum, right? Not necessarily.
What is often ignored is that requirements of comparable complexity do not necessarily take an equally comparable amount of time to implement. Complex tasks, for example, can be implemented more quickly by experienced developers, or they can simply be completed quickly. In contrast, tasks with low complexity may take a long time because they may simply be busywork.
In my eyes, if time estimation is the goal, mapping tasks with shirt sizes or other size specifications makes more sense. In doing so, we can assign a defined number of hours to the sizes. The estimating people assign these sizes to the individual tasks and thus gradually determine the expected effort for a sprint or project.
The use of shirt sizes has the advantage that, for example, "L" can also be used to indicate From-To durations. Of course, these must then be considered in the overall estimate. For the estimator, however, the estimation process is made much easier.
#NoEstimates
A more radical and fundamentally different approach is taken in the idea that has been discussed under the hashtag #NoEstimates and has created quite some buzz. Being radical means nothing more than finding and using an approach that avoids all the mentioned problems from the very beginning.
The idea does not mean that no forecasting can or should be done at all. Rather, this approach does an end to classic techniques in an increasingly complex world of software development. Instead, it relies on historical information and on increasing security over the lifetime of the project.
Under the hashtag mentioned, you can find many articles on this topic, and it is worth taking a look. To summarize, the idea aims at
avoiding efforts for inaccurate estimations in a constantly changing environment
and to use past iterations/cycles and the experience from them (in an agile approach to implementation) for predictions.
Furthermore, various ideas are also discussed that have risk mitigation as a goal. This can be achieved, for example, through chunked funding of milestones or features and also by the resolution to be ready to deliver or finish the project at any time.
Using the knowledge of previous sprints or milestones, predictions become more accurate during implementation. This idea contrasts with classic planning methods that have fixed budgeting over a long period as a goal. To solve this problem, management must be willing to question their approach and take steps towards the operationally involved people.
As with any method, there are proponents and opponents of this approach.
More information:
Estimates from first hand
Following the DevOps approach "You build it, you run it.", I would like to phrase a "You estimate it, you build it.".
Software developers design and implement solutions. They develop the most suitable solution approach from their perspective within the given solution space, which is limited by the given conditions. This solution approach must be estimated!
It does not help if developers are "dictated" a given time frame for the development of a solution. Even worse, if this framework is given by people who do not know the solution space or simply have no experience in development. Such an approach generates many things -- but no motivation among project participants.
Therefore, the following applies: An estimate cannot be made more precisely than by the people implementing the software. Because they can also estimate themselves best and price in the know-how of the team. This is the reason, among other things, estimates by the team are always assumed in the Scrum Framework and are executed regularly.
Inclusion of the estimator in customer communication
Every estimate is followed by questions. Both in internal communication and in external customer communication. Typically, these are queries that want to shed light on the context of the estimate. Understandably, people usually want to know the intended solution approach on which the estimates are based. Even though this may have already been outlined in written or verbal form, detailed questions will always arise. Care should be taken to ensure that these questions are answered primarily by the estimators to avoid misunderstandings and complicated communication paths.
Taking responsibility, know own limits
Different roles mean different responsibilities. And as always, if one is unable to perform a task in one's own area of responsibility, for whatever reason, help must be found. This requires knowing one's own weaknesses and limitations. And yes, it also requires that you sometimes step outside your comfort zone. False pride is of no use to anyone. Last of all, oneself.
The customer, the party requesting estimates for a software solution, must provide information necessary for a realistic estimate upon request. In the best case, missing information is discussed by the client and the contractor on an equal level, i.e., people who are technically or thematically able to discuss the requirements. We have long outgrown the age of "silent post", meaning that communication may only take place through one point, and so has software development.
The contractor must ensure that he uses an estimation method that fits the agreed project approach and that he has the necessary information available in the required level of detail. If knowledge gaps regarding the requirements cannot be addressed internally, information must be gathered with the customer. And this must be done in direct communication, without any detours.
Estimators and developers should at all times request the things they need to realize a task. In the context of estimation, this is information needed to consider requirements. Directly addressed to these individuals: Request information needed to fill knowledge gaps. Your name is associated with your estimates. Remind others of their roles and responsibilities if necessary.
Let's learn -- let's adapt to these complex circumstances
As we have seen, estimates are often complex and time-consuming. However, through estimation cycles, we also contribute our part to risk mitigation. We learn details of the software to be implemented and its environment, and constantly expand our horizons through consistent requirements engineering. If you consider one or the other piece of advice, I would be happy about it. You will notice that you can make estimating and also dealing with it much easier.
Each team has to learn for itself which technique it knows best and when to use it. Experiment, do things differently, challenge yourself, learn what works best for you and your environment.\
The management, which is truly interested in low-risk projects, should give you space, involve you in the communication towards the client and, if necessary, verify and adjust their procedures. Only together, complex software projects succeed.
Be transparent with the customer at all times and challenge him. The customer is the origin of requirements and will help you to eliminate gaps in your knowledge. Never compromise this knowledge for the wrong reasons -- even if this means longer estimation sessions or intensive requirements management.
Estimation will certainly not become easier in the future. As you can see, estimation projects are strongly influenced by external influences. However, it does not help to go into a defensive mode here. We have to keep up with the increasingly complex world and find new ways of dealing with it. This applies equally to top management, project management, consulting, development and operations. Get on board -- there is no other way.
If the problems described above sound familiar to you, and you need support or consulting, please do not hesitate to contact us at https://www.devlix.de/kontakt/.
Frank is writing for the devlix Blog at https://www.devlix.de/blog
This article was published first here (german): https://www.devlix.de/schaetzungen-im-projektalltag/
Top comments (0)