Since we've taken on 8 interns in Aista, and I am not sure about their experience level, and I need to be creative in regards to the process of following up on our interns - I wanted to write something publicly about waterfall software development. Why? Because the task they will be doing at Aista is to implement a working piece of software in 6 to 8 weeks, using the waterfall software development process.
"Waterfall" has a bad reputation. For software developers and architects it's almost a curse word. This reputation is really not justified, at least not for most projects. To understand why, let's first ask ourselves what waterfall is, for then to ask ourselves what the primary argument against waterfall is.
What is waterfall and BDUF
BDUF implies "Big Design Up Front", and is closely tied to the waterfall mindset, implying you start out with a big design describing what you want to build, down to the smallest detail. When you have created your design using for instance Figma or something similar, you start modelling your database. When you're done modelling your database you start implementing code wrapping your database. This is why it's called "waterfall" because the process moves in one direction, the same way a waterfall does, and the assumption is that you can't go back and change your design after the process has started. The argument against waterfall is as follows.
If you don't get the design correctly applied initially, the project is bound to fail, since it doesn't accommodate for change in the requirements after the project has been initiated.
However, the assumption that waterfall is a "one directional process" is simply not true. Just because you're using waterfall as the primary process for your project's delivery, doesn't imply you cannot change the rule parts of the time. Hence, claiming that waterfall doesn't allow for change is simply not true.
The second point is that if you already know more or less exactly what you want to create, having a BDUF will lubricate your software development experience, resulting in that all your team members can work more autonomously, with fewer meetings to collaborate with each other, since everybody knows what they're supposed to create - At least more or less. With a pure Agile software process the only thing you'll achieve is to spend all your time in meetings quarrelling over use cases trying to integrate other developers changes into your own parts.
The third point which is somewhat unique to Magic, is that if you've got a database design that's accurately applied to model your project's requirements, Magic literally creates your code automatically using the database schema as a "formal specification", wrapping a CRUD API around your database tables, for then to optionally create an Angular project for you.
Everything that can be formally described can be automated
To illustrate the problem with Agile here, imagine telling Toyota's car robots the following; "Yeah, we're about to create some cars, we don't know how many wheels they'll need, and we don't know what engines to use - But feel free to 'change this' as the owner's requirements 'changes over time'". This is the Agile software development process applied to car manufacturing.
Needless to say, but the above of course is madness. Still we tend to take it for granted as the only sane software development process, while it's really quite insane ...
If you're telling me Agile is so fantastic, go build a house using the "Agile software process"
If you have no idea what to create, my advice is to ask the customer questions, and spend more time in the project investigation phase instead of starting out by implementing use cases.
PLAN what you want to create! Plan it WELL!
Modularity
The idea with Agile software development, is that instead of using BDUF, you have the customer feed you with use cases. The problem with this approach of course, is that you'll need to implement everything in a modular approach, where everything might potentially change, and unless everything accommodates for this future change, you'll have to re-implement the same thing dozens of times to accommodate for this change. Unless you want to re-implement everything multiple times, you're forced to creating a design that is so modular in its approach, that you end up adding 10 additional layers of abstractions on top of your thing, where most of your implementations needs to be re-written further down the road, creating breaking changes to the app as a whole, all the time. Basically, your process becomes a never ending cycle of having to apply breaking changes to your project.
A "pure" Agile software development process inevitably leads to the Big Ball of Mud
Yet again, if you don't believe me, try to create a house using "use cases" and "Agile processes". How many doors do you think you'll have to discard if you do before the customer is happy you think? And sure, you might have a door and an entrance, but there's no guarantee of that the door is correctly applied. Below is a house that might have been implemented using an "Agile house building process". I'll leave it up to to my readers to see the problem here ...
Similar results will be experienced if you implement software exclusively using the same process. The funny thing is that if you break the above house down into its use cases, it's probably a perfect match from the software developer's point of view.
- It's got a door - Check
- It's got a roof - Check
- It's got windows - Check
- Etc, etc, etc ...
The fact that the house was delivered upside down is from the software developer's perspective irrelevant, since he or she perfectly implemented every single use case the customer had. Ironically, the customer can't even complain about the delivery, since the software developer can prove he delivered exactly what the customer told him to implement ...
Waterfall vs. Agile, the showdown at dawn
Facts are, neither of these processes are correct. The truth is as always somewhere in between. Do as much as possible using waterfall, but as you do make sure you accommodate for change as the customer gives you feedback. If you can do everything using waterfall than prefer it, because it's simply superior in every single regards. Below is a building that was created using the waterfall process to illustrate its beauty.
Conclusion to my student interns
Spend more time up front designing and modelling what you want to build, because the more time you spend up front, the more likely it is that you'll deliver something useful. Then only when absolutely necessary resort to Agile and change. Don't fall for the "Agile Kool Aid". The world has enough garbage software projects as is ...
When you know exactly what you want to create, most of the job is already done!
Top comments (19)
Sorry, Thomas, but "the assumption that waterfall is a 'one directional process'" is not an assumption at all. It is the DEFINITION of the waterfall process. Hence if you iterate, then you are not strictly doing waterfall. You're doing some kind of hybrid.
And that, frankly, is what most enterprises are doing no matter how much they claim they're doing agile. Bullshit. What they're doing is a mix of agile and waterfall with, typically, a lot of useless crap thrown it to make it look more impressive.
Waterfall writes a spec, implements it, tests it, delivers it. Period. If you go back and change the spec, then you're not really doing waterfall.
So essentially you've found a hybrid that works for you (or so you believe) and it perhaps leans more toward waterfall. Nice. But it's simply not waterfall.
It's an exaggeration obviously to shake up things, also written because I'm tutoring a bunch of students whom I don't want to fall for the "let's just start coding Kool Aid" which is tempting because of the hype surrounding Agile ...
Oh, yeah. I know exactly what you mean. The temptation to jump straight to coding is almost irresistible to most devs, just as most students in school never wrote an outline, they just jumped into the essay... with predictable results.
I loved the comment on your other post from the guy (there were probably several) who complained about your over the top language. I guess they've never heard of hyperbole.
Hahaha :D
Sometimes you have to exaggerate to get a point through, I'm fairly good at it unfortunately, giving predictable results (angry commenters that is ... ;)
But I don't complain, all eyeballs are good eyeballs ... ^_^
Lot of companies sell Scrum and agile methods as the best thing that ever made... The outcomes?
Another guy here said Agile was an excuse to be lazy (kind of). Your comment definitely fits down these lines of thought ...
Hello Thomas,
First of all, this is not my first language, so forgive me if I write some "no-sense" english.
BDUF is based on a premise that says "late changes will cost more". It's classical and you can find it in many books prior 2k like Pressman and Sommerville. And I do believe we can say it is not false cause we build software like a stack and structs you build later often are based on those we created earlier. But I think you are missing one point: an app is not like a house. A plenty of books claims that one of the biggest mistake was trying to apply classical engineering approach, such as the civil one, to software development. If your family grow you can build an extra bedroom but you're more likely move to another house.
Things doesn't work so well in software development. Your business will change cause, in fact, we barely know our business until we got to production. You can do thousands of interviews but there is huge difference between what your customers ask and what they need. Your business will change cause, unlike a house, there are competition. You do not need to change your house when your neighbor build a better one. Your business will change case because society changes, government changes, tax changes. Your business will change because you want to go to new countries, new markets. You can't just throw away your entire base code and build a new one just every time something like this happens.
So we have 2 apparently contradictory statements here:
But we don't need to try too hard to realize that they are not contradictory: that's just reality. That's why software development is hard and expansive. And that's why waterfall or scrum are, both, doomed to fail. In fact, partially failing.
I do agree with you about all the Scrum bullshit. I think Scrum is one of the dumbest thing we did in last 30 years. Show me a Scrum team and I show you unhappy people: managers, programmers, customers... And yet you will always find avid supporters of it.
But Agile is not Scrum. The Agile Manifesto is simple, plain and concise. It just argues "while there is value in the items on the right, we value the items on the left more". It doesn't say there is no value in processes, tools, documentation and following a plan. But when there is a conflict between "respond to change" and "follow a plan", let's respond to change. Said that, I wouldn't even say that Scrum, as it is today, is Agile. See how anything you change in Scrum, people say it's failing because of that.
This shouldn't be as hard as Scrum makes it out to be.
The article is an "extreme counter argument" not created to show the absolute truth, but rather to be thought provoking, hopefully resulting in "killing holy cows", allowing the reader to become inspired and think for himself.
However, you've got a lot of really great points, and I am thankful for your enlightening comment :)
Ah yes delivery! You see...
“the whole point of the wish business was to see to it that what the client got was exactly what he asked for and exactly what he didn't really want.”
( Terry Pratchett talking about how demons service clients in hell in the book "Eric" )
Good luck working on a real world software project where the requirements don't change.
Maybe originally, but spare a thought for the poor folks that have to keep it running several years down the line, when business requirements have changed.
If your project is sufficiently large enough, the requirements are likely to change within the implementation timeframe, which is where some form of agile methodology really pays dividends over BDUF.
Agile also doesn't mean that you skip design up front, it just means that you don't try to design your entire system up front.
Some other guy gave a similar comment here, where he said that what I proposed wasn't really waterfall, but a combination. I can agree with that. I still think the article is highly valuable, since I've seen too many attempts at "going Agile" that simply failed because everything ended up a cabbage, and we spent half our days in meetings.
For the record, I think you should try to design your entire system up front. I do believe you should be aware of that you'll probably fail, and be ready to apply (some) changes as you start implementing ... ;)
Nice post!
Have this experience that since most companies has some timeconsuming sort of Agile way of working with sprints, sprint planning, backlog grooming, Scrum poker, Scrum masters etc. etc. this also means they do not have the time or manpower to apply other methodologies or routines relevant for the company itself or industryrelated ones.
Such as documenting requirements ..
The story is always the same "..just pick a User story or task from the todo column in the board.."
Well, problem is that the task at hand is described with a few words in the header of the card ;)
I was an architect once having responsibilities for 8 developers, I spent half my day in meetings; Grooming sessions, velocity evaluation sessions, 2 standups each day, each lasting for 45 minutes, etc, etc, etc ...
According to neutral research in the subject the average software developer is supposed to be able to create between 325 and 750 lines of code per month. Ignoring whether or not this is a good metric for productivity, my devs (8 devs) delivered 23 lines of code in two months.
When I asked for help from my manager, his reply was; "Stop being so darn productive" - I delivered my resignation a week later ... :/
That's agile used for managers control, which Is, I believe how most companies apply it, and Is pretty much against agile principles.
I dont think ther Is anything inherently bad about those methodologies. You should use whatever suits the culture of the company, the nature of your product, buisness and the need of your users or clients
I think the fact is that "true" agile is not allowing anything of resemblance of waterfall comes near to it, and that's a problem, because there's a pattern worldwide that we're using both with different flavours. But for me all boils down to the type of client that you're working with, some of them know more of less what they want and you can "squeeze" more information to get a good plan and effectively execute it. But that's not always your client, some clients really change their mind everytime and on top of that don’t have the availability that you'll require to plan as waterfall suggest.
So yeah, I'd say, mix things when ever necessary to accommodate to your context and pick the best of both worlds.
Yup, but if I had used your opinion as my header, nobody would have read the article, and I wouldn’t be able to lead them to the truth. Which ofc is the middle way … 😊
Hahaha, I wanted to use a header down the line of your comment, but I figured waterfall is more eye catching :)
Software Engineering (1968, p.21):
Kinslow: The design process is an iterative one…
Ross: The most deadly thing in software is the concept, which almost universally seems to be followed, that you are going to specify what you are going to do, and then do it. And that is where most of our troubles come from. The projects that are called successful, have met their specifications. But those specifications were based upon the designers’ ignorance before they started the job.
If you read the article once more I’m not advocating 100% waterfall, without the ability to apply change, I’m just exposing 100% Agile for what it is, which is pure madness … 😕