As developers, sooner or later we will end up in a situation where we have to work with a legacy codebase. That can be the whole application or just part of the application written at the very start of the project. It's hard to fight legacy as we don't know how each part of the application will grow and at which scale. Making everything easy to scale is not very efficient so you need to guess which parts of the application are most likely to be improved in the future.
But on one of our projects, we faced a unique challenge where we created a legacy even before the initial release. For quite some time we tried to fix it or improve it, but in the end, we decided that we should rewrite the whole project from scratch.
Here are the top 5 reasons why we ended up doing it and changes we made in a rewrite.
1. Unknown territory
Before this one, we successfully developed more than 100 projects and we thought this should be just one more of them. The initial estimate was much bigger compared to anything else we were working on before. Just a few months into the project there were already red flags. The team was not comfortable with the chosen technologies as they had little experience with them. From the project very beginning, we used agile methodology in contrast to a waterfall on our previous projects. We started with very little documentation (as agile suggests) and worked from sprint to sprint. When the project grew in size, we faced constant refactors of our previous work to introduce new features. The combination of these few problems drastically increased feature estimates.
We also didn't have a lot of experience with the business domain, so we needed client input to discover even the basic information about their day-to-day work.
To fight these challenges, a few new roles were introduced: product manager, business analyst, and tech lead. Some were given to the experienced people already on the project, but some required completely new colleagues. Also, we decided to create a project specification with all the features, and a long-term development plan. This plan enabled us to develop features with upcoming features in mind. Together with all the organizational changes, we introduced new processes and improved old ones. This was supported by new tooling (Jira, Confluence) and integrations with tools already in our stack (GitHub, CI:CD).
2. Technology choice
One of the main reasons why we faced some walls during development is technology choice for a project of such size. Our core technologies were not a problem, but JSON:API as API standard gave us plenty of issues. We had a lot of actions on top of domain models and this didn't work very well with the resource-based approach. In addition to this, maintaining API documentation took us too much time. Generators helped with new resources, but not so much with modifying old ones.
In a rewrite, we swapped JSON:API with GraphQL which gave us much-needed flexibility in communication between API and front-facing app. Accessing the data remained very similar to what we used to in JSON:API, but mutations allowed us to modify data in a more action-focused way.
3. Architecture
We were very comfortable with MVC, so we decided to keep it in this project as well. The only difference is that we switched to a headless architecture where a back-end team develops API and a front-end team develops a front-facing app. When the project scaled, folder structure became very messy and there were no restriction to use a logic of multiple domain contexts in the same place. With everything mentioned, adding new features to such codebase became very hard even with a good code coverage in tests.
This was one of the hardest challenges to fix as we had to change it from the ground up. After some research and a few prototypes, we decided to use DDD and Layered architecture principles in our codebase. It gave us a very descriptive folder structure in every bounded context and boundaries for cross-context communication.
4. Onboarding difficulty
In a project with a complex domain and messy codebase with no clear structure, onboarding of new colleagues was very difficult. It took us around 3 months before a new person on the project became productive. As this was a long-term project, it's expected that from time to time we will need to onboard a new person, and losing too much time on it is something we wanted to avoid.
Just by implementing a new architecture, everything mentioned here was drastically improved. We had an option to onboard people only in one specific domain context and they could finish their tasks without worrying about breaking other parts of the application.
5. Long-term motivation
Every developer wants to work on an interesting, challenging, well-structured project with little to no stress. With everything we were facing, it was very hard to keep a good team spirit. Instead of feeling proud on finding the solution for a complex domain problem, we faced technical challenges where domain logic was very simple. This made us feel like we are not progressing as we should.
Clear hierarchy, roles, and responsibilities in combination with modern technologies drastically improved team morale. To ensure it stays this way, after every sprint, we grade our happiness and try to figure out what could be improved.
Conclusion
It's never an easy decision to abandon everything you were building for years, but sometimes this is the best possible decision. It took us 10 months to rewrite everything and during this process, we didn't face any major blockers or technical limitations. The team became more stable and fueled by the same goal. By looking at the project's future, the rewrite was the best decision we could make. The mistake was that we didn't do it earlier.
Top comments (7)
Great article!
At my work (working as a frontend developer) I am currently working in a team rewriting a large e-commerce app. It needs to be refactored from the ground up because of performance problems and it just generally is difficult to refactor. It is actually really an interesting process and rewarding to be a part of this proccess. Can't wait to see the finished app.
Glad you like it.
Rewrite is usually the last resort, and it should be. When you start spending more time on refactors than on developing new features, this is a good sign to start thinking about rewriting. In a microservice architecture, it's much easier as you can rewrite service-per-service. In a monolith, most frequently it's all or nothing and time-consuming.
Yeah, totally agree! I do not think anyone does a full rewrite for the fun of it 😂 But sometimes it is the only real way forward.
Kudos on being so open, honest, and articulate about this project and what you’ve learned!
“ … we successfully developed more than 100 projects and thought this should be just one more ...”
“ … initial estimate was much bigger compared to anything else we were working on before.”
“The team was not comfortable with the chosen technologies … “
“ … didn't have a lot of experience with the business domain … “
“ … new roles were introduced: product manager, business analyst, and tech lead.
“ … decided to create a project specification with all the features, and a long-term development plan.”
In fifty-plus years in software engineering, these issues have come up in project after project … so often I added the quotes: these ‘problems’ seem to be the ‘norm.’
Any shop that wants/expects to grow and flourish would do well to read and reread your post … and give a lot of thought and discussion to these issues. (IMHO) This kind of work makes software engineers out of coders/programmers.
This is a brave story and as always, the challenge is to be able to see the bigger picture and the challenges that come with that, before starting to program.
if you never have build a house, you would have to break it down more than once, because you did know the base should be able to carry that much.
Agile is about "lean and mean" but still done with common sense.
I once had to make the decision to throw away a value of over 2 million just because of about the same reason. And afterwards, that was cheap, because we would have made a sinkhole with much more spend.
So did you add absolutely zero features with N people in 10 months? That sounds really expensive. How did you manage to justify that massive cost overrun? Who paid for it?
It took 10 months to add all the features we had before, but we also added some new features along the way.
And yes, it was expensive, but we have long-term plans so a good foundation is a must.
As we are will be using this project for our future product, we are looking at this as an investment.