Microservces Anti-Patterns and Ten Tips To Fail Badly

In this post, I will give a run down of a youtube video by David Schmitz of anti-patterns and tips to defeat microservices.

10) Go Full-Scale Polyglot

One of the benefits of using microservices is that you can use different programming languages in different services. This freedom of choice can be abused with inappropriate use of multiple technologies. There is a phrase "CV driven engineering" which is when devs use frameworks and tools because it will look good on their CV. No-one will be able to support all the code and customers will be the worse for it.

9) The Data Monolith

The next anti-pattern is a shared database. You can share a database cluster with total isolation of tables. What is being called out here is sharing data via the database. If you go to another microservices tables, and they change their tables, things explode. Some decades ago on the first platform where I was a tech lead the strategy was to have a single shared database with many teams writing micro-frontends. A single sign-on service meant that we could link between applications and the user had the illusion of a single big service but each area of the system was built by different teams. Where the technical strategy was profoundly flawed was that teams were to exchange data via Stored procs. Databases aren't designed to act as APIs to encapsulated business logic. In effect teams just gave other teams access to the procs that their own app used. This is just like querying each other's tables. 💩 💥 💀

8) The Event Monolith

There is a popular pattern of event sourcing and taken to its logical extreme it is known as the "inside-out database" pattern that is gaining popularity in the Kafka world. What David's video explains is that unless you take a lot of care exposing events between teams gives you high coupling and can lead to copy-and-paste fixes and enhancements being added to every service. At a very high level, you can get the same problems as sharing database tables. Appending entries to a shared event log is just like appending rows to a shared database table. No team should expose an event stream to anther team without it being a contract that they will maintain strict rules of backwards compatibility just as they must with any microservices API. Avro can be used to add attributes in a way that doesn't break old processes reading the new events. Upgrading data or fix data related bugs needs careful planning and practice or you will have monolithic headache.

7) The Homegrown Monolith

Obviously, something new as cool as microservices requires that you go and write your own framework. After all, you are one of the smartest folks around and if google can do it then so can you. Yet it is very hard to avoid a change-the-world event when you are writing your own framework while you write your application. A homegrown framework is a monolithic force as well as being immature and a distraction from adding business features.

6) Use The Meat Cloud

David specifically picks out large financial services organisations here. The "meat cloud" is using lots of people in an ops team rather than self-service automation. Typically teams doing microservices have a continuous integration and continuous deployment pipeline for their application code. Yet big firms believe that it is optimal to have separate teams that manage all the infrastructure needing a support ticket to make any changes. Since they have people sitting around waiting to do work they manually click about to set things up rather than automate anything. To prevent folks from breaking expensive shared tools they lock them down to only do what they anticipated you need to do. The experienced person who set it up isn't around to actually run the shop. You have someone without the big picture clicking the mouse day-to-day. This separates who wants something to be done, from the people who know how it can be done, from the people who have the privileges to get it done, from the people who have the time to get it done. So nothing gets done.

5) The Distributed Monolith

The anti-pattern here is to build an app as though it is a monolith yet deploy it as many microservices. This combines to give you the worst of both worlds. In a microservices environment, failures are more common than in a monolithic environment for the same level of application code quality. This is due to the fact that networks are not perfect and due to the mathematics of cumulative probabilities. If you have a problem where only 1 in 365 requests are failing in a monolith that is a bad enough problem. If you are using microservices where 23 inter-service calls are required to build a page then 1 in 365 requests failing means that 50% of customer pages will fail. The network is not reliable. Here is a post about real-world outages showing that network reliability is a fallacy. A service mesh may help but is that itself a complex distributed system that will take effort to set up and run.

4) The Single Page Application (SPA) Monolith

This is another hidden monolith anti-pattern. The problem is that your single page application can become the monolith. That has all the problems of a traditional monolith with the added twist that is running in many different browsers. David points out that it feels easy to just put new logic into the front-end when your requirements suddenly change but that convenience is the road to having a big ball of mud application. Approaches such as React.lazy applied to React Routes with code splitting may allow you to create a modular architecture for your frontend to avoid this trap. Such advanced techniques go well beyond the basics and will require effort to set up and will be a challenge to retrofit to a fast growing code base.

3) The Decision Monolith

We can all take a dig at architects who restrict the use of new technologies and who proscribed J2EE as the solution to all things. Yet it is hard to not say no to new ideas in the interests of reducing risks. We should ensure that we don't have a dogmatic and inflexible approach and that we create ways to experiment and to continuously introduce new innovations. 

2) The Monolithic Business

Microservices cannot be a technology only matter. If we have monolithic processes then we are really doing waterfall in disguise. This is also known as "wagile", "water-scrum-fall", "dark agile", "faux agile" and a host of other names. As well as how the process works we should also call out monolithic attitudes and approaches outside of technology in terms of business engagement. The essence of agile and microservices is that we release little and often to get valuable feedback. If there is no real feedback loop between the business user and what we are building then we are not optimizing business value.

1) Use A HR Driven Microservices Architecture

The notable quote here is: 

"It is easier to search for React developers than for general craftswomen."

This is basically saying that if your hiring practices focus on specializations then you are de-optimising your ability to build, deploy and support microservices. It is also a known bad practice to organise teams into specialisms. You need to organise around small mixed disciplined teams that are empowered to design, build, deploy and support what they create in as self-service model as possible. Yet when I talk to developers I am often surprised with how they self identify as in a given specialism such as "I am a Java developer" or "I am a React developer". That just perpetuates silos and allows organisations to box you into a silo and disempower you while claiming that is what makes you happy. Call yourself a software engineer and be intellectually curious as to how everything works. Infrastructure and networks in the cloud are software defined and has an API you can make use. Aim to be a maker of things not the ​holder of particular skils.

That's all folks. It would be great to hear about your experiences of what works and what doesn't.