How to improve your problem-solving (and analysis) skills

Improving your problem analysis skills

Before getting into problem solving, I think that there is a key area that many people overlook. That is problem analysis. Problem-solving and problem analysis skills are probably some of the most essential skills needed in today's jobs. There may be multiple ways to solve a given problem, but if we don't tackle the right one, we will end up having to solve a different version of it later.

We're going to take a look at some common tools that can help us, individually and as a team, to get better at problem analysis and when we can use each of these tools.

5 Whys

The 5 Whys framework has been covered in many blog posts so I'll summarize it here with some links for further reading of some of the better posts. I first learned about it at a startup I was at, especially in terms of finding the root cause of an issue like "Why did our site just go down?"

5 Whys basically says, ask Why did [something happen]? 5 times (or really as many times as it takes until you have reached the end of what you, or your team, can take action on. Essentially finding the root of the problem.

We'll look at that question and use 5 Whys to analyze it.

Q. Why was our site having issues?

A. The users were unable to authenticate

Q. Why were the users unable to authenticate?

A. The authentication microservice was returning 403s incorrectly

Q. Why was the authentication microservice returning 403s?

A. It was unable to make a connection to the database

Q. Why was it unable to maintain a connection with the database?

A. The database connection ran out of threads to run.

Q. Why did the database run out of threads?

A. The microservice was sending a request every few microseconds

Q. Why was the microservice sending so many requests?

A. The connected devices (several thousand IoT devices) were trying to authenticate every 5 seconds and unable to retrieve valid tokens

Q. Why were they trying to authenticate every 5 seconds? (Alternatively we could also have asked "Why were they unable to retrieve valid tokens?")

A. Their firmware was coded to try and authenticate every 5 seconds if they had not gotten a valid token.

A. (Alternative answer) The token was marked as invalid because the timestamp was off

Here we have 7 Whys with two root cause problems that can be addressed by the team. One issue was related to a daylight savings time issue so the token was seen as invalid by the IoT device. The second issue was the fact that it was making a request every 5 seconds to authenticate when in reality the authentication was only needed on a specific request.

Short-term solution was to kill all traffic going to that microservice for a specific range of IP addresses until we could address the bigger problems that were causing outages for many more customers who needed to authenticate.

If we hadn't dug in and tried to answer these 5 Whys we might have only stopped at one level and just shut down the affected devices. We would not have learned that it was actually a smaller subset of those devices that needed to be fixed and identified a bad architecture decision made earlier.

In short, this tool helps identify where things are broken and how they got broke. It's great for identifying issues within a system before they become big enough to break something else.

I've found that most teams struggle with this kind of thing. They tend to focus on fixing the immediate symptoms instead of digging deeper to find the real underlying causes. This leads them to make decisions based on their current understanding which often results in new bugs being introduced.

When to use 5 Whys?

You can use 5 Whys in more than just analyzing something that has just broken. You can also use it for:

• Processes that aren't working as well as you would like
• Analyzing customer behavior (doing X instead of Y)
• Part of a post-mortem
• Decision-making errors

Think like an entrepreneur

What do I mean by "Think like an entrepreneur?" I mean, in short, try and find the root problem that the customer is trying to solve. Again it means digging below the surface-level and asking deeper questions.

Many entrepreneurs fall into the trap of "oh, I have several customers that want me to solve problem X for them so I'll build that." They build it, and...crickets. No one ponies up the money. Unfortunately for these entrepreneurs they fell into the trap of listening to their customers. But wait you say, you're supposed to listen to your customers, right? Yes...and no. It's the questions that you ask them, that's what determines how they will respond.

When you're an entrepreneur you can't waste time building something for the wrong problem, if you do that your company could be dead very soon. How do you find the right problem to solve for users? There are several ways to do this:

• Ask lots of open questions about their process
  • What do they do each day?
  • What takes up the most time?
  • Is there a problem they've tried solving with other tools? What worked what didn't?
  • If they haven't actively tried to spend money solving said problem you have to wonder how much of a problem it really is
• Build small things that solve a part of the problem and test it out.
  • If you can solve for part of the problem then try that first, see who bites.
  • Only build as much as you need to.
• The user thinks that the problem is X, but through research and talking with customers it seems like the real problem is Y.
  • Here's an example - User thinks that what they want is a better email tool to manage all of their conversations. Slack looked at the communication issues and built a different tool to manage team communications. (I'm not going to get into an argument as to whether or not one is better than the other, this is just what I have seen in the industry.)
• Maybe the problem is more with a process rather than a specific aspect of it.
• Or conversely maybe the process is fine, but there is one aspect of it that could be streamlined and improved.
• Research problems that people are having on forums like Quora, StackOverflow/Exchange, Reddit, or specific industry message boards.
  • Look for patterns
  • If multiple people are having the same problem you might be able to dig deeper and see what the ultimate end goal is that they're trying to achieve. There may be another way to achieve the goal that goes around the problem.
  • Have others come up with solutions to the problem? Are they good? Time consuming?
• Kill your darlings - If you think a particular feature of your application isn't being used, try killing it, see if you get some angry users or if it is just a few.

All of these tools can be used to build and solve internal as well as external problems for a company.

Think like a customer service rep

Another tool I like to use is to listen to actual customers. If your company is large enough to have a customer service team, or a customer success team they are a great resource to turn to to see what issues customers are running into.

What are some things you can do here?

• Look at a batch of recent tickets, see if there's any patterns that are emerging or anything that sounds interesting
• Actually hop on some email responses or listen in to some calls. Are customers wowed by your application? Or do they always seem to have some kind of complaint?
• Look at what parts of your application customers use the most - there are many tools you can use like hotjar, google analytics, etc. that will help you understand where users are going in your app and where they drop off.
• What is the primary function that customers use your app for? Only by listening to customers can you get a true feel for how your app fits into their daily workflow. Maybe there's one thing that people do a lot of that you had no idea about or thought was an ancillary function of your app.

Bottom line

The bottom line with all of these is asking questions. Not only that it's asking specific types of questions, especially depending on the answers you get back.

• For internal problems ask the 5 (or more) Whys to your team (and others inside the company) to get at the true heart of the problem, not just a surface level.
• For external problems talk to your customers either through open-ended questions or specific flow questions for how they use your application
• Examine customer behavior inside your application to gain insight into how they use it
• Use tools like analytics to identify what parts of the application are being used the most, try and drill down to find out why by talking to customers or examining customer calls/emails for additional insight
• Talk to customer support/success to see what tickets come up the most or to see how customers commonly use your application

As you practice each of these skills you will start to flex your problem analysis skills further and just make them stronger. As you improve these you can also improve your problem solving skills which we'll go over next. Thankfully these aren't an either/or proposition you can improve both of these together or separately, both are extremely important when it comes to being a well-sought-after software developer.

What is problem solving?

So, problem solving in short is looking at a specific problem and then brainstorming a number of ways that you could possibly solve that problem. Some solutions may be easy, some solutions may be very hard, uneconomical or just plain silly. Coming up with unique solutions to problems is what a lot of startups are founded on. They saw a specific problem and created a unique solution for it. I'll give a few examples here.

Netflix

Netflix saw a problem with Blockbuster's late fees and their solution was, simply, no late fees, in fact there isn't even a rental store. You choose which movie you want, it gets shipped to you, you watch it when you're ready and when you're finished you put it back in the mail. When Netflix receives the video back then they send you the next one in your list.

Apple

Apple saw the burgeoning homebrew computer movement starting, where geeks would get together to show off the computers they had built by hand. Both Steve Wozniak and Steve Jobs saw that in order for normal people to start using computers it would have to be more mainstream, people did not want to build their own computers by hand. So they created their very first Apple-1 computer and showed it off to the Homebrew Computer Club to get feedback from their first early adopters.

Hotmail

The two cofounders were working on a separate product but became frustrated by the fact that they could not access their personal email from behind their employer's firewall. This became the seed for what would become Hotmail, a way for them to anonymously access their own email through the web instead of having to download a program locally to do it.

Why Are Problem Solving Skills Important?

Problem solving skills are important to everyone, in our day to day lives we are constantly bombarded with decisions that require us to reason through the various options. Sometimes it may boil down to something incredibly simple like - do I have oatmeal or an omelette for breakfast. An omelette will give me more energy and I have a heavy workout to do this morning so that makes the most sense for me right now, but oatmeal is a lot quicker to make.

Common Problem-Solving Steps

Before we dive in and try and come up with some ways to improve your problem-solving skills it would be good to identify some common steps involved in problem-solving.

1. Identify the problem

First we need to identify the core problem, some good tools for how to do this are above so I'll just briefly reiterate them here:

• 5 Whys
• Think like an entrepreneur
• Think like a customer service/success rep

Bottom line - ask enough questions until you can go no deeper in what is actionable by you or your team.

2. Define the main elements of the problem

Next you need to define what the main pieces of the problem are. When you're talking about web development you could be talking about:

• Front-end issues (UI interactions not working correctly, errors not showing, or some other user-facing issue)
• Back-end issues (it could be business logic or some other part of the back-end stack causing the issue that will need to be addressed)
• Integration issues (all applications have multiple other systems they rely on to function properly - database, cloud services, etc. are any of these to blame?)
• Deployment issues (was something misconfigured on deployment? Are there not enough containers running? Is it a scaling issue?)
• Process issue (or maybe there was a process with a missing piece, or a process that wasn't followed for some reason)

All of these, and more, are possible pieces to the problem (and the resulting fix)

3. Find and examine possible solutions

Finding possible solutions is a tricky step in the problem-solving process. What I like to do is to get a group of people from the different areas - back-end, front-end, operations, devops and brainstorm. A brainstorming session should be time-boxed so that we aren't wasting too much time trying to come up with the most elegant solution. Depending on the type of problem though there may be time to do a few exploration spikes to examine some possible solutions after brainstorming.

When brainstorming I like to have a set agenda that looks something like this:

• Brief explanation of the problem or incident with data or examples to illustrate the issue.
• Brief Q&A period where notes are taken on this discussion as sometimes this results in other areas that we may need to examine later.
• 10 -15 Minute period where everyone writes down their idea of a possible solution.
• Throw solutions up on the board.
• Group the solutions and discuss them to see which ones are most viable.
• Outline action items or next steps with specific deadlines so everyone is clear as to who is responsible for what after the meeting is over.

4. Act on resolving the problem

Once you have decided on a set of possible viable solutions you will need to evaluate each solution based on some criteria and then execute on the plan. This is the criteria I tend to use when doing these kinds of evaluations.

• Time to fix
  • How long will it take to apply the possible solution?
  • Is this quick enough? Are we under a deadline to get something functional again?
• Degree of fix
  • Is it a band-aid solution or is it a long-term fix?
  • If we have to apply a band-aid now and a longer-term fix later, how do we make sure this is pushed to the top of the work pile?
• Test the fix
  • How will you test the fix once it is in place?

5. Look for lessons to learn

When the problem is solved, it's time to look back and see what happened to ensure that this kind of problem is mitigated as much as possible. I like to do public post-mortems whenever there is an outage that effected customers so that the whole company can learn from the problem and we can work together as a team to improve things.

What Are the Different Types of Problem-Solving Skills?

• Defining the Problem: Deeply understanding a problem through r5 Whys, customer research, or internal digging all lead us to being able to define the problem better.

• Brainstorming: Working alone you can probably brainstorm at least 2 or 3 possible solutions, but by working with others you can better bounce ideas off of each other and inspire each other. Time-boxing is a great way to ensure you keep the process moving as quickly as possible. Once time is up then organize all the solutions and group them together to see what patterns emerge.

• Analyzing: Evaluating each solution, using specific criteria (see above for time to fix, degree of fix and how you will be testing the fix) you can determine what some of the better solutions might be for the given situation. Many times there will be two possible solutions - one short-term fix that should be implemented now and one long-term fix to prevent this from happening in the future.

• Managing Risk: Each solution will come with its own set of risks. Make sure to document each one so that it's clear what could go wrong. Is there a way to reduce a given risk? Does the risk trail off over time?

• Deciding: Once the solutions have been narrowed down don't waste time in making a decision, sometimes failure to make a decision to "wait for more data" only compounds the issue. Make the best decision you can with the data you can. If there is significant risk that can be reduced by gathering more data make sure to weigh that in your decision making process.

• Managing Emotions: Many times, especially with a large group of customers being affected by an issue, tempers will be running high. Additionally people may be looking to play the blame game. Emotional intelligence or EQ can play a big role here to help you to remain calm and to focus on the problem and the solution. My favorite mantra was "I don't care who caused this issue, at this point I just want to get our customers back up and running as quickly as possible." Also, if you or your team was responsible for the incident, take ownership of it, but also focus on the end solution, not blaming someone. Egos may also get involved so it is important to try and remove your ego as much as possible. You are all working on this solution together, there is no need to get defensive.

Different ways to find solutions

One of the biggest pieces up there in that whole action plan is "Find and examine possible solutions" because it is sometimes the trickiest thing to actually do. Finding solutions takes a variety of different skills and sometimes a different way of approaching things.

List out all the possible solutions (yes, all, no matter how silly they might be)

By methodically trying to list out all possible solutions you're actually forcing your brain to go in different directions. Maybe one solution is to build a brand new application, or have people skip logging in altogether. Who knows? Set a timer for 10 - 15 minutes and write down as many solutions as you can think of. Don't stop writing until the timer goes off.

What is the simplest solution that could work?

As you go through your exercise you will undoubtedly see a variety of super-complex and super-simple solutions. Look at ones that are the simplest first. Is there something there? Can it be simpler?

Solve the wrong thing (think laterally)

Instead of trying to solve the problem, solve something else entirely opposite from the problem. This technique forces you to look at the problem from the opposite end and may free your brain up from its preconceptions about a solution.

Change your language

Instead of saying words like "We need...", "To fix this we should..." try words like "Imagine if..." or "If I had a magic wand I could ...". This language may shift how you think about a problem from a hard edge to something more soft and malleable.

General ways to improve problem solving

These are more geared towards longer-term improvement that help you stretch your muscles when you don't have a specific problem in front of you.

Start an idea journal

The basic idea here is that whenever an idea occurs to you about a problem you've witnessed, or a product that could be made better, jot it down in a notebook. The neat thing about this process is that the more you do it, the more that ideas will seemingly pop into your head!

If you really want to try exercising it further though you can try the following:

• Read 1 - 2 business articles that highlight some new application, process, or company and what they're doing
• Write down the problem the company is trying to solve and what their solution is
• Do some problem analysis of your own, or write down a different way that same problem could be solved

This helps in at least a couple of areas - it exposes you to different markets and problems that you may not be familiar with and it forces you to analyze things and come up with your own solutions.

Do logic puzzles

This more directly engages specific problem solving skills. Try something like word puzzles, sudoku, strategy games, etc.

Work out

Working out, especially with music can help increase blood flow to the brain and can improve cognition.

Share your ideas

Sharing your ideas and discussing them with others will help you see things from their perspective. Be sure to listen once you've shared though, it's important for the person you're sharing with to feel valued. Write down their input, don't respond right away, think about it and then come up with either a refinement or further questions.

Mind maps

Mind maps are great to show relationships of different ideas and helps you think about things in a different "space". Most people tend to think in lists, but forcing you to associate ideas and things together in a different arrangement can sometimes trigger new associations in your own mind.

Take a break

Sometimes what we most need is some time away from the problem. Go for a walk, switch to another task, whatever it might be. Let it settle in the back of your mind and focus on something else. You may find that while you're doing something else your mind is actually still working on the problem for you. Just be prepared to write down those notes when it pushes its way back to the forefront.

In Conclusion

Improving how you approach problems and their various solutions takes time, practice and communication with your team-mates. It also requires being able to recognize which tool works best for any particular situation.

Did I miss your favorite analysis or problem solving tool? Drop it in the comments below!

Get your free ebook

"The 30-Minute Project: How to improve Your Side Project Game to Become a Better Developer"
at 30-Minute Project