This article was originally published a day earlier at https://maximorlov.com/4-reasons-why-your-docker-containers-cant-talk-to-each-other/
After you finally containerise your Node.js app, you're eager to see if it works. You run it, but then this happens:
"Error: connect ECONNREFUSED"
Your application fails to connect to the database. But why? Connecting to the database from localhost works without a hitch. Also, the app used to work fine before without containers.
You looked for help in the official Docker docs, and even with those instructions, you can't get two containers to talk to each other.
Networking is a complicated topic. Add containers to the mix, and it becomes a real headache. You could read several books and spend days trying to understand the fundamentals of networking.
It would be much nicer to fix this seemingly trivial problem and get on with your day to work on features that matter.
In this article, you'll read four possible reasons why your containers might fail to communicate with each other and a fix for each of them. These quick troubleshooting steps could save you wasted hours on debugging connectivity issues.
Do the containers share a network?
Containers can only communicate with each other if they share a network. Containers that don't share a network cannot communicate with one another¹. That's one of the isolation features provided by Docker.
A container can belong to more than one network, and a network can have multiple containers inside.
To find out if two containers share a network, first list all the networks of one of the containers. It doesn't matter which one you pick. If container A can talk to container B, then by default in Docker networks, container B can also talk to container A.
# List all networks a container belongs to
docker inspect -f '{{range $key, $value := .NetworkSettings.Networks}}{{$key}} {{end}}' [container]
You should see a space-delimited output of all the networks the container is attached to. Then, for each of these networks, list all containers inside that network.
# List all containers belonging to a network by name
docker network inspect -f '{{range .Containers}}{{.Name}} {{end}}' [network]
If you don't find the container you're trying to connect to in any of the networks, then the containers don't share a network. You can attach an already running container to a network with the following command:
# Attach a running container to a network
docker network connect [network] [container]
You can also specify a network when you start a container with the --network (or --net) flag as follows:
# Start a container attached to a specific network
docker run --network [network] [container]
¹ An exception is when a container publishes/maps its ports to the host network. Other containers on the same host will have access to the container through the ports on the host. Publishing a container's port is meant for the container to be accessible from the public or the host system. I'd recommend to not rely on this for inter-container communication.
Is DNS resolution available inside the network?
DNS resolution allows you to connect to a container by using its name instead of the assigned IP address.
If you don't specify a network when you start a container, Docker will attach it to a default network. A limitation of this network is that it doesn't have built-in DNS resolution. To connect to other containers by their name, you'll have to create a network. If you're using Docker Compose, then a network is created for you.
To quickly find out if a container can reach another container by its name, you can use the handy CLI tool ping. What's nice about ping is that it comes pre-installed on most images, even on the minimal Alpine image.
# With containerA already running, test if containerA can connect to containerB by using its name
docker exec [containerA] ping containerB] -c2
You should see two packets being transmitted and received. If you see Name or service not known you cannot connect to that container by using its name.
If your containers are on the default Docker network, you can create a network with:
# Create a network
docker network create [network]
Subsequently, you can attach your containers to this network, as shown in the previous section.
Are you using the correct IP address?
IP addresses are everywhere. With so many of them, it's easy to mix them up and hard to know if you've grabbed the correct one.
A container belonging to more networks will have a different IP address within each network. When connecting two containers, you have to use their assigned IP addresses within the network they share.
You can use ping again to find out if an IP address is reachable from a container:
# Find out if an IP address is reachable from a container
docker exec [container] ping [ip_address] -c2
If you see lost packets and Destination Host Unreachable, then that IP is not reachable from the container.
After having figured out which network is shared by two containers, as shown in the first section, you can find out the IP of a container with this command:
# Get a container's IP address inside a specific network
docker inspect -f '{{.NetworkSettings.Networks.[network].IPAddress}}' [container]
Don't forget to replace [network] with the shared network's name. If you're not using a custom network, then you can fill in bridge — the default Docker network name.
Use this IP address to connect to this container from any other container on the same network. Look carefully at the output and preferably copy/paste it. IP addresses can differ by just a single number, which can easily be missed and lead to wasted hours debugging.
Is the network configured to block inter-container communication?
Containers that share the same network can communicate with each other by default. However, it can happen that your network's configuration changed at some point. In Docker, the setting responsible for this is called inter-container communication, or ICC.
To check if your network has ICC disabled, run the following command:
# Get ICC setting for a specific network
docker inspect -f '{{index .Options "com.docker.network.bridge.enable_icc"}}' [network]
If the output is false, ICC is disabled, and containers in that network cannot communicate with each other. The output can also be empty, in which case ICC is enabled because that's the default.
You can't change a network's configuration after creation, so enabling ICC on an existing network isn't possible. You'll have to create a network and attach all containers to the new one.
# Create a network and explicitly enable ICC
docker network create -o com.docker.network.bridge.enable_icc=true [network]
The -o com.docker.network.bridge.enable_icc=true part is optional since ICC is enabled by default.
Still not working?
If you went through all the troubleshooting steps and you still can't connect to a container, don't despair. Many things can go wrong in networking, and you're not the only one that finds this frustrating.
One thing you can do is to make sure it's indeed a connectivity issue and not something else. With a web server for example, if you get a 404 response but you can ping the container, then connectivity is not the problem. More likely, the endpoint you're trying to reach doesn't exist. I've seen this happen to others a few times so make sure to double-check the URL.
Write clean code. Stay ahead of the curve.
Every other Tuesday, I share tips on how to build robust Node.js applications. Join a community of developers committed to advancing their careers and gain the knowledge & skills you need to succeed.
Top comments (1)
Very usefull - Maybe we should test IP before DNS :)