Initially, I planned to publish all how-to s for advanced network configurations for virtual machines in one article. However, I am unstoppable when it comes to writing, and the article became quite long. So, I decided to split the theory from the hands-on configurations, XML/Shell scripting, etc.
You might think you don’t need to read this, but I strongly discourage you from jumping directly to the next article and blindly copying my configurations without understanding what you’re doing.
NB! I’m not a network engineer, nor have I taken courses on this. I’m actually just a dev who’s just passionate about Debian and spends way too much time on my PC. So, please don’t throw slippers at me if I end up sharing any misleading information. I’m simply documenting and describing some of my experiments in this field.
Terminology I will use in this article, sometimes in acronym form – I try to avoid it, but sometimes it just happens automatically:
- Host – This refers to your PC, on which you set up virtualization and create virtual machines.
- Guest – This is any virtual machine you create. VM/VMs – A virtual machine/Virtual Machines.
- ISP – Internet Service Provider.
- LAN – Local Area Network.
- NAT – Network Address Translation.
-
DEFAULT network – This refers to the Libvirt's virtual network that virtual machines are connected to by default, within the scope of
qemu:///system. If you do not understand the difference betweenqemu:///systemandqemu:///sessionand how to switch between them, please refer to the previous article. In this article I will be creating/configuring VMs that are in the scope ofqemu:///system. - Packet - A network packet is a formatted unit of data carried by a network.
When it comes to networking, it’s really hard to decide where to even start explaining. To avoid turning this article into some kind of networking handbook—which I don’t have the expertise to write anyway—I’ll probably have to skip over some fundamental concepts.
However, I’ll do my best to simplify things and provide schemes for the main concepts so you can (hopefully!) follow along.
Here is the road-map for this article:
➀ About Public IP address and LAN
- ➀.➀ Inbound vs Outbound traffic
➁ Understanding your LAN and private IP address(s) of your host machine
- ➁.➀ Network Interfaces: general information
- ➁.➁ Network Interfaces: MAC addresses
- ➁.➂ IPv4 vs IPv6 addresses
- ➁.➃ IPv4 address ranges reserved for private networks
- ➁.➄ IPv4 addresses structure and CIDR
➂ Libvirt's DEFAULT Network: about NAT mode
- ➂.➀ About virtual network switches
- ➂.➁ Libvirt wants
iptables, Debian hasnftables: what to do? - ➂.➂ DEFAULT Libvirt's network: what's under the hood?
- ➂.➃ Nftables rulesets: tables and chains explained
- ➂.➄ About how NAT works
Let's start!
I’ve already written some explanations of how your home network works, about public IP address, and ports in this article.
Here is the schematic representation of the most common setup of local "home" network with WiFi router playing the central role in it:
➀ About Public IP address and LAN
The most important takeaway from this scheme is that all your devices—whether they connect to your Wi-Fi router wirelessly or physically (via Ethernet cable)—are part of a local network, your home network. This LAN (Local Area Network) is created and managed by your Wi-Fi router.
Most likely*, only your router has a public IP address! Your devices do not (I wrote most likely because if you’ve configured your Home Lab to use IPv6, the story is quite different. But I’m guessing you wouldn’t be reading this article if you were capable of doing so! :) ) Instead, each device connected to your Wi-Fi network is assigned a private IP address—one private address per device. If, for some reason, something gets messed up with your router and it assigns the same IP address to two devices, those devices will start having problems accessing the internet (they will start losing some packets - for example, pinging will be unstable, with some percentage of packets lost).
You can check your public IP address using website like DNS Leak Test, which will show you IP address with which you visited this site (what is DNS and DNS "leaking" I will explain later in this article). Your Public IP address is discovered by servers hosting the websites you visit. For example, when I upload an image here in articles, the Dev.to servers see the request coming from my public IP address that’s pushing the data.
➀.➀ Inbound vs Outbound traffic
For now, what you know for sure is that a Wi-Fi router is the magic box that allows you to access internet resources from your devices. This is quite obvious because you paid for exactly that. But how does it actually work?
In the previous part of this series, I mentioned that I couldn’t host anything on my PC - a website - without tweaking the router. In general, when someone tries to reach you via your public IP address, they send you packets with some data/requests. These packets first reach your Wi-Fi router as it is the one having public IP address, not your devices connected to WiFi! What happens next depends on your router’s configuration.
Will it drop the packets, effectively blocking them, or will it redirect them to the appropriate member of your local network (for example, your PC)? If it does redirect, the router uses the private IP address (as he knows all the private addresses of connected devices) of the target device (your PC) to forward the packets. Router will also have to forward the packets to the correct port. However, for all of this to happen, the router needs to be explicitly configured. By default, most Home-purpose routers will just block incoming traffic, and the packets will simply be dropped. This is why I cannot host any website on my PC without modifying the configurations of WiFi router.
However, I can download whatever I want by default. When I download something, it comes to my computer in form of packets with data as well. And the stuff that I am downloading is coming in, not out. So, it is also a sort of incoming traffic, and it does not get blocked by WiFi router at all.
The key difference in these two examples is who initializes the communication. When you download something—i.e using wget—it’s your device that sends the request, initializing the connection. The server responds, sends packets with requested data, they arrive to your WiFi router, it redirects them to the device that made a "request" - and that’s why it works - WiFi router does not impede this process, because it is outbound traffic.
Inbound traffic, on the other hand, refers to situations where the connection isn’t initiated by your device—like when uninvited guests show up at the door of your house. In those cases, the firewall rules of your router sends them away. These rules protect your local home network from unexpected or unwanted traffic.
In the scope of this article on virtualization, the focus will be on networking "locally". It won’t be about creating networks that make your VMs publicly accessible, vulnerable, or anything of that sort. I won’t touch my router configurations, and all configurations will be done on networks that operate behind a Wi-Fi router.
➁ Understanding your LAN and private IP address(s) of your host machine
As I mentioned, the Wi-Fi router creates a LAN (local area network). Any devices you connect to the Wi-Fi join this LAN, and their communication—both with each other (if configured so) and with the outside world (web) —is managed by the Wi-Fi router.
The first thing to do is to get familiar with the private IP addresses of your host machine - what do they mean, why they are like this, how to manage them.
If you’re using ONLY Network Manager to handle your network connections... well, put it aside for now. Your new best friend is the iproute2. It should already be installed by default on Debian. The thing is, when you install Network Manager, it can start conflicting with iproute2 configurations. However, on Debian, this is partially resolved by the default setup, where Network Manager only manages wireless network interfaces.
➁.➀ Network Interfaces: general information
Now, let’s take a look at what’s going on with the networks on my host machine.
$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> ...
....
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> ... state UP ....
....
3: wlxXXXXXX43643754XX: <BROADCAST,MULTICAST,UP,LOWER_UP> ... state UP
...
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> .... state DOWN ...
....
What I have:
lo: This is the loopback interface. It’s a virtual network interface used by my OS to communicate with itself. It’s always there—no need to touch it! It won’t participate in the networking configurations I’ll be covering in this article.
eno1: This is my network interface that comes with the Ethernet cable that physically links my PC to the Wi-Fi router. *This guy will be playing the key role in the networking setups for my VMs.
*
wlxXXXXXX43643754XX: This is the wireless network interface that comes from my USB Wi-Fi adapter. In my setup, it has higher priority - this interface is used for communications with WiFi router and within the LAN. However, this wireless network interface WILL NOT participate in the networking setup for the virtualization process because it’s problematic. Wireless network devices use different drivers, different protocols, and it isn’t trivial to make them to participate in virtual networks or bridging. While it’s definitely possible, I already have a physical cabled connection, so why complicate my life?
virbr0: The last guy in the list, which is currently DOWN, is a virtual bridge. It’s managed by libvirt/qemu. It is created when the libvirt daemon is first installed and started. However, it remains down unless any libvirt's network that uses this network interface starts (i.e sudo virsh net-start default).
Let's look into more details about each network interface:
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> ...
....
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> ... state UP ....
link/ether 12:ab:c3:d4:ef:56
inet 192.168.1.X/24 brd 192.168.1.255
inet6 fe80::278e:1234:l678:123/64
3: wlxXXXXXX43643754XX: <BROADCAST,MULTICAST,UP,LOWER_UP> ... state UP
link/ether 98:zy:xb:67:kl:34
inet 192.168.1.Y/24 brd 192.168.1.255
inet6 fe80::kre7:b5b5:z8y6:987/64
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> .... state DOWN ...
link/ether 77:mo:5g:k9:r0:46
inet 192.168.122.1/24 brd 192.168.122.255
In this article section, I will explain what are these values: link/ether, inet and inet6.
➁.➁ Network Interfaces: MAC addresses (link/ether)
First, I want to point your attention to the fact that eno1 and wlxXXXXXX43643754XX Network Interfaces are providing the connection to the same network, my home local network (LAN), managed by my Wi-Fi router. Just the origin of eno1 is the cable (Ethernet), and the origin of wlxXXXXXX43643754XX is the Wi-Fi USB adapter, so a wireless connection. Why do they have different IP addresses even if they are "attached" to the same network?
All other devices connected to my house Wi-Fi have private IP addresses - like the laptop at 192.168.1.A, the phone at 192.168.1.B, the PlayStation at 192.168.1.C, etc. So why does my single desktop PC have two addresses in the same network? Does this cause confusion?
My PC can connect to a home local network (and to any network that is not virtualized by the PC itself=existing only "inside" the PC) through a NIC—Network Interface Card/Contrtoller. This is a hardware component that is often integrated into the motherboard.
If you’re using a laptop that doesn’t have a port for an Ethernet cable, it most likely doesn’t have a NIC either. So, even if you imagine buying something like a USB-to-Lightning-to-Type-C adapter just to connect an Ethernet cable to port of Type C, it won’t work. This is because it’s not just about the "shape" of the cable, but the capability to process the type of communication that comes through it - and this capability is granted by NIC.
The solution is to purchase a proper USB hub with a built-in NIC and Ethernet port. The laptop example is a good one because it shows that an Ethernet cable is not the cornerstone of connectivity to networks. Such laptops indeed connect wirelessly, if a network of interest allows so. In this case, they do not use a default NIC but rather a WNIC—a Wireless Network Interface Card.
A WNIC in a desktop computer often needs to be purchased separately—either as a PCIe network card (to be attached to the motherboard's PCI slot) or as part of a USB Wi-Fi adapter.
In my case, I have them both, integrated NIC and WNIC of WiFi USB adapter:
#list of Ethernet controllers
$ lspci | grep -i ethernet
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. Controller
#list of WNIC
$ lsusb
Bus 002 Device 002: ID 1234:5678 TP-Link 802.11ac NIC
My Wi-Fi router perceives connection from each (W)NICs as a separate initialization of a connection and assigns a distinct private IP address to each. That’s why my PC, which has two network interfaces, ends up with two IP addresses—one for each (W)NIC. The router cannot identify that both NICs physically belong to the same device (my PC) because the physical identifiers are the NICs themselves (not a CPU, not a motherboard ecc)!
Each NIC has its own unique MAC address (Media Access Control address), which serves as the unique permanent identifier. Not only network interfaces, but every piece of network-connected hardware, MUST HAVE a unique MAC address in a network. This is different from an IP address because a MAC address is permanent: every NIC has one and only one MAC address, hardcoded by the manufacturer (though, sometimes it is possible to change it). You can see the MAC addresses of your network interfaces using ip link or ip a commands—in the lines labeled link/ether.
While MAC addresses are permanent, an IP address is temporary and can change quite often - every time a device connects to a network, it is possible that WiFi router can assign to it a new private IP address (the probability of it depends on the network configuration and the device’s own settings).
You can think of a MAC address as a device’s permanent name, while an IP address is more like instructions for other devices on how to communicate with it. For example, your device might always be MAC number A, but at any given moment, it can be located at IP address B or IP Address C ecc.
Now that I’ve covered MAC addresses, it’s time to explain IP addresses.
➁.➂ IPv4 vs IPv6 addresses and NAT
To see the IP addresses of your PC's network interface(s) (roughly speaking Private IP address(es) of your PC) you can execute ip a command. You can scroll up and take another look at my output of this command. Now you know that the link/ether field tells you the MAC address(es) of network interfaces. If you’re familiar with IP addresses, you can guess that inet field exactly contains IP address information (the address though which other devices on the same network can communicate with PC). However, 'inet' field is not the one that shows you IP address of a network interface. inet6 field (if you have it) shows you another address and it is also IP address (even if it looks more like a MAC address)!
My PC's eno1 interface (Ethernet) has inet 192.168.1.X/24 and inet6 fe80::278e:1234:l678:123/64; wlxXXXXXX43643754XX interface (wireless) has inet 192.168.1.Y/24 and inet6 fe80::kre7:b5b5:z8y6:987/64; virbr0 virtual network interface managed by libvirt has only inet 192.168.122.1/24.
Both inet and inet6 fields of ip a output contain completely valid IP addresses - it’s not like what you see in inet6 field is just an encrypted, altered, or transformed version of inet field value, nor does it indicate a completely different network. The value of inet field is the IPv4 address, while inet6 displays the IPv6 address.
IP stands for Internet Protocol and it has two major versions: IPv4 and IPv6. IPv6 is the newer, more advanced version, offering enhanced features, greater capabilities, and significant potential for addressing future network demands.
In a previous article, I already touched on the topic of IPv4 vs. IPv6 differences. I’ll share the most relevant takeaway as it relates to this article:
A unique public IP address is a scarce resource! Actually, a unique public IPv4 address is in deficit. Internet Protocol version 4 (IPv4) forms the foundation of most Global Internet traffic today. An IP Address represented under IPv4 is composed of four sets of numbers ranging from 0 to 255, separated by periods(.).
If you do the straightforward math - total four numbers in an IPv4 address; each number can be in range between 0 and 255 (256 possible values) - 256 * 256 * 256 * 256 = 4,294,967,296 total addresses.
The first distinction is that IPv6 addresses are not just numerical; they are alphanumeric. IPv6 can provide many more unique addresses than the IPv4 system (340 trillion trillion trillion unique addresses vs 4 billion). But this is not the only difference. IPv6, as a protocol, is more modern and from the beginning was designed to be more secure.
It is not like IPv4 was designed without security in mind, just when it was created, the internet had far fewer users, and the number of devices worldwide was significantly lower. Let me illustrate how it affects the security mechanisms.
Security mechanisms, such as encryption, inevitably introduce additional computational load. For example, imagine my PC is communicating with your PC using an encryption mechanism. All the packets we exchange are heavily encrypted, and our devices communicate using public IP addresses, which act like the destination addresses for our data packets (like mailing letters). My device with its public IP address communicates with your device at its public IP address.
This scenario demonstrates the end-to-end principle of communication: my device can communicate with yours securely, with security mechanisms implemented directly in the communicating end nodes (our PCs, that have encryption/decryption keys). The intermediary nodes (like gateways and routers) don’t take part in our secure communication.
So, the Gateways and Routers are bad? Not at all! However, in this example, gateways and routers can A) introduce computational overhead B) make some encryption mechanisms impossible.
Getting closer to the topic... This idealized scenario of end-to-end communication is possible today, but only if we use IPv6 and our devices are properly configured to support it. Why? Because IPv6 adreesses poll is really huuuge, so every device can have a public IPv6 address, enabling direct communication between them without needing intermediary translation.
Each packet that we exchange in my example as any network packet consists of control information (headers) and user data (payload). Control information provides data for delivering the payload (e.g., source and destination network addresses, error detection codes, or sequencing information).
So, in case of IPv4. My PC sends to my WiFi router a packet that it is meant for you. But my PC does not have a Public IP address! So the source IP address in the headers is abracadabra for the web scope and it needs to be_ translated_, according to some rules. This job is done by NAT - Network Address Translation mechanism of my WiFi router. And here it is why this intermediary node in our communication is a baddie - NAT introduces processing overhead because it rewrites packet headers for every packet that passes through the router. And this happens not only on my side, but also on your side!
Some encryption techniques are meant for end-to-end communication only with consistent IP addresses - so they are not possible if NAT (in its default form) is in the middle.
With IPv4, gateways, routers become unavoidable due to the scarcity of public IPv4 addresses, and so does NAT. I started introducing NAT right here; however, I will elaborate more on this in the next sections, as the libvirt's DEFAULT network is based on the NAT mechanism.
_A little thought experiment: how do you think the internet would be if IPv6 were fully adopted and replaced IPv4? What would happen to the prices for internet service? Would the speed of the internet change? PS. you can check out what is CGNAT :) _
In this article, I will use IPv4 addresses for any network configuration. Unfortunately, Internet Protocol version 4 (IPv4) still forms the foundation of most global internet traffic today. Plus, as I stated earlier, this article is focused on creating a local network rather than configuring any VM to be remotely reachable from the outside.
The scope of IPv4 reserved addresses for private networks is quite large for a home setup, so let's move to it.
➁.➃ IPv4 address ranges reserved for private networks
According to standards set forth in Internet Engineering Task Force (IETF) document RFC-1918 , the following IPv4 address ranges are reserved by the IANA for private internets, and are not publicly routable on the global internet:
10.0.0.0/8 IP addresses: 10.0.0.0 – 10.255.255.255
172.16.0.0/12 IP addresses: 172.16.0.0 – 172.31.255.255
192.168.0.0/16 IP addresses: 192.168.0.0 – 192.168.255.255
Note that only a portion of the “172” and the “192” address ranges are designated for private use. The remaining addresses are considered “public,” and thus are routable on the global Internet.(Source)
Key takeaway is that there is set of IPv4 IP addresses that are reserved for private networks. The reservation ensures that there will be no conflicts with global - public IP addresses.
➁.➄ IPv4 addresses structure and CIDR
All IPv4 IP addresses has the same structure - 4 numbers divided by dots: x.x.x.x. The trailing slash with a number after it is not a part of IP address, it is CIDR (Classless Inter-Domain Routing) notation.
Take a detailed look at this again:
- 10.0.0.0/8 IP addresses: 10.0.0.0 – 10.255.255.255
- 172.16.0.0/12 IP addresses: 172.16.0.0 – 172.31.255.255
- 192.168.0.0/16 IP addresses: 192.168.0.0 – 192.168.255.255
The largest network range (with the most IP addresses) is represented by 10.0.0.0/8. The CIDR number after the slash (e.g., "/8") is not random; it refers to the number of bits allocated for the network portion of the IP address.
A "/8" means that the first 8 bits are reserved for the network, leaving 24 bits for the host part. This allows for 16,777,216 available addresses for devices (hosts) within the network. The smaller the CIDR number (like "/8"), the larger the number of available IP addresses, because fewer bits are used for the network portion, and more are left for devices.
The IP IPv4 address is 32 bits long, divided into four groups (octets) of 8 bits each. For example, 192.168.1.0 is written as:
192 . 168 . 1 . 0
11000000.10101000.00000001.00000000
Each number is an 8-bit block (each block represents one of the four octets). The CIDR prefix tells how many of these 32 bits are used for the network portion.
Lets return to my outputs from ip a:
- eno1: inet 192.168.1.X/24 brd 192.168.1.255
- wlxXXXXXX43643754XX: inet 192.168.1.Y/24 brd 192.168.1.255
- virbr0: inet 192.168.122.1/24 brd 192.168.122.255
eno1 and wlxXXXXXX43643754XX network interfaces connect my PC to the local network managed by my WiFi router. Router assigned them private IP address 192.168.1.X and 192.168.1.Y. These IP addresses were vacant, so they were assigned by router's DHCP (Dynamic Host Configuration Protocol). CIDR /24 tells me that in this local network there are 256 "spots" for devices:
/24 means the first 24 bits are for the network. This leaves 8 bits for the host (the devices within the network). Network bits: 24 bits; host bits: 8 bits; 2^8 = 256 possible IP addresses in this network (but in practice, some are reserved for special purposes).
The virbr0 network interface is a different case. First, it is managed by libvirt/QEMU, as I mentioned, not by my WiFi router, and it is a virtual bridge. The Network to which this network interface connects also has a CIDR of /24, meaning there are 256 available addresses.
However, remember the last information I shared in the previous part of this series on virtualization? RECAP: I created a VM connected to the DEFAULT network, and I mentioned that even though it can reach the internet, no device connected to my local home network can access this VM via SSH, ping, or anything else (except HOST!). This is because any 192.168.122.X address is not part of my local home's 192.168.1.X/24 network! The 192.168.1.X/24 range covers addresses from 192.168.1.0 to 192.168.1.255, but any 192.168.122.X address is outside of that range. As a result, there’s no connection, no communication.
After this long introductory session, let’s get back to virtualization with virsh, QEMU, and KVM, focusing on network configurations.
➂ Libvirt's DEFAULT Network: about NAT mode
libvirt uses the concept of a virtual network switch. The network interface you saw in my ip a outputs in the previous section, called virbr0, is nothing more than a virtual network switch automatically created and managed by libvirt.
➂.➀ About virtual network switches
I don’t know if you’re familiar with physical network switches, but these guys look like this (not so friendly for trypophobic folks, hehe):
Getting closer to understanding networks: my PC has an Ethernet cable that’s plugged into the Wi-Fi router. But what if I plug it into my laptop instead? Haha, I can’t do that because my laptop doesn’t have an Ethernet port. However, if I could, that connection would create a small network of two devices (my PC and my laptop), allowing them to communicate with each other.
But what if I wanted to attach something else—a second laptop? It would be pretty hard to do with the scarcity of Ethernet ports (even if both laptops had one Ethernet port).
If I had a network switch, though, I could plug in all the devices I wanted—up to the number of available ports on the switch. The network switch would handle the communication between them by forwarding packets between connected devices!
So, the virtual network switch works kinda the same way, just for VMs. The default virbr0 virtual network switch is used when VMs are connect to the DEFAULT network. This virtual network switch enables them to communicate easily with each other.
Here is the DEFAULT network:
$ sudo virsh net-list --all
Name State Autostart Persistent
--------------------------------------------
default active no yes
This DEFAULT network operates in Network Address Translation (NAT) mode (the one I introduced above discussing IPv4 vs IPv6).
By default, a virtual network switch operates in NAT mode (using IP masquerading rather than SNAT or DNAT).
This means any guests connected through it, use the host IP address for communication to the outside world. Computers external to the host can't initiate communications to the guests inside, when the virtual network switch is operating in NAT mode. (Libvirt: Virtual Networking)
Libvirt's documentation specifies, that the NAT is set up using iptables rules.
And here we arrive at another important player in the whole networking process – iptables. It is often perceived juts as a tool that protects your system from unauthorized access/traffic based on some rules. However, the reality is that it is much, much more than that, as it can also be used for traffic manipulation, forwarding, NAT (Network Address Translation), and more.
Iptables provides packet filtering, network address translation (NAT) and other packet mangling.
NOTE:iptableswas replaced bynftablesstarting in Debian 10 Buster. (Debian Wiki: iptables)
I am using Debian Sid, so I definitely have nftables and not the legacy iptables.
Is nftables just a modern version of iptables with fixed vulnerabilities, faster performance, and so on, but still it is just theiptables under the hood? NO. They are two different frameworks designed to do the same job—'mangling' network traffic. However, think of 'different frameworks' in this context as you would if you have experience with Python: it's like TensorFlow and PyTorch. In web development, it's like React and Angular. You cannot write a neural network using PyTorch and then expect to just copy-paste the network source code into TensorFlow and have it work. The same goes for nftables and iptables. They are different, with different syntax, and different logic, especially when it comes to IPv6 traffic.
It's better not to mix the two, thinking, 'Oh, for an issue Y I'll write and add some rules in iptables, but then for the issue X I found a tutorial for nftables, so I'll add rules this way'. (Played Witcher 3? Remember what happened to Geralt when he was courting both Triss and Yennefer? Well, the same can happen to your network traffic if you start playing around with different tools for network traffic management)
However, even if you do so (use both the nftables and the legacy iptables tool at the same time), Debian has you covered in a certain way. First, the iptables utility is not installed on the system by default. If it is installed, the iptables utility will, by default, use the nftables backend. But, again:
Should I mix nftables and iptables/ebtables/arptables rulesets?
No, unless you know what you are doing. (Debian Wiki: nftables)
➂.➁ Libvirt wants iptables, Debian has nftables: what to do?
As I mentioned before, libvirt's DEFAULT network functions in NAT mode, and this NAT mode is defined using iptables rules. So, when you installed libvirt tools, it most probably pulled in iptables as a dependency. Indeed:
$ aptitude why iptables
i libvirt-daemon-system Depends libvirt-daemon-driver-nwfilter (= 10.10.0-3)
i A libvirt-daemon-driver-nwfilter Depends iptables
But I already discouraged you from using iptables, hehe. So, what's the plan? First, most likely, you don't even have nftables up and running yet :D. Because if you did, and it was running as a systemd service, you would have encountered some troubles starting VMs with the DEFAULT network. I'll show you why:
#if for some reason you do not have nftables installed:
# $ sudo apt install nftables
$ sudo systemctl status nftables
#is it active? No? Then, start it
$ sudo systemctl startnftables
# and ENABLE it so it will start on the Boot
$ sudo systemctl enable nftables.service
FYI: Take a look at the nftables ruleset "in use" using command sudo nft list ruleset when nftables.service is stopped and when it's started (or before starting it and after). Compare the two and try to find libvirt's NAT configuration :).
#And now I bring UP DEFAULt libvirt network
$ sudo virsh net-start default
error: Failed to start network default
error: internal error: Failed to apply firewall command 'nft -ae insert rule ip libvirt_network guest_output iif virbr0 counter reject': Error: Could not process rule: No such file or directory
insert rule ip libvirt_network guest_output iif virbr0 counter reject
OOPS! I broke everything. Libvirt wants iptables, not nftables.
Whatever wants Libvirt, but in Russian there is saying: "eat what is given". There’s a legit solution to make libvirt work well with nftables. And it’s not just some workaround.
Maybe you are familiar with UFW—Uncomplicated Firewall—this tool is built on top of iptables, making it easier to manage iptables rules. You write simplified rules, and it translates them into iptables rules in the underground. A similar interface also exists for nftables! It’s called firewalld.
First, let’s clarify why this isn’t just a workaround that adds extra software to your system just to make something work for VMs:"
You should consider using a wrapper instead of writing your own firewalling scripts. It is recommended to run
firewalld, which integrates pretty well into the system. See also https://firewalld.org/ (Debian Wiki: nftables)
And...:
The firewalld software takes control of all the firewalling setup in your system, so you don't have to know all the details of what is happening in the underground. There are many other system components that can integrate with firewalld, like NetworkManager, libvirt, podman, fail2ban, docker, etc.(Debian Wiki: nftables)
So, firewalld, besides all its perks in firewall configuration and network traffic management, is the bro that will make libvirt function correctly with nftables. It will ensure that all the NAT rules, written with love by the libvirt devs, are active. That means the DEFAULT network based on NAT rules will work again.
$ sudo apt install firewalld
$ sudo systemctl start firewalld
$ sudo systemctl enable firewalld
#have a look on the new ruleset firewalld brought with it to nftables
$ sudo nft list ruleset
What about libvirt's ruleset for NAT?
$ sudo nft list ruleset | grep libvirt
#nothing
#now i start DEFAULT network
$ sudo virsh net-start default
Network default started
$ sudo nft list ruleset | grep libvirt
iifname "virbr0" jump mangle_PRE_libvirt
iifname "virbr0" jump nat_PRE_libvirt
iifname "virbr0" oifname "virbr0" jump nat_POST_libvirt
...
$ sudo nft list tables
table inet filter
table inet firewalld
table ip libvirt_network
table ip6 libvirt_network
So, there's a separate "table" for libvirt_network. This table is managed by libvirt, and libvirt fills this table with rules when its virtual network switch becomes active (virbr0). This happens only if at least one network using the virtual bridge virbr0 is started.
➂.➂ DEFAULT Libvirt's network: what's under the hood?
As I mentioned before, the libvirt DEFAULT network's NAT is set up using iptables rules. And in my case Libvirt is forced to use what is available on my Debian - nftables. So, if I explore existing rules in nftables, I for sure should find there Libvirt's default network rules, according to which traffic circulate between VMs and FROM VMs TO the "outside world".
And here's how the network magic happens for VMs: when they interact with each other and with the host, and access the internet to fetch whatever they're commanded to:
$ sudo nft -a list ruleset
table ip libvirt_network { # handle 6
chain forward { # handle 1
type filter hook forward priority filter; policy accept;
counter packets 0 bytes 0 jump guest_cross # handle 7
counter packets 0 bytes 0 jump guest_input # handle 5
counter packets 0 bytes 0 jump guest_output # handle 3
}
chain guest_output { # handle 2
ip saddr 192.168.122.0/24 iif "virbr0" counter packets 0 bytes 0 accept # handle 13
iif "virbr0" counter packets 0 bytes 0 reject # handle 10
}
chain guest_input { # handle 4
oif "virbr0" ip daddr 192.168.122.0/24 ct state established,related counter packets 0 bytes 0 accept # handle 14
oif "virbr0" counter packets 0 bytes 0 reject # handle 11
}
chain guest_cross { # handle 6
iif "virbr0" oif "virbr0" counter packets 0 bytes 0 accept # handle 12
}
chain guest_nat { # handle 8
type nat hook postrouting priority srcnat; policy accept;
ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 1 bytes 40 return # handle 21
ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return # handle 20
meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 # handle 19
meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 # handle 18
ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade # handle 17
}
}
➂.➃ Nftables rulesets: Libvirt Network's table and chains explained
Let’s break it down:
First, libvirt_network rules have their own table. In nftables, tables act as "containers" within the overall ruleset (to see the full list: sudo nft list ruleset). Tables contain chains, sets, maps, flowtables, and stateful objects.
Each table belongs to exactly one family. If you want to apply a specific set of rules to some network traffic, you must first define the table by specifying its type. The type determines that only traffic of this specific type will be filtered by the rules in that table.
For example, the table ip libvirt_network rules only filter IPv4 traffic/packets because its table is assigned the ip family:
table ip libvirt_network
A chain is essentially a list of rules. In the case of libvirt_network ruleset, everything starts with the forward chain, which is responsible for filtering incoming traffic packets.
table ip libvirt_network {
chain forward { #rules}
chain guest_output {#rules}
chain guest_input {#rules}
chain guest_cross {#rules}
chain guest_nat {#rules}
}
______ A |-----------------|F
|packet| ---> C | chain forward |I ---|
¯¯¯¯¯¯ C | type: filter |L |
E | hook: forward |T |
P | policy: accept |E |
T |-----------------|R |
-----packet----> |
|
Routing decision:
<---?--- | |----?---->
|-----------------| | -----------------|
| | ? |
|------------------------| | | |------------------------|
| chain guest_input | | V | chain guest_output |
| inbound packets | | | outbound packets |
| destined for guests | | | from guests |
| ? | | | ? |
|------------------------| | |------------------------|
| | |
|-----------|------------| | |-------------|----------|
| | | | |
+ACCEPT REJECT- | +ACCEPT REJECT-
+If: Anything- | +If: Anything-
+outgoing else- | +source IP: else-
+interface: - | +192.168.122.0/24 -
+virbr0 - | + -
+ - | +incoming -
+connection - | +interface: -
+state: - | +virbr0 -
+established OR - | +++++++++++++++++++++++++++
+related - |
+ - | |--------------------------|
+destination: - | | chain guest_cross |
+in 192.168.122.0/24 - |------| traffic between |
+++++++++++++++++++++++++++ | guests on virbr0 |
| ? |
|--------------------------|
+ACCEPT REJECT-
+If: No rule-
+incoming interface: -
+virbr0 -
+ -
+outgoing interface: -
+virbr0 -
++++++++++++++++++++++++++++
##########################################################
# Routing decision is made! #
# IF: #
# outbound packets from guests #
##########################################################
:==========: ______ :==========:
:guest VM : --> |packet| : Other VM :
:==========: ¯¯¯¯¯¯ :==========:
| ^
V |
|------------------------| |--IF traffic FROM: NO NAT
| | | (NO masquerade)
| chain guest_nat |--->| guest subnet TO: ^
| type: nat | | (224.0.0.0/24) OR --------|
| hook: postrouting | | (255.255.255.255)
| priority: srcnat; | |==================================
| policy: accept; |--->| IF traffic FROM:
| | | guest subnet TO: ---------|
| -----------------------| | DIFFERENT subnet V
| MASQUERADE traffic (NAT)
|
INTERNET <--------
I hope this ASCII scheme brought some clarity to how VMs attached to the DEFAULT libvirt's network communicate with the host, with each other, and with the outside world. Here’s some more info.
A network packet pops up on the virbr0 interface and then gets filtered (by chain forward) and routed or dropped - based on the nftables rules.
For example, when one VM sends a packet to another VM (if both are connected to DEFAULT network, it’s routed to that VM. The rules in chain guest_cross are triggered—there’s no rejection, so the packet is delivered without any special mangling, since it goes from virbr0 to virbr0 network interface. Chain guest_nat also participates, but NAT does not apply here because it’s an internal connection, and the rules say that when it’s from the same subnet to the same subnet, there’s no NAT.
Another example: the host sends a packet to a VM. This triggers the chain guest_input. The connection tracking state is established in this case (the VM has already initiated a connection to the host). And here it is! This is why you cannot connect to the VM remotely (even from another device connected to the same local home network as host and configured port forwarding!)—it will be new connection, not established!
Also, if you try to connect to the VM from a laptop on the same Wi-Fi as the host, it gets dropped because the VM is on a different network (remember, 192.168.122.x is not a part of 192.168.1.0/24 network!). But you configured port forwarding? Still nope. Packets will be dropped. Because again, it’s a new connection.
Yet another example: the VM wants to run sudo apt update && sudo apt upgrade. It needs to fetch data from the Debian repositories. This is governed by chain guest_output, so the rules are triggered and they don’t cause the drop of packets. Then the guest_nat chain is activated because traffic is going from the guest subnet to an external subnet. Without NAT, it wouldn’t work at all—the VM network is different from your local home network (WiFi), so they don’t directly communicate. They’re isolated from each other, and virbr0 doesn’t bridge these two networks!
To summarize: there are only two possible outcomes for the network traffic you initialize - FROM and TO your virtual machines that are attached to the DEFAULT network—it will either reach its destination (ACCEPTED) or be dropped (REJECTED).
I hope it’s clear when it comes to host-to-VM, VM-to-host traffic, and communication between VMs. 1) There are no specific nftables network filtering/mangling rules blocking it. 2) These communications are possible by the specifically configured and virtualized virbr0 virtual network switch.
However, HOW does traffic TO the internet—and especially FROM other networks (like your home LAN) work — can still be challenging to understand. NAT (Network Address Translation) might still appear like a black box. So, in the final section of this article, I’ll try to simplify things and explain it in detail.
Understanding NAT is crucial for all IPv4 traffic, as NAT has become a widespread networking configuration. This is due to the inherent limitations IPv4 faces in the modern world, making NAT a kind of symbiotic solution for IPv4 networks.
➂.➄ About how NAT works
Let’s get back to the NAT chain of the libvirt_network ip table. This chain doesn’t just apply filtering rules to network traffic—it handles postrouting rules, meaning it sees all packets after routing, right before they leave the local system.
For example, imagine your VM wants to connect to the internet to update itself when you run sudo apt update. This command fetches the data about the versions of your system’s packages against the repository versions. The packets from the VM are sent to http://deb.debian.org/debian/ (the Debian package repo source link).
These packets (fetching request) end up on virbr0, which acts like an airport for them. It decides if these “passengers” (packets) need to take an “international flight” (outside the guest VM network) or a “domestic flight” (within the guest VM network).
NB! I will cover DNS servers and gateways that act as routers in detail in the next article! For now, let’s simplify things (even though it’s not fully technically accurate) and say that the virbr0 virtual network switch handles this somehow, so I can focus on explaining NAT (otherwise I will never finish this article T_T).
Let's get back to the postrouting rules specified in the guest_nat chain and disaamble them to see how they worl
$ sudo nft -a list ruleset
...
chain guest_nat { # handle 8
type nat hook postrouting priority srcnat; policy accept;
ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 1 bytes 40 return # handle 21
ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return # handle 20
meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 # handle 19
meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 # handle 18
ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade # handle 17
}
}
..
Handles are used natively by nftables to make it easier to reference specific rules when you need to modify or transform them, so they’re quite handy for me right now—I don’t have to retype rules.
Handles #20 and #21 aren’t what I’m looking for to illustrate NAT in action. These handles specifically ignore packets traveling within the same guest network.
But handles #19, #18, and #17 are exactly where NAT is in action! These three handles are categorized based on the communication protocol in use—TCP (#19), UDP (#18), and all other protocols (#17).
All three handles process traffic in the same way. The key word in all these rules is masquerade.
The term masquerade hints what happens to the source address of the network traffic—it gets "masked".
Let’s follow the network packets from a VM that executed sudo apt update. First, since network communication of this type uses the TCP protocol, I have to look into the rule of handle #19:
ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 -->
--> This is like a programming `if` condition:
If the **s**ource **addr**ess (saddr) is within the 192.168.122.0/24 subnet
AND the **d**estination **addr**ess (daddr) is outside of this subnet:
masquerade to :1024-65535
Masquerade replaces the original source address with something else. In this case, it replaces it with the private IP address of the host. More specifically, it replace it with the private IP of the network interface with the highest priority (if there’s more than one, as in your case).
And what’s with the numbers after the colon? Those are ports—specifically the ephemeral port range (1024-65535). Why such a big range? And aren’t these ports busy on the host?
This port range is the ephemeral port range, which is designed to avoid conflicts with privileged ports (0–1023). Privileged ports are reserved for well-known system services, so this range ensures the masqueraded traffic doesn’t interfere with them.
Oh I left alone the packets from sudo apt update. Here they are: after masquerading:
- The packets originally departed from the VM at 192.168.122.10 on port 80 (since it’s HTTP traffic)
- They reached
virbr0(the virtual network switch). -
virbr0routed them to go outside the DEFAULT network (where VM belongs to). - Before leaving, they were masqueraded: each packet’s header was modified: the source address was replaced from 192.168.122.10:80 → to 192.168.1.5:12345. Here, 192.168.1.5 is the private IP address of the host (my PC), and 12345 is an ephemeral port assigned dynamically for this communication.
Here is the peculiar schema:
And that’s how NAT works. That’s all. Hahaha, just kidding.
That’s how the FIRST NAT worked. Now, these poor little packets have to "embark" on yet another "plane". They’re off to their next layover, where they’ll get NAT'd (masqueraded) AGAIN— new masks for everyone! This time, the layover is at the WiFi router.
The NAT’d packets leave the host with source address 192.168.1.5:12345. They arrive to my Wi-Fi router on the LAN side.
Wi-Fi Router NAT
The WiFi router sees packets from 192.168.1.5:1234 → to 123.123.7.132:80 (I am sooo bad, I do not know the IP address of Debian Servers & lazy to check).
NAT rewrites the source IP AGAIN from 192.168.1.5 to my router’s public IP (111.111.111.111) and change the port to another ephemeral one (54321).
NB! This is important! This is the cornerstone of weak routers (from the point of hardware):
Network packets don’t just travel to the Debian servers for a one-way trip; it’s always a round trip! At some point, response packets will come back. In the case of success, the VM expects to receive information about the versions of the packages it requested, to figure out what’s outdated.
Now, with all this masquerading, we end up in a situation much like those in movies—a classic masquerade ball drama (someone gets kissed because they were mistaken for someone else under their mask). To prevent these kinds of situationships, the WiFi router steps in as the responsible dude in charge. The router keeps track of who is going where, and if something comes back from "there," it makes sure it gets sent to the right who. To do this, router keeps updated a sort of table (thankfully it’s .xlsx). This table looks something like this:
|Private IP | Public IP | Source Port | Destination Port |
|192.168.1.5 | 123.193.7.162 | 54321 | 80 |
|192.168.1.20 | 111.153.6.132 | 35465 | 443 |
|192.168.1.11 | 125.161.7.162 | 34564 | 27017 |
|192.168.1.16 | 3.193.5.132 | 1224 | 22 |
.......................
#This table can be quite long if many devices are connected to the same WiFi and actively use Internet!
And then, the router keeps this information... in the table.
Here’s where an important point comes in: this table can grow very quickly. If your router is fast and powerful, no big deal—but if you cheaped out on it, you might run into trouble. A router that can’t handle more than a certain number of rows (based on its hardware limitations) will start to slow down significantly as the table grows beyond.
Again I left alone travelling network packets. Here they are:
The packets departed to the Debian server with my public IP and an ephemeral port: 111.111.111.111:54321.
The Debian server responds, and the response arrives from 123.123.7.132:80 to the router. The router then checks its table and says, "Ah, here it is!" It finds the matching entry in its tracking table and figures out which device the response is meant for. At this point, the router rewrites the headers of the packet AGAIN, removing its public IP address (111.111.111.111) and restoring the private IP and port from the table. Finally, it sends the packet back to my PC (the host) at 192.168.1.5:12345.
The host sees in the packets the destination 192.168.1.5:12345, checks its NAT table from the guest_nat chain, and recognizes this was originally from 192.168.122.10:80.
It rewrites AGAIN the destination from 192.168.1.5:12345 back to the VM’s IP and port: 192.168.122.10:80.
The packet is finally forwarded to the VM at 192.168.122.10:80.
The VM sees a response from http://deb.debian.org/debian/ to 192.168.122.10:80, completes the TCP handshake, and gets the repository data.
And now, that’s truly that's all! This is how NAT works. Enjoying IPv4? Still thinking IPv6 is difficult and scary?
For a little fun (and for little support for my emotional state after writing this article), try counting and writing in the comments how many times the network packets from the VM, headed to the internet, were rewritten along the way :).
Now after this TINY introduction to networking, it is time to get to the hands-on configurations. How can I make virtual machines accessible from other devices connected to the same LAN as the host—or even remotely? While I won’t be covering remote access here, making VMs accessible from other devices on the same home network can be quite convenient.
For example, I have a pretty powerful Desktop PC in terms of specs, and maybe sometimes I’m lazy and want to work from my laptop, which is like a toy in comparison to my PC. When I say I am lazy, I mean I don't want to sit properly at my desk; I want to loaf on the sofa with my laptop. However, I still want to use the resources of my PC. I can, of course, connect via SSH to my host machine, but it may be that I just want to connect to the VM with MongoDB to do something or check on it.
There are three main ways to do so:
- Port forwarding & Custom NAT
- Bridged networking (aka "shared physical device")
- PCI Passthrough of host network devices
All of them I will cover in the next article.
Top comments (0)