Cross-posted from my personal blog https://blog.nicolasmesa.co.
Disclaimer: Hello! This is my first blog post! I’m super excited to share it with the world but should also warn you that it’s a bit long and repetitive. Hopefully, I’ll become a better writer as I blog more. I hope you enjoy it :)
Introduction
I’ve heard this question many times during my career and want to try to answer it. I’m going to dive deep into the networking part of the question since this is the part that I find the most interesting.
A lot of stuff happens under the hood in the network to get an HTTP request from the browser to the server and then an HTTP response from the server back to the browser. To be able to explain what happens, I’m going to make some assumptions about the network setup and the status of the network.
Assumptions
Network status
- Our computer is connected to the network via Ethernet.
- Our computer already has an IP address (more on what the IP address is later).
- Our computer sits behind a NAT (this is the case in many home networks).
- Our ARP cache is clear.
- Our DNS cache is clear.
- Our router’s ARP cache contains an entry for its default gateway.
- Our network is
192.168.0.0/24. - The URL we’re going to use is
http://www.example.com. - The DNS server’s cache has an entry for
www.example.com. - We’re going to ignore TCP sequence numbers.
Configuration
Our computer (computer making the HTTP request)
- IP Address:
192.168.0.10. - Default gateway:
192.168.0.1. - DNS Server:
8.8.8.8. - MAC Address:
AA:AA:AA:AA:AA:AA. - TCP source port:
9999. - DNS source port:
3333.
Our router / NAT
- Internal IP Address:
192.168.0.1. - External IP address:
51.0.0.20. - External TCP source port:
15000. - External DNS source port:
10000. - External net mask:
255.255.255.0(or/24). - Internal MAC address:
BB:BB:BB:BB:BB:BB. - External MAC address:
CC:CC:CC:CC:CC:CC. - Default gateway:
51.0.0.1. - Default gateway’s MAC address:
DD:DD:DD:DD:DD:DD.
www.example.com server
- The IP address of
www.example.comis93.184.216.34.
Step by step
To get this process started we type the URL http://www.example.com in the web browser and press enter. This will instruct our browser to create an HTTP request to send to www.example.com. The HTTP request will look something like this:
GET / HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
HTTP request to www.example.com.
Our computer will now try to connect to www.example.com in TCP port 80 (the default HTTP port). First, it needs to create a TCP SYN segment to initiate the TCP connection. The SYN segment contains information to establish the connection including a random source port (9999 from our assumptions), destination port (80) and a flag (SYN).
This TCP segment is then encapsulated in an IP packet that contains the source IP (192.168.0.10) and the destination IP. For the destination IP, our computer goes to its DNS cache and looks for an entry containing www.example.com. Since its cache is empty, the creation of the IP packet needs to wait until we can get the IP address of www.example.com. So far our incomplete IP packet looks something like this:
IP[
SourceIp=192.168.0.10,
DestIp=????,
TCP[
SourcePort=9999,
DestPort=80,
Flags=[SYN]
]
]
Incomplete SYN IP packet (note the ???? in the DestIP field).
To get the IP address of www.example.com, we need to use a service called DNS. DNS stands for Domain Name System and is the system responsible for translating human-readable names (such as www.example.com) into IP addresses. Our computer creates a DNS request that includes the type of request (A) and the domain name to resolve (www.example.com). This DNS request is then encapsulated in a UDP datagram whose header contains a random source port (3333 from our assumptions) and the destination port (53 is DNS’s well-known port). This datagram is then placed inside an IP packet whose header has the source IP address (192.168.0.10) and the destination IP address (8.8.8.8). Note that the DNS server was configured beforehand in our computer so we already know what its IP address is.
We now want to send this IP packet but our computer doesn’t know where to send it. The computer looks at its route table to see which route matches the 8.8.8.8 IP address. The route table looks something like this:
Destination Netmask Gateway Interface
0.0.0.0 0.0.0.0 192.168.0.1 eth0
192.168.0.0 255.255.255.0 0.0.0.0 eth0
Route table of our computer.
Our computer uses the longest prefix match to select who to send the packet to. In this case, it selects the first route which is our default gateway (192.168.0.1). This means that the IP packet will be sent to our router and the router will take care of forwarding it to the next hop (which is hopefully closer to the DNS server). The routers in the path will keep forwarding the packet based on their routing tables until it reaches the DNS server (8.8.8.8). This process happens every time our computer needs to send an IP packet and, in our case, it will always choose to send the packet to our default gateway (192.168.0.1).
Now that the IP packet is ready, it is wrapped in an Ethernet frame. The Ethernet header contains the source MAC address (AA:AA:AA:AA:AA:AA) and the destination MAC address (this would be the router’s MAC address in this case). To get the router’s MAC address, our computer looks in its ARP table to see if it has an entry to convert the router’s IP address (192.168.0.1) to the router’s MAC address. Since the ARP cache is empty, we need to find out what is the MAC address before we can send the IP packet. This is how our incomplete Ethernet frame looks like:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=????,
IP[
SourceIp=192.168.0.10,
DestIp=8.8.8.8,
UDP[
SourcePort=3333,
DestPort=53,
DNS[
Query=A,
Domain=www.example.com
]
]
]
]
Incomplete Ethernet frame for the DNS query (note the ???? in the DestMac field).
To convert IP addresses to MAC addresses, the ARP protocol is used. ARP stands for Address Resolution Protocol. The ARP packet contains a question asking who-has 192.168.0.1 tell 192.168.0.10 along with the source MAC address (AA:AA:AA:AA:AA:AA), the destination MAC address (FF:FF:FF:FF:FF:FF which is the broadcast address), the source IP address (192.168.0.10), and the destination IP (192.168.0.1 which is the IP we’re asking about). The ARP packet is then wrapped in an Ethernet frame containing the source MAC address (AA:AA:AA:AA:AA:AA) and for the destination MAC address, the broadcast address is used (FF:FF:FF:FF:FF:FF). The frame is sent over our private network and looks something like this:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=FF:FF:FF:FF:FF:FF,
ARP[
Type=Request,
SourceIp=192.168.0.10,
DestIp=192.168.0.1,
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=FF:FF:FF:FF:FF:FF
]
]
ARP packet to find out our router’s MAC address.
There is something worth noting at this point. This is the first interaction that our computer has had with the network!
Since the broadcast MAC address (FF:FF:FF:FF:FF:FF) was used, this frame will reach every node on our private network. Every node (except for our router) will end up discarding the packet since the destination IP address (192.168.0.1) is not their IP.
When our router receives this packet, it will see that 192.168.0.10 is asking for its MAC address. It can also see in the ARP packet the source MAC address of our computer. The router saves this information in its ARP table (192.168.0.10 -> AA:AA:AA:AA:AA:AA) and creates an ARP reply saying 192.168.0.1 is-at BB:BB:BB:BB:BB:BB. This ARP reply is then wrapped in an Ethernet frame with the source MAC address (BB:BB:BB:BB:BB:BB) and destination MAC address (AA:AA:AA:AA:AA:AA). The Ethernet frame is now sent over the network. One thing to note here is that the destination MAC address is no longer the broadcast MAC address so only our computer will process it. From now on, our router will be able to convert our computer’s IP address (192.168.0.10) to its MAC address (AA:AA:AA:AA:AA:AA) by referring to its ARP cache. The Ethernet frame containing this reply looks something like this:
Ethernet[
SourceMac=BB:BB:BB:BB:BB:BB,
DestMac=AA:AA:AA:AA:AA:AA,
ARP[
Type=Reply,
SourceIp=192.168.0.1,
DestIp=192.168.0.10,
SourceMac=BB:BB:BB:BB:BB:BB,
DestMac=AA:AA:AA:AA:AA:AA
]
]
ARP reply where our router tells its MAC address to our computer.
Our computer receives the Ethernet frame, sees the ARP reply and saves the MAC address of our router in its ARP table (192.168.0.1 -> BB:BB:BB:BB:BB:BB). From now on, any time our computer needs to convert our router’s IP address (192.168.0.1) to its MAC address (BB:BB:BB:BB:BB:BB), it simply needs to refer to its ARP cache. We’re making progress! We now have the MAC address of our router, which we need to send the IP packet with the DNS query to get the IP of www.example.com, which we need to send a TCP SYN packet, to establish a TCP connection over port 80, which we need to send the HTTP request.
Since our computer now knows the MAC address of the router, it fills in the blanks of the Ethernet frame containing our DNS query and sends it over the network to (BB:BB:BB:BB:BB:BB). This happens with every Ethernet frame that needs to be sent to our router. The frame looks something like this:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=BB:BB:BB:BB:BB:BB,
IP[
SourceIp=192.168.0.10,
DestIp=8.8.8.8,
UDP[
SourcePort=3333,
DestPort=53,
DNS[
Query=A,
Domain=www.example.com
]
]
]
]
Complete Ethernet frame for the DNS query.
One thing to note is that the DestIp field is not the router’s IP address; it’s still 8.8.8.8.
The router receives this frame and looks at the destination IP (8.8.8.8) in the IP header. It also uses the longest prefix match to select where to forward this packet and ends up selecting its default Gateway (51.0.0.1). This decision is made every time the router needs to forward a packet to an IP address in the public Internet such as 8.8.8.8 (and later 93.184.216.34).
Based on our assumptions our router is also a NAT. This means that our router needs to translate the packet before it can be forwarded. NAT stands for Network Address Translation. It is widely deployed to have multiple devices share a single public IP address. Our internal (or private) network uses private IP addresses (192.168.0.1 and 192.168.0.10) but the public Internet uses public IP addresses. When we need to send a packet to the public Internet, the NAT takes care of replacing the private IP address with the public IP address (51.0.0.20). The NAT uses 5 values to identify a connection: protocol (UDP / TCP), source IP, internal source port, destination IP and external source port. The external source port is chosen randomly to make sure the connection identifier is unique. Based on our assumptions, the router decides to use port 10000. The entry in the NAT table looks something like this:
UDP 192.168.0.10 3333 -> 8.8.8.8 10000
NAT table. Note that the destination port (53) is not used to identify the connection.
Our router uses the information in the NAT table to replace the source IP address of the packet (192.168.0.10) with the public IP address (51.0.0.20), and the internal source port (3333) with the external source port (10000). This makes that IP packet routable in the public Internet. Our router performs this translation for any packet that needs to go out to the public Internet including TCP/IP packets. We’ll call this process internal-to-external translation.
Our router wraps the packet in a new Ethernet frame with the source MAC address of CC:CC:CC:CC:CC:CC, and a destination MAC address of DD:DD:DD:DD:DD:DD (we assumed this value was already cached so there’s no need for ARP). Every packet that our router forwards to the public Internet goes through the same process so we won’t bring this up again. The Ethernet frame ends up looking something like this:
Ethernet[
SourceMac=CC:CC:CC:CC:CC:CC,
DestMac=DD:DD:DD:DD:DD:DD,
IP[
SourceIp=51.0.0.20,
DestIp=8.8.8.8,
UDP[
SourcePort=10000,
DestPort=53,
DNS[
Query=A,
Domain=www.example.com
]
]
]
]
Publicly routable packet with DNS query (note the change in SourceIp and SourcePort).
The Ethernet frame is sent to the default gateway which is most likely a router owned by our ISP and it is up to the ISP to get our packet to 8.8.8.8.
Once our packet reaches 8.8.8.8 (Google’s DNS server), the server takes a look at the query (A for www.example.com) and searches its cache. We’re assuming that the cache entry exists, so it knows that www.example.com maps to 93.184.216.34. The DNS server creates a new DNS response message containing this information as the answer to the query. The response is wrapped in a UDP datagram with source port 53 and destination port 10000. The UDP datagram is then wrapped in an IP packet with source IP 8.8.8.8 and destination IP 51.0.0.20 (our public IP). The packet is finally wrapped in an Ethernet frame (we’ll ignore this part) and sent through the network. The IP packet ends up looking like this:
IP[
SourceIp=8.8.8.8,
DestIp=51.0.0.20,
UDP[
SourcePort=53,
DestPort=10000,
DNS[
Answer=93.184.216.34,
Domain=www.example.com
]
]
]
IP packet containing the DNS reply.
The packet traverses the network and eventually reaches our router. Our router needs to translate this packet before it forwards it into our private network. The router looks at the protocol (UDP), source IP (8.8.8.8) and the destination port (10000) to find an entry in the NAT table with those values. With the values from the table, the router translates the destination IP to be our computer’s IP (192.168.0.10) and the destination port to be our computer’s source port (3333). This makes that IP packet routable in our private network. The same process occurs for any packet coming from the public Internet including TCP/IP packets (only packets whose properties can be found in the NAT table are translated, the others are dropped). We’ll call this process external-to-internal translation. After this, the router wraps the packet in an Ethernet frame with source MAC BB:BB:BB:BB:BB:BB and destination MAC AA:AA:AA:AA:AA:AA and sends it over the private network (this happens every time our router forwards a packet to our computer so we won’t bring it up again). The Ethernet frame with the translated IP packet looks something like this:
Ethernet[
SourceMac=BB:BB:BB:BB:BB:BB,
DestMac=AA:AA:AA:AA:AA:AA,
IP[
SourceIp=8.8.8.8,
DestIp=192.168.0.10,
UDP[
SourcePort=53,
DestPort=3333,
DNS[
Answer=93.184.216.34,
Domain=www.example.com
]
]
]
]
Privately routable packet containing the DNS reply (note the change in DestIp and DestPort).
With the IP address of www.example.com, our computer fills in the destination IP in our incomplete SYN packet. The IP header is wrapped in an Ethernet frame and sent through the private network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=BB:BB:BB:BB:BB:BB,
IP[
SourceIp=192.168.0.10,
DestIp=93.184.216.34,
TCP[
SourcePort=9999,
DestPort=80,
Flags=[SYN]
]
]
]
Privately routable packet containing the TCP SYN packet.
Our router receives the packet and extracts the protocol (TCP), the source IP (192.168.0.10), the source port (9999), the destination IP (93.184.216.34) and generates a random number to be used as the external source port (15000 from the assumptions). These values identify the TCP connection and are saved in the router’s NAT table. Our router can reference the NAT table to identify packets that need to be translated and what values to use for the translation. The entry in the NAT table looks something like this:
TCP 192.168.0.10 9999 -> 93.184.216.34 15000
Our router uses our public IP address (51.0.0.20) to replace the source IP and the port generated in the previous step (15000) to replace the source port. It then wraps the translated IP packet in an Ethernet frame and sends it over the network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=CC:CC:CC:CC:CC:CC,
DestMac=DD:DD:DD:DD:DD:DD,
IP[
SourceIp=51.0.0.20,
DestIp=93.184.216.34,
TCP[
SourcePort=15000,
DestPort=80,
Flags=[SYN]
]
]
]
Publicly routable SYN segment (note the changes to SourceIp and SourcePort).
When the packet reaches 93.184.216.34 (the IP address of www.example.com), the server looks at the destination port of the TCP header (80) and checks if an application is listening on that port. Since there is an HTTP server running on that port, the server needs to reply with a SYN/ACK. The server creates a TCP segment and sets its source port to 80, the destination port to 15000 (the source port of the SYN packet that was just received), and SYN/ACK for the flags. It then wraps the segment in an IP packet with source IP of 93.184.216.34 and destination IP of 51.0.0.20. Finally, it wraps the IP packet in an Ethernet frame (we’re going to ignore this part). The packet is then sent out to the network and looks something like this:
IP[
SourceIp=93.184.216.34,
DestIp=51.0.0.20,
TCP[
SourcePort=80,
DestPort=15000,
Flags=[SYN, ACK]
]
]
IP packet containing the TCP SYN / ACK.
When the packet reaches our router, it performs an external-to-internal translation. The translated packet is wrapped in an Ethernet frame and sent over the private network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=BB:BB:BB:BB:BB:BB,
DestMac=AA:AA:AA:AA:AA:AA,
IP[
SourceIp=93.184.216.34,
DestIp=192.168.0.10,
TCP[
SourcePort=80,
DestPort=9999,
Flags=[SYN, ACK]
]
]
]
Privately routable packet containing the TCP SYN / ACK.
Our computer looks at the destination port (9999) and the TCP flags (SYN/ACK) and checks the state of this connection. The only thing missing to establish this connection is to send an ACK back to the server (93.184.216.34). Our computer creates a new TCP segment with source port 9999, destination port 80 and ACK as its only flag. This segment is then wrapped in an IP packet with source IP 192.168.0.10 and destination IP 93.184.216.34. The IP packet is wrapped in an Ethernet frame and sent over the private network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=BB:BB:BB:BB:BB:BB,
IP[
SourceIp=192.168.0.10,
DestIp=93.184.216.34,
TCP[
SourcePort=9999,
DestPort=80,
Flags=[ACK]
]
]
]
Privately routable packet containing the ACK to complete the three-way handshake.
Our router receives this frame and uses the information in the NAT table to perform an internal-to-external translation. Then, it wraps the translated IP packet in an Ethernet frame and sends it over the public network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=CC:CC:CC:CC:CC:CC,
DestMac=DD:DD:DD:DD:DD:DD,
IP[
SourceIp=50.0.0.20,
DestIp=93.184.216.34,
TCP[
SourcePort=15000,
DestPort=80,
Flags=[ACK]
]
]
]
Publicly routable packet containing the ACK to complete the three-way handshake.
The packet traverses the Internet until it gets to 93.184.216.34. The server uses the source port (15000) and the source IP (50.0.0.20) to check the connection state and marks the connection as ESTABLISHED. Note that the server doesn’t return an ACK for the ACK it just received.
Our computer is now ready to send the actual HTTP request. It wraps the HTTP request in a TCP segment with source port 9999, destination port 80 and sets the flags to ACK/PUSH. This TCP segment is then wrapped in an IP packet with source IP 192.168.0.10 and destination IP 93.184.216.34. Then, it wraps the IP packet in an Ethernet frame and sends it through the private network. The frame looks something like this:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=BB:BB:BB:BB:BB:BB,
IP[
SourceIp=192.168.0.10,
DestIp=93.184.216.34,
TCP[
SourcePort=9999,
DestPort=80,
Flags=[ACK, PUSH],
HTTP[
GET / HTTP/1.1
Host: www.example.com
...
]
]
]
]
Privately routable packet containing the HTTP request.
Our router receives the frame and does an internal-to-external translation to the IP packet. It then wraps the translated packet in an Ethernet frame and sends it over the public network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=CC:CC:CC:CC:CC:CC,
DestMac=DD:DD:DD:DD:DD:DD,
IP[
SourceIp=50.0.0.20,
DestIp=93.184.216.34,
TCP[
SourcePort=15000,
DestPort=80,
Flags=[ACK, PUSH],
HTTP[
GET / HTTP/1.1
Host: www.example.com
...
]
]
]
]
Publicly routable packet containing HTTP request.
The packet reaches 93.184.216.3 (www.example.com) and the server uses the source IP (51.0.0.20) and source port (15000) to verify that the connection exists and is ESTABLISHED. The PUSH flag in the TCP segment instructs the server to send the body of the TCP segment (the HTTP request) to the application listening on port 80 (the HTTP server). The HTTP server processes the HTTP request and generates an HTTP response. An HTTP response from www.example.com looks something like this:
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Type: text/html
Date: Sat, 11 Aug 2018 20:45:46 GMT
Etag: "1541025663"
Expires: Sat, 18 Aug 2018 20:45:46 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Server: ECS (oxr/830D)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 606
<!doctype html>
<html>
...
</html>
Sample HTTP response from www.example.com (snipped almost all of the body).
Note: The server should have sent and ACK to acknowledge that this TCP segment was received. However, this ACK can also be sent along with the HTTP response, so in the interest of not being even more repetitive, we’ll assume this is the case.
The HTTP response is wrapped in a TCP segment with source port 80, destination port 15000 and flags ACK/PUSH. Then, the TCP segment is wrapped in an IP packet with source IP 93.184.216.34 and destination IP 50.0.0.20. The packet is then wrapped in an Ethernet frame (that we are also going to ignore) and sent through the public network. The IP packet looks something like:
IP[
SourceIp=93.184.216.34,
DestIp=50.0.0.20,
TCP[
SourcePort=80,
DestPort=15000,
Flags=[ACK, PUSH],
HTTP[
HTTP/1.1 200 OK
Accept-Ranges: bytes
...
<!doctype html>
...
]
]
]
IP packet containing the HTTP response.
Our router receives the packet and performs an external-to-internal translation. After that, the IP packet is wrapped with an Ethernet frame and sent over the private network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=BB:BB:BB:BB:BB:BB,
DestMac=AA:AA:AA:AA:AA:AA,
IP[
SourceIp=93.184.216.34,
DestIp=192.168.0.10,
TCP[
SourcePort=80,
DestPort=9999,
Flags=[ACK, PUSH],
HTTP[
HTTP/1.1 200 OK
Accept-Ranges: bytes
...
<!doctype html>
...
]
]
]
]
Privately routable packet with the HTTP response.
Our computer receives the packet and looks up the connection information. The PUSH flag instructs our computer to send the TCP body (the HTTP response) to the application that opened the connection (our web browser). Our web browser now has the HTTP response and can use it to render the web page. From the networking side, however, we’re not done. The server doesn’t know that we received the TCP segment since we haven’t acknowledged it (sent an ACK back). Since the browser is not going to send more requests to www.example.com, it will also close the connection. These two actions can be combined into one by sending the ACK and the FIN flags in the same TCP segment (we’re going to assume this happened to avoid even more repetition).
Our computer creates a new TCP segment with source port 9999, destination port 80 and FIN/ACK as its flags. It places this segment in an IP packet with source IP 192.168.0.10 and destination IP 93.184.216.34. The IP packet is wrapped with an Ethernet frame and is sent over the network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=BB:BB:BB:BB:BB:BB,
IP[
SourceIp=192.168.0.10,
DestIp=93.184.216.34,
TCP[
SourcePort=9999,
DestPort=80,
Flags=[ACK, FIN]
]
]
]
Privately routable packet containing the ACK for the HTTP response and the FIN to close the TCP connection.
The router gets this frame and performs an internal-to-external translation. Then, it wraps the IP packet in an Ethernet frame and sends the frame to the public network. The frame looks something like this:
Ethernet[
SourceMac=CC:CC:CC:CC:CC:CC,
DestMac=DD:DD:DD:DD:DD:DD,
IP[
SourceIp=50.0.0.20,
DestIp=93.184.216.34,
TCP[
SourcePort=15000,
DestPort=80,
Flags=[ACK, FIN]
]
]
]
Publicly routable packet with the ACK for the HTTP response and the FIN to close the TCP connection.
When the server receives this packet, it now knows that the HTTP response was received. From the flags, it can also tell that our computer wants to close the connection. The server creates a new TCP segment with source port 80, destination port 15000 and flags FIN/ACK. It wraps this TCP segment in an IP packet with source IP 93.184.216.34 and destination IP 50.0.0.20. The IP packet is sent over the network. The IP packet looks something like this:
IP[
SourceIp=93.184.216.34,
DestIp=50.0.0.20,
TCP[
SourcePort=80,
DestPort=15000,
Flags=[ACK, FIN]
]
]
IP packet containing the FIN to close the connection.
When our router receives the packet, it performs an external-to-internal translation and then wraps the packet in an Ethernet frame. It then sends the frame through our private network. The frame looks something like this:
Ethernet[
SourceMac=BB:BB:BB:BB:BB:BB,
DestMac=AA:AA:AA:AA:AA:AA,
IP[
SourceIp=93.184.216.34,
DestIp=192.168.0.10,
TCP[
SourcePort=80,
DestPort=9999,
Flags=[ACK, FIN]
]
]
]
Privately routable packet containing the FIN to close the connection.
Our computer receives this packet and finds a FIN in the flags. The connection is now closed, all that remains is for our computer to send an acknowledgement to the server that it received the FIN/ACK packet. It creates a TCP segment with source port 9999 and destination port 80. The segment is wrapped in an IP packet with source IP 192.168.0.10 and destination IP 93.184.216.34. The IP packet is wrapped in an Ethernet frame and sent over the private network. The frame looks something like this:
Ethernet[
SourceMac=AA:AA:AA:AA:AA:AA,
DestMac=BB:BB:BB:BB:BB:BB,
IP[
SourceIp=192.168.0.10,
DestIp=93.184.216.34,
TCP[
SourcePort=9999,
DestPort=80,
Flags=[ACK]
]
]
]
Privately routable packet containing last ACK to acknowledge that the connection has been closed.
The router receives this frame and performs an internal-to-external translation. The IP packet is wrapped in an Ethernet frame sent over the public network. The Ethernet frame looks something like this:
Ethernet[
SourceMac=CC:CC:CC:CC:CC:CC,
DestMac=DD:DD:DD:DD:DD:DD,
IP[
SourceIp=50.0.0.20,
DestIp=93.184.216.34,
TCP[
SourcePort=15000,
DestPort=80,
Flags=[ACK]
]
]
]
Publicly routable packet with translated packet with last ACK.
When the server gets this packet, it looks at the flags and confirms that our computer got the FIN TCP segment.
Conclusion
Wow, that was a long post! I think we can all agree that a lot happens at the network layer from the time you type a URL in the browser and press enter until you get to see your web page. There are a few important things to call out though:
- This was just one HTTP request! Usually, a page that has external scripts, images, stylesheets, etc, makes tens or hundreds of requests!
- With only one request, we were able to see the value of caching. We only had to ask for the MAC address of the router once and then we were able to fetch that MAC address from the cache. If we had made more requests, DNS caching would have also played a role since we would have cached the IP address of
www.example.comand wouldn’t have had to ask our DNS server for it. - Caching plays a big role in HTTP as well. If our browser had cached this page before, we wouldn’t have had to make any network calls!
- There is some overhead while establishing the TCP connection. HTTP clients try to keep the connection alive (with the
Connection: Keep-AliveHTTP header) to avoid this overhead and reuse the same connection. A problem with this header is that HTTP clients can only make one request a time (per TCP connection). HTTP/2 and QUIC solve this problem by multiplexing the connection.
Thanks for reading! I hope you enjoyed it and learned something!
Top comments (1)
That is extremely demonstrative thank you!