Leapcell

Posted on Feb 15

Understanding HTTP Caching: Strong and Negotiated Cache

#webdev #programming #networking #backend

Browser Caching Mechanism

We all know that when we open a webpage in a browser, the browser sends a request to the corresponding server based on the URL entered to fetch the required data resources. However, during this process, the page might take some time to load (causing a white screen) before it is rendered on the screen.

When trying to improve user experience, various caching techniques become essential, such as DNS caching, CDN caching, browser caching, and local page caching. A good caching strategy can reduce redundant resource requests, lower server overhead, and improve page load speed.

This article will focus on HTTP strong caching and negotiated caching.

Basic Principles

When a browser loads a resource, it first determines whether to apply the strong caching strategy by examining the Expires and Cache-Control request headers. This determines whether the browser should request the resource from the remote server or retrieve it from the local cache.

Strong Caching

In browsers, strong caching is divided into two categories: Expires (defined in the HTTP/1.0 specification) and Cache-Control (defined in HTTP/1.1).

Expires

Expires is a standard from HTTP/1.0, used as a response header field to indicate the expiration time of a resource. The value is an absolute timestamp, returned by the server.

When a browser first requests a resource, the server's response header includes the Expires field.
The next time the browser requests the same resource, it checks the Expires value from the previous response.
If the current request time is earlier than the expiration time specified by Expires, the browser directly uses the cached resource.

However, since Expires relies on the local system time, discrepancies between the client and server clocks can lead to cache inconsistencies.

Cache-Control

As mentioned earlier, Expires has a drawback: if the client’s local time differs from the server’s time, cache accuracy can be affected. To address this issue, HTTP/1.1 introduced the Cache-Control field, which takes precedence over Expires. Unlike Expires, Cache-Control defines cache expiration using a relative time instead of an absolute timestamp.

The most common Cache-Control directives include:

max-age: Specifies the number of seconds (e.g., 3600) during which the resource remains fresh (i.e., current time + 3600 seconds).
s-maxage: Similar to max-age, but specifically for caching on proxy servers.
private: The resource can only be cached by private caches (i.e., by the client but not shared caches like proxy servers).
public: The resource can be cached by both the client and proxy servers.
no-store: Prevents caching of any kind.
no-cache: The resource is stored in the local cache, but it must be revalidated with the origin server before being served to the client.

Negotiated Caching

Strong caching is determined by the browser alone. If strong caching is not hit, the browser will send a request to the server to check if negotiated caching applies. If the cache is valid, the server returns a 304 Not Modified status instead of new resource data.

Negotiated caching (also called conditional caching) is determined by the server and involves two sets of paired headers. When the browser makes its first request, the server includes either a Last-Modified or ETag header in the response. Subsequent requests include the corresponding request headers (If-Modified-Since or If-None-Match) to determine whether the resource has changed.

Last-Modified: Indicates the last modification time of the resource, returned by the server.
If-Modified-Since: Sent by the browser in a request, containing the previous Last-Modified value.
ETag: A unique identifier for a resource. When the resource changes, the ETag value changes accordingly. Unlike Last-Modified, which might change even if the file content remains the same, ETag ensures more accurate cache validation.
If-None-Match: Sent by the browser in a request, containing the previously returned ETag value.

Strong Caching and Request Flow

When a browser requests a resource, it first checks if there is a local cache record. If no record exists, it sends a request to the server and stores the returned Last-Modified value.
If a cache record exists, the browser first checks if strong caching is still valid (Cache-Control takes precedence over Expires). If the cache is still valid, the browser serves the cached resource (HTTP status 200).
If strong caching has expired, the browser initiates a request using the negotiated caching strategy. The server first checks the ETag value:
- If the client’s ETag matches the server’s, the server returns 304 Not Modified (without sending the resource data).
If the server does not use ETag, it checks If-Modified-Since. If the provided value matches the server's last modification time, the server returns 304 Not Modified.
If neither ETag nor If-Modified-Since matches, the server sends a fresh resource and updates the cache.

Why Is ETag Needed?

ETag was introduced to address issues with Last-Modified, including:

Timestamp inaccuracies: A file’s last modified timestamp may change even when the content remains the same, leading to unnecessary cache invalidation.
High-frequency modifications: Last-Modified only supports second-level granularity, which may not be sufficient for frequently updated files. ETag ensures better precision.
Some servers cannot accurately determine a file’s last modification time, making ETag a better alternative.

Differences in HTTP Status Codes

200: Request successful, server returns a fresh copy of the resource.
200 (from memory cache / from disk cache): Strong caching is still valid, so the browser loads the resource from local cache.
304 Not Modified: The request used negotiated caching, and the server determined that the cached resource is still valid.

Note:

from memory cache: The resource was retrieved from memory (e.g., after a soft refresh).

from disk cache: The resource was retrieved from disk storage (e.g., after reopening the browser tab).

Cache Priority Rules

If both Expires and Cache-Control exist, Cache-Control takes precedence.
- Cache-Control > Expires
If both strong caching and negotiated caching exist, the browser first checks strong caching. If the cache is still valid, it is used. Otherwise, negotiated caching is applied.
- Strong Caching > Negotiated Caching
If both ETag and Last-Modified exist, ETag takes precedence.
- ETag > Last-Modified

Additional Notes

In the HTTP/1.0 specification, there was an older caching directive called Pragma, which had the same effect as Cache-Control: no-cache, forcing the browser to revalidate the resource with the origin server.

Caching priority order:

Pragma -> Cache-Control -> Expires -> ETag -> Last-Modified

Heuristic Caching

If a response does not include Expires or Cache-Control headers but does include Last-Modified, browsers apply a heuristic caching strategy.

The formula used by the browser is:

(currentTime - lastModified) * 0.1

This caching strategy is only applied when the server does not explicitly define a cache policy.

For further reading: HTTP Heuristic Caching (Missing Cache-Control and Expires Headers) Explained

Other Considerations

Negotiated caching is effective only when combined with strong caching. Without strong caching, negotiated caching has no meaning.
Most web servers enable negotiated caching by default, typically using both Last-Modified and ETag simultaneously.

Important Scenarios

In distributed systems, ensure that Last-Modified timestamps are consistent across servers. Otherwise, load balancing may cause inconsistent cache validation, leading to unnecessary resource re-fetching.
In distributed systems, it is recommended to disable ETag, since different servers might generate different ETag values, causing unnecessary cache misses.

We are Leapcell, your top choice for hosting backend projects.

Leapcell is the Next-Gen Serverless Platform for Web Hosting, Async Tasks, and Redis:

Multi-Language Support

Develop with Node.js, Python, Go, or Rust.

Deploy unlimited projects for free

pay only for usage — no requests, no charges.

Unbeatable Cost Efficiency

Pay-as-you-go with no idle charges.
Example: $25 supports 6.94M requests at a 60ms average response time.

Streamlined Developer Experience

Intuitive UI for effortless setup.
Fully automated CI/CD pipelines and GitOps integration.
Real-time metrics and logging for actionable insights.

Effortless Scalability and High Performance

Auto-scaling to handle high concurrency with ease.
Zero operational overhead — just focus on building.

Explore more in the Documentation!

Read on our blog

Forem