K8s Ingress vs. OpenShift Route

This article aims to clear some confusion between Ingress and Route, and hopefully enable you to take advantage of the power of both.

Recently starting to deal with OpenShift clusters, I found most of the resource types in OCP are the native K8s resources I’m familiar with. Most of them, except for Route.

Ingress

Ingress is the native API object K8s defined to enable external access into services in the cluster.

It is THE resource type people usually expect to see in AKS, GKE, EKS, or in-house K8s installations when they connect their service to the vast internet.

A bare Ingress resource YAML manifest is like an interface, where the actual behavior heavily depends on how it is provisioned in your cluster - more specifically, the Ingress Conroller backing it.

For example, if your cluster uses ALB to back your Ingress (which is the most-likely case for AWS EKS), you will need some alb-specific annotations like this as part of the official example:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: IP

Normally when you try to use Ingress you would want to:

1. Find the Ingress Controller installed in your cluster.
2. If none-installed, look up one that's supported by your cloud provider (including private/hybrid) and install it.
3. Add the annotation based on the manual of that controller.

Route

Route, on the other hand, is not a native K8s thing, it only exists in the OpenShift world.

It depends on another Openshift concept - Router. Consider Router as plugins deployed on your cluster nodes that enables configurable routings. By routings, I mean traffic from external clients to in-cluster application services.

Note that a Router is installed on a node, so to reach the Router you need to reach the node first.

For example, if an HAProxy Router is used, external traffic is expected to go to the particular node hosting the HAProxy, then being routed to the service/pod.

According to the dev guide, there are 3 ways to create a Route.

1. The Openshift web console UI
2. oc CLI, such as oc expose svc/frontend --hostname=www.example.com
3. YAML manifest, such as

apiVersion: v1
kind: Route
metadata:
  name: frontend
spec:
  host: www.example.com
  path: "/test" 
  to:
    kind: Service
    name: frontend

Like Ingress, Route also supports annotations, for example router.openshift.io/cookie_name="my_cookie". But note that the annotations have different prefixes than those of Ingress resources, thus their annotations are not interchangeable.

However, the main method to control the behavior of a Route, rather than through annotations, is by changing the Router Environment Variables.

Ingress to Route

Starting from Openshift 3.10, Ingress objects created in Openshift can be translated into Route objects.

If you know the difference between Ingress and routes well, this is very convenient.

On the other hand, however, it can easily mislead people trying to take advantage of this translation.

As described in the section above, Routes:

1. Doesn't respect Ingress annotations. As a result, your fine-tuned annotations on your Ingress manifest file would NOT affect the true behavior of the created Route.
2. Needs traffic to first reach the node. So you will need to configure DNS records to Router nodes separately.

According to the dev guide:

DNS resolution for a host name is handled separately from routing; your administrator may have configured a cloud domain that will always correctly resolve to the OpenShift Container Platform router, or if using an unrelated host name you may need to modify its DNS records independently to resolve to the router.

This has to be taken into consideration and worked upon with the cluster admin for anyone familiar with K8s Ingress but not Openshift Route.