Custom Resource Definitions (CRDs) in Kubernetes: Extending the API for Custom Resources

Custom Resource Definitions (CRDs) in Kubernetes

In Kubernetes, Custom Resource Definitions (CRDs) allow users to extend the Kubernetes API by defining their own custom resources. A CRD enables you to create resources that are not part of the standard Kubernetes resources (like Pods, Services, and Deployments), allowing you to tailor Kubernetes to meet your specific application needs.

By defining a CRD, you can integrate your application's resources into Kubernetes' declarative management model, enabling better automation, monitoring, and scaling of your applications.

In this guide, we will explore what CRDs are, how to create and use them, and some best practices for working with CRDs in Kubernetes.

---

What Are Custom Resource Definitions (CRDs)?

A Custom Resource Definition (CRD) is a way to extend Kubernetes with new types of resources that behave like native Kubernetes resources. CRDs allow users to define custom objects that can be managed by Kubernetes in the same way as Pods, Services, Deployments, and other built-in resources.

Once you create a CRD, Kubernetes will treat it as a new API resource type. For example, you can create a resource type called Database to represent your custom database configurations.

A CRD defines:

• The API version and group to which the custom resource belongs.
• The resource name, typically plural (e.g., databases, webhooks).
• The resource schema, which specifies the properties that the resource will have.

Once the CRD is created, users can create, update, delete, and list custom resources just like any other Kubernetes object.

---

Key Concepts of CRDs

1. Custom Resource (CR): After defining a CRD, the custom resource is an instance of the custom object you created. For example, if you define a CRD named Database, you can then create a custom resource called MySQLDatabase with specific properties like version, replica count, and storage configuration.

2. CRD Object: The CRD itself is a Kubernetes resource of kind CustomResourceDefinition, and it describes the structure, validation, and behavior of the custom resource.

3. Kubernetes API: Once a CRD is created, the Kubernetes API server is extended to allow the management of custom resources using the kubectl command-line tool, as well as the Kubernetes API.

---

Creating a Custom Resource Definition (CRD)

Creating a CRD involves the following steps:

1. Define the Custom Resource Definition: You must create a YAML file that defines your custom resource's schema.

2. Apply the CRD to the Kubernetes Cluster: Once the CRD is defined, you apply it using kubectl apply to register the custom resource type.

3. Create Custom Resources: After the CRD is created, you can define instances of your custom resource.

Example: Creating a Simple CRD

Let's create a simple CRD for a resource called Database.

1. Define the CRD (in database-crd.yaml):

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.mycompany.com
spec:
  group: mycompany.com
  names:
    kind: Database
    plural: databases
    singular: database
    shortNames:
    - db
  scope: Namespaced
  versions:
  - name: v1
    served: true
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              engine:
                type: string
                description: The database engine (e.g., MySQL, PostgreSQL)
              replicas:
                type: integer
                description: The number of replicas for the database
              version:
                type: string
                description: The database version

1. Apply the CRD to Kubernetes:

kubectl apply -f database-crd.yaml

This will create the Database custom resource in the mycompany.com API group, and the API server will now be able to manage Database resources.

1. Create a Custom Resource (in my-database.yaml):

apiVersion: mycompany.com/v1
kind: Database
metadata:
  name: my-database
spec:
  engine: MySQL
  replicas: 3
  version: 8.0

1. Apply the Custom Resource:

kubectl apply -f my-database.yaml

This creates an instance of the Database resource. You can manage it with kubectl in the same way as other Kubernetes resources:

kubectl get databases

---

CRD Validation and Schemas

You can define complex validation schemas for your CRDs using OpenAPI v3 schemas. This allows you to ensure that custom resources are structured correctly when created or modified. You can validate fields such as:

• Required fields (e.g., engine, replicas)
• Type constraints (e.g., ensuring that replicas is an integer)
• Enum values (e.g., restricting engine to specific values like MySQL, PostgreSQL)

Here is an extended example of CRD validation:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.mycompany.com
spec:
  group: mycompany.com
  names:
    kind: Database
    plural: databases
    singular: database
  scope: Namespaced
  versions:
  - name: v1
    served: true
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              engine:
                type: string
                enum: ["MySQL", "PostgreSQL"]
                description: "The database engine"
              replicas:
                type: integer
                minimum: 1
                description: "Number of database replicas"
              version:
                type: string
                pattern: "^\\d+\\.\\d+\\.\\d+$"
                description: "Version in x.y.z format"

In this example:

• The engine field is constrained to either "MySQL" or "PostgreSQL".
• The replicas field is constrained to be a minimum of 1.
• The version field must follow the pattern x.y.z.

---

Using CRDs with Controllers

CRDs are often used in conjunction with Kubernetes Controllers to manage the lifecycle of custom resources. Controllers are responsible for ensuring that the state of custom resources matches the desired state.

For example, if a custom resource defines a Database resource, a controller might automatically deploy a StatefulSet, create persistent volumes, or manage backups for the database.

• Custom Controllers: Controllers watch the state of custom resources and trigger actions based on events (e.g., create, update, delete).
• Operator Pattern: The operator pattern is an example of using controllers to manage complex applications, like databases or message queues, based on the lifecycle of custom resources.

Best Practices for CRDs

1. Versioning: When updating your CRD, ensure that you handle versioning properly. Create new API versions when you change the schema to ensure backward compatibility.

2. Use Namespaces for Isolation: Like other Kubernetes resources, CRDs are often namespaced. Be mindful of organizing resources in different namespaces for better isolation.

3. Defining Defaults: Set sensible defaults for your custom resource fields where applicable, either in the CRD schema or in the application code, to make resources easier to use.

4. Avoid Complex and Large Schemas: While CRDs support complex schemas, it’s generally a good practice to keep the resource schema simple to avoid performance bottlenecks or operational complexity.

5. Use OpenAPI Validation: Take advantage of OpenAPI schema validation for type-checking and enforcing constraints on the custom resources.

6. Consider Operators: If the custom resource represents a complex application or service, consider writing an operator that watches for changes and manages the deployment, scaling, and updates of your application.

---

Conclusion

Custom Resource Definitions (CRDs) are a powerful feature of Kubernetes that allow you to extend its API and create custom resources that integrate seamlessly into Kubernetes' declarative management system. CRDs enable you to model and manage domain-specific resources while using the same Kubernetes tools and processes as native resources.

By understanding how to define and use CRDs, you can take full advantage of Kubernetes' extensibility to fit your specific use cases. Whether you're building a custom operator or just managing unique resources, CRDs make it easier to automate and scale your workloads within Kubernetes.

---