In this post, we will be going over the fastest no-frills approach to getting your operator off the ground using kubebuilder. The post assumes knowledge of the following:

  • Kubernetes and how it works
  • Kubernetes custom resource definitions
  • Kubernetes Operators and reconciliation loops
  • Setting up a local cluster, I use kind for my k8s orchestration needs
  • Golang

The task is to create an operator that operates on a Kubernetes CRD TodoList. It listens on the pods available in the system. If there are any pods with the same name as the TodoList, it marks the status as True.

This operator only operates on the operator-namespace namespace.

The first step is to install kubebuilder using the following command:

curl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH) && chmod +x kubebuilder && mv kubebuilder /usr/local/bin/

Check if it’s installed properly by running kubebuilder version .

Install the kubernetes cluster using kind

kind create cluster –name operators

Setup the initial project, APIs, groups and kinds

kubebuilder init –domain sarmag.co –repo sarmag.co/todo
kubebuilder create api –group todo –version v1 –kind TodoList

This will create the required scaffolded project, group, API and kind.

Once that’s done, there are 2 main files that we have to update

  • api/v1/todolist_types.go
  • internal/controller/todolist_controller.go

In the todolist_types.go file, update the required specification and status of the CRD.

type TodoListSpec struct {  
 Task string \`json:"task,omitempty"\`  
}  
  
type TodoListStatus struct {  
 IsCompleted bool \`json:"status,omitempty"\`  
}

You can refer to the file here https://github.com/gauravsarma1992/todolist-k8s-operator/blob/main/api/v1/todolist_types.go#L24-L30.

In the todolist_controller.go file, update the reconcilation logic with the following:

func (r \*TodoListReconciler) Reconcile(ctx context.Context, req ctrl.Request) (result ctrl.Result, err error) {  
 var (  
  todoList todov1.TodoList  
  podList  corev1.PodList  
  logger   logr.Logger  
  
  isCompleted bool  
 )  
  
 logger = log.FromContext(ctx)  
 logger.Info("Reconciling TodoList")  
  
 if err = r.Get(ctx, req.NamespacedName, &todoList); err != nil {  
  logger.Error(err, "Error in fetching Todolist")  
  err = client.IgnoreNotFound((err))  
  return  
 }  
  
 if err = r.List(ctx, &podList); err != nil {  
  logger.Error(err, "Error in fetching pods list")  
  return  
 }  
  
 for \_, item := range podList.Items {  
  if item.GetName() != todoList.Spec.Task {  
   continue  
  }  
  logger.Info("Pod just became available with", "name", item.GetName())  
  isCompleted = true  
 }  
  
 todoList.Status.IsCompleted = isCompleted  
 if err = r.Status().Update(ctx, &todoList); err != nil {  
  logger.Error(err, "Error in updating TodoList", "status", isCompleted)  
  return  
 }  
  
 if todoList.Status.IsCompleted == true {  
  result.RequeueAfter = time.Minute \* 2  
 }  
 return  
}

You can refer to the complete file here https://github.com/gauravsarma1992/todolist-k8s-operator/blob/main/internal/controller/todolist_controller.go#L48-L89.

Once the controller is done, you can create and deploy your code to the kubernetes cluster already created.

make manifests  
make install  
make run

This will run the manager with the required reconciliation logic hooked in to the k8s cluster.

Testing time!!#

Create the todolist object

apiVersion: todo.sarmag.co/v1  
kind: TodoList  
metadata:  
  name: jack   
  namespace: operator-namespace  
spec:  
  task: jack

This creates a todolist object called jack in the k8s namespace namedoperator-namespace .

You can refer to the full file here https://github.com/gauravsarma1992/todolist-k8s-operator/blob/main/samples/todo.yml#L1.

apiVersion: v1  
kind: Pod  
metadata:  
  name: jack  
  namespace: operator-namespace  
spec:  
  containers:  
  \- name: ubuntu  
    image: ubuntu:latest  
    \# Just sleep forever  
    command: \[ "sleep" \]  
    args: \[ "infinity" \]

You can refer to the full file here https://github.com/gauravsarma1992/todolist-k8s-operator/blob/main/samples/pod.yml#L1-L12.

Once you create the pod, you will see this specific log line https://github.com/gauravsarma1992/todolist-k8s-operator/blob/main/internal/controller/todolist_controller.go#L48-L89. This means that the operator was able to find a pod with the same name as the task.

Watching on pod events#

One problem with the current approach is the operator only listens on the events of the TodoList type, whereas it should also monitor the pod events so that it can update the state accordingly. In order to ensure the reconcilation loops when the pod events change, chain the following method on the manager.

func (r \*MyController) SetupWithManager(mgr ctrl.Manager) (err error) {  
  err = ctrl.NewControllerManagedBy(mgr).  
    For(&todov1.TodoList{}).  
    Watches(&source.Kind{Type: &corev1.Pod{}}, &handler.EnqueueRequestForObject{}).  
    Complete(r)  
  return  
}

If you want to ensure that the operator only watch on the pods it has created, you can create the pod and set the OwnerReferences by calling SetControllerReference on the pod.

You can then create the manager by using the Owns method

func (r \*MyController) SetupWithManager(mgr ctrl.Manager) (err error) {  
 err = ctrl.NewControllerManagedBy(mgr).  
  For(&todov1.TodoList{}).  
  Owns(&corev1.Pod{}).  
  Complete(r)  
  return  
}

Watching on external events#

What if we want the reconcilation loop to run on external events as well?

You can create a goroutine which sends an event to the reconciliation loop every 5 seconds

func (r \*TodoListReconciler) startTickerLoop(periodicReconcileCh chan event.GenericEvent) {  
 var (  
  ticker \*time.Ticker  
  count  int  
 )  
 ticker = time.NewTicker(time.Second \* 5)  
 defer ticker.Stop()  
  
 for {  
  select {  
  case <-ticker.C:  
   periodicReconcileCh <- event.GenericEvent{Object: &todov1.TodoList{ObjectMeta: metav1.ObjectMeta{Name: "jack", Namespace: "operator-namespace"}}}  
  
   count += 1  
   if count > 100 {  
    return  
   }  
  }  
 }  
}

You can then change the manager setup to also watch on the periodReconcileCh channel

func (r \*TodoListReconciler) SetupWithManager(mgr ctrl.Manager) (err error) {  
 var (  
  periodicReconcileCh chan event.GenericEvent  
 )  
 periodicReconcileCh = make(chan event.GenericEvent)  
 go r.startTickerLoop(periodicReconcileCh)  
  
 err = ctrl.NewControllerManagedBy(mgr).  
  For(&todov1.TodoList{}).  
  Watches(&source.Kind{Type: &corev1.Pod{}}, &handler.EnqueueRequestForObject{}).  
  Watches(&source.Channel{Source: periodicReconcileCh}, &handler.EnqueueRequestForObject{}).  
  Complete(r)  
 return  
}

You can hook in the above channel with external events, expose an API which can trigger the loop, etc.

If you want the reconciliation loop to requeue itself after some duration even when it’s successful, you can use RequeueAfter as shown here https://github.com/gauravsarma1992/todolist-k8s-operator/blob/main/internal/controller/todolist_controller.go#L112-L125.

References#