This is the second part of our series focusing on Kubernetes Operators, and it shows how you can build a Kubernetes Operator based on the Bitnami Apache Helm chart. Note that you can refer to the steps in this tutorial to build an operator for your own applications.
kubectl get all --namespace olm
NAME READY STATUS RESTARTS AGE pod/catalog-operator-64b6b59c4f-brck9 1/1 Running 0 3m28s pod/olm-operator-844fb69f58-fn57f 1/1 Running 0 3m28s pod/operatorhubio-catalog-5s8k2 1/1 Running 0 3m4s pod/packageserver-65df5d5cc9-nz26h 1/1 Running 0 3m2s pod/packageserver-65df5d5cc9-x7hwc 1/1 Running 0 3m2s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/operatorhubio-catalog ClusterIP 10.103.1.120 <none> 50051/TCP 3m3s service/v1-packages-operators-coreos-com ClusterIP 10.99.75.171 <none> 443/TCP 3m3s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/catalog-operator 1/1 1 1 3m28s deployment.apps/olm-operator 1/1 1 1 3m28s deployment.apps/packageserver 2/2 2 2 3m2s NAME DESIRED CURRENT READY AGE replicaset.apps/catalog-operator-64b6b59c4f 1 1 1 3m28s replicaset.apps/olm-operator-844fb69f58 1 1 1 3m28s replicaset.apps/packageserver-65df5d5cc9 2 2 2 3m2s
In this section, we’ll walk you through the process of setting up the Apache Operator using Bitnami’s Helm chart.
helm repo add bitnami https://charts.bitnami.com
"bitnami" has been added to your repositories
operator-sdk new
command, and passing it the following arguments:apache-operator
).--api-version
flag with the Kubernetes apiVersion. The format is $GROUP_NAME/$VERSION
. In this tutorial, we’ll be using appfleet.com/v1alpha1
.--kind flag
with the name of the Kubernetes CRD (Apache
)--type
flag with the type of operator. We’ll be using helm
. Other valid types are Go
and Ansible
helm-chart
flag with the name of the Helm chart (bitnami/apache
)operator-sdk new apache-operator --api-version=appfleet.com/v1alpha1 --kind=Apache --type=helm --helm-chart=bitnami/apache
INFO[0000] Creating new Helm operator 'apache-operator'. INFO[0001] Created helm-charts/apache INFO[0001] Generating RBAC rules WARN[0001] The RBAC rules generated in deploy/role.yaml are based on the chart's default manifest. Some rules may be missing for resources that are only enabled with custom values, and some existing rules may be overly broad. Double check the rules generated in deploy/role.yaml to ensure they meet the operator's permission requirements. INFO[0001] Created build/Dockerfile INFO[0001] Created watches.yaml INFO[0001] Created deploy/service_account.yaml INFO[0001] Created deploy/role.yaml INFO[0001] Created deploy/role_binding.yaml INFO[0001] Created deploy/operator.yaml INFO[0001] Created deploy/crds/appfleet.com_v1alpha1_apache_cr.yaml INFO[0001] Generated CustomResourceDefinition manifests. INFO[0001] Project creation complete.
This command creates the following directory structure:
tree apache-operator -L 2
apache-operator ├── build │ └── Dockerfile ├── deploy │ ├── crds │ ├── operator.yaml │ ├── role.yaml │ ├── role_binding.yaml │ └── service_account.yaml ├── helm-charts │ └── apache └── watches.yaml
Things to note:
watches.yaml
file:cat apache-operator/watches.yaml
--- - version: v1alpha1 group: appfleet.com kind: Apache chart: helm-charts/apache
Dockerfile
uses the quay.io/operator-framework/helm-operator:v0.15.1
image as the base, and then it copies watches.yaml
file and the Helm charts:cat apache-operator/build/Dockerfile
FROM quay.io/operator-framework/helm-operator:v0.15.1 COPY watches.yaml ${HOME}/watches.yaml COPY helm-charts/ ${HOME}/helm-charts/
apache-operator
directory, and then entering the following operator-sdk build
command:operator-sdk build apache-operator:v0.1
INFO[0000] Building OCI image apache-operator:v0.1 Sending build context to Docker daemon 64.51kB Step 1/3 : FROM quay.io/operator-framework/helm-operator:v0.15.1 ---> 450a3ca2d02d Step 2/3 : COPY watches.yaml ${HOME}/watches.yaml ---> Using cache ---> db5c285c02fb Step 3/3 : COPY helm-charts/ ${HOME}/helm-charts/ ---> 50255ede17de Successfully built 50255ede17de Successfully tagged apache-operator:v0.1 INFO[0003] Operator build complete.
docker images | grep apache
apache-operator v0.1 50255ede17de 38 seconds ago 174MB
You can think of Quay as something similar to GitHub, but for Docker images. It’s a registry where you can host images and share them. There are a couple of ways to set up quay.io
with Docker. For the sake of simplicity, you’ll use the docker login
command. It’s important to note that the docker login
command stores the password you enter as plain-text. Thus, you should first generate an encrypted password.
quay.io
password:quay.io
by entering the following command:docker login -u="<YOUR_USERNAME>" -p="<YOUR_ENCRYPTED_PASSWORD>" quay.io
WARNING! Using --password via the CLI is insecure. Use --password-stdin. Login Succeeded
To do this, you first need to properly tag the image with the hostname of the registry, and the name of your repository. Then, you can push the image.
apache-operator
into the quay.io/andreipope
repository:docker tag apache-operator:v0.1 quay.io/andreipope/apache-operator:v0.1
Note that the image name is comprised by a slash-separated list of the:
quay.io
)andreipope
)apache-version
).☞ The name of our repository is andreipope
, but yours will be different.
docker push
command:docker push quay.io/andreipope/apache-operator:v0.1
b8325e5fabd7: Pushed 2f4354fc6a73: Pushed b496b494f6f9: Pushed 9fd48ecc1227: Pushed 0141daa77f22: Pushed 27cd2023d60a: Pushed 4b52dfd1f9d9: Pushed v0.1: digest: sha256:7230926984fc3d688e02764748441a74907eeb772e3b51eb06b1bac225ba9f98 size: 1778
You are now ready to deploy the Apache Operator. Before that, you must customize the specs.
deploy/operator.yaml
file in a plain-text editor and update the placeholder image: REPLACE_IMAGE
with the location of your image (quay.io/andreipope/apache-operator:v0.1
)Your deploy.operator.yaml
file should look similar to the following:
apiVersion: apps/v1 kind: Deployment metadata: name: apache-operator spec: replicas: 1 selector: matchLabels: name: apache-operator template: metadata: labels: name: apache-operator spec: serviceAccountName: apache-operator containers: - name: apache-operator # Replace this with the built image name image: quay.io/andreipope/apache-operator:v0.1 imagePullPolicy: Always env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: OPERATOR_NAME value: "apache-operator"
kubectl create
commands to deploy the Apache Operator:kubectl create -f deploy/service_account.yaml kubectl create -f deploy/role.yaml kubectl create -f deploy/role_binding.yaml kubectl create -f deploy/operator.yaml
serviceaccount/expressjs-operator created role.rbac.authorization.k8s.io/expressjs-operator created rolebinding.rbac.authorization.k8s.io/expressjs-operator created deployment.apps/expressjs-operator created
kubectl get deployment NAME READY UP-TO-DATE AVAILABLE AGE apache-operator 1/1 1 1 24s
☞ Note that the deployment doesn’t define the spec for the Apache cluster. You’ll describe the Apache cluster in the next section, once the Operator is running.
kubectl get pods
kubectl get pods NAME READY STATUS RESTARTS AGE apache-operator-6d5795f879-np6pr 1/1 Running 1 38s
In the Set Up the Apache Operator section, you created a CRD defining a new kind of resource, an Apache cluster. Now, you will apply that spec so that the Operator starts watching the Apache resources. Then, you will deploy the Apache cluster itself.
kubectl apply -f deploy/crds/appfleet.com_apaches_crd.yaml customresourcedefinition.apiextensions.k8s.io/apaches.appfleet.com created
customresourcedefinition.apiextensions.k8s.io/apaches.appfleet.com created
kubectl apply -f deploy/crds/appfleet.com_v1alpha1_apache_cr.yaml
apache.appfleet.com/example-apache created
kubectl get pods
NAME READY STATUS RESTARTS AGE apache-operator-6d5795f879-np6pr 1/1 Running 4 2m53s example-apache-7cf789fc98-462dr 1/1 Running 0 39s
kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE apache-operator 1/1 1 1 5m43s example-apache 2/2 2 2 3m29s
At this point, you have a running Apache cluster. To add another instance, you must modify the replicaCount
field in the deploy/crds/appfleet.com_v1alpha1_apache_cr.yamlapache.appfleet.com/example-apache
file. Then, you need to apply the new spec.
deploy/crds/appfleet.com_v1alpha1_apache_cr.yamlapache.appfleet.com/example-apache
file in a plain-text editor, and specify spec.replicaCount: 2
.apiVersion: appfleet.com/v1alpha1 kind: Apache metadata: name: example-apache spec: # Default values copied from <project_dir>/helm-charts/apache/values.yaml affinity: {} cloneHtdocsFromGit: enabled: false interval: 60 git: pullPolicy: IfNotPresent registry: docker.io repository: bitnami/git tag: 2.25.0-debian-10-r0 image: debug: false pullPolicy: IfNotPresent registry: docker.io repository: bitnami/apache tag: 2.4.41-debian-10-r0 ingress: annotations: {} certManager: false enabled: false hostname: example.local secrets: null tls: - hosts: - example.local secretName: example.local-tls livenessProbe: enabled: true failureThreshold: 6 initialDelaySeconds: 180 path: / periodSeconds: 20 port: http successThreshold: 1 timeoutSeconds: 5 metrics: enabled: false image: pullPolicy: IfNotPresent registry: docker.io repository: bitnami/apache-exporter tag: 0.7.0-debian-10-r0 podAnnotations: prometheus.io/port: "9117" prometheus.io/scrape: "true" resources: limits: {} requests: {} nodeSelector: {} podAnnotations: {} readinessProbe: enabled: true failureThreshold: 6 initialDelaySeconds: 30 path: / periodSeconds: 10 port: http successThreshold: 1 timeoutSeconds: 5 replicaCount: 2 resources: limits: {} requests: {} service: annotations: {} externalTrafficPolicy: Cluster httpsPort: 443 nodePorts: http: "" https: "" port: 80 type: LoadBalancer tolerations: {}
kubectl apply -f deploy/crds/appfleet.com_v1alpha1_apache_cr.yaml
apache.appfleet.com/example-apache configured
kubectl get pods
NAME READY STATUS RESTARTS AGE apache-operator-6d5795f879-np6pr 1/1 Running 4 4m10s example-apache-7cf789fc98-462dr 1/1 Running 0 116s example-apache-7cf789fc98-fvttm 0/1 ContainerCreating 0 1s
Wait a bit until the second container is created:
kubectl get pods
NAME READY STATUS RESTARTS AGE apache-operator-6d5795f879-np6pr 1/1 Running 0 5m2s example-apache-7cf789fc98-462dr 1/1 Running 0 2m48s example-apache-7cf789fc98-fvttm 1/1 Running 0 53s
In the above output, note that two example-apache
pods are running.
The Apache Operator creates a Kubernetes service which is basically an endpoint where clients can obtain access to the group of pods running Apache. You will use this service to access your cluster using a web browser.
kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE apache-operator-metrics ClusterIP 10.104.41.83 <none> 8686/TCP,8383/TCP 5m46s example-apache LoadBalancer 10.99.54.115 <pending> 80:31058/TCP,443:30362/TCP 3m51s kubernetes ClusterIP 10.96.0.1 <none> 443/TCP
service/example-apache
service:kubectl port-forward service/example-apache 80:80
Forwarding from 127.0.0.1:80 -> 8080 Forwarding from [::1]:80 -> 8080
Now you should have a good understanding of how Kubernetes Operators work. Furthermore, by completing this tutorial you learned how create an operator for your application using a Helm chart.
Thanks for reading!