Lab: Tekton on OpenShift

OpenShift Deployment using Tekton Pipelines

Tekton is an open source project to configure and run CI/CD pipelines within a OpenShift/Kubernetes cluster.

Introduction

In this tutorial you'll learn

  • what are the basic concepts used by Tekton pipelines

  • how to create a pipeline to build and deploy a container

  • how to run the pipeline, check its status and troubleshoot problems

Also, check out this very good tutorial by Red Hat.

Prerequisites

Before you start the tutorial you must set up a OpenShift environment with Tekton installed.

Follow this guide on installing OpenShift Pipelines

Create a route for the OpenShift registry if you have not done so already.

oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge

Estimated time

1 hour

Steps

1. Tekton pipeline concepts

Tekton provides a set of extensions to Kubernetes, in the form of Custom Resources, for defining pipelines. The following diagram shows the resources used in this tutorial. The arrows depict references from one resource to another resource.

crd

The resources are used as follows.

  • A PipelineRun defines an execution of a pipeline. It references the Pipeline to run.

  • A Pipeline defines the set of Tasks that compose a pipeline.

  • A Task defines a set of build steps such as compiling code, running tests, and building and deploying images.

We will go into more detail about each resource during the walkthrough of the example.

Let's create a simple pipeline that

  • builds a Docker image from source files and pushes it to your private container registry

  • deploys the image to your Kubernetes cluster

2. Clone the repository

You should clone this project to your workstation since you will need to edit some of the yaml files before applying them to your cluster. Check out the beta-update branch after cloning.

git clone https://github.com/odrodrig/tekton-tutorial
cd tekton-tutorial
git checkout beta-update

We will work from the bottom-up, i.e. first we will define the Task resources needed to build and deploy the image, then we'll define the Pipeline resource that references the tasks, and finally we'll create the PipelineRun resource needed to run the pipeline.

3. Create a task to clone the Git repository

The first thing that the pipeline needs is a task to clone the Git repository that the pipeline is building. This is such a common function that you don't need to write this task yourself. Tekton provides a library of reusable tasks called the Tekton catalog. It provides a git-clone task which is described here.

The task is reproduced below so that we can talk about it.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-clone
spec:
workspaces:
- name: output
description: The git repo will be cloned onto the volume backing this workspace
params:
- name: url
description: git url to clone
type: string
- name: revision
description: git revision to checkout (branch, tag, sha, ref�)
type: string
default: master
- name: submodules
description: defines if the resource should initialize and fetch the submodules
type: string
default: "true"
- name: depth
description: performs a shallow clone where only the most recent commit(s) will be fetched
type: string
default: "1"
- name: sslVerify
description: defines if http.sslVerify should be set to true or false in the global git config
type: string
default: "true"
- name: subdirectory
description: subdirectory inside the "output" workspace to clone the git repo into
type: string
default: "src"
- name: deleteExisting
description: clean out the contents of the repo's destination directory (if it already exists) before trying to clone the repo there
type: string
default: "false"
results:
- name: commit
description: The precise commit SHA that was fetched by this Task
steps:
- name: clone
image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:latest
script: |
CHECKOUT_DIR="$(workspaces.output.path)/$(params.subdirectory)"
cleandir() {
# Delete any existing contents of the repo directory if it exists.
#
# We don't just "rm -rf $CHECKOUT_DIR" because $CHECKOUT_DIR might be "/"
# or the root of a mounted volume.
if [[ -d "$CHECKOUT_DIR" ]] ; then
# Delete non-hidden files and directories
rm -rf "$CHECKOUT_DIR"/*
# Delete files and directories starting with . but excluding ..
rm -rf "$CHECKOUT_DIR"/.[!.]*
# Delete files and directories starting with .. plus any other character
rm -rf "$CHECKOUT_DIR"/..?*
fi
}
if [[ "$(params.deleteExisting)" == "true" ]] ; then
cleandir
fi
/ko-app/git-init \
-url "$(params.url)" \
-revision "$(params.revision)" \
-path "$CHECKOUT_DIR" \
-sslVerify="$(params.sslVerify)" \
-submodules="$(params.submodules)" \
-depth="$(params.depth)"
cd "$CHECKOUT_DIR"
RESULT_SHA="$(git rev-parse HEAD | tr -d '\n')"
EXIT_CODE="$?"
if [ "$EXIT_CODE" != 0 ]
then
exit $EXIT_CODE
fi
# Make sure we don't add a trailing newline to the result!
echo -n "$RESULT_SHA" > $(results.commit.path)

A task can have one or more steps. Each step defines an image to run to perform the function of the step. This task has one step that uses a Tekton-provided container to clone a Git repo.

A task can have parameters. Parameters help to make a task reusable. This task accepts many parameters such as:

  • the URL of the Git repository to clone

  • the revision to check out

Parameters can have default values provided by the task or the values can be provided by the Pipeline and PipelineRun resources that we'll see later. Steps can reference parameter values by using the syntax $(params.name) where name is the name of the parameter. For example the step uses $(params.url) to reference the url parameter value.

The task requires a workspace where the clone is stored. From the point of view of the task, a workspace provides a file system path where it can read or write data. Steps can reference the path using the syntax $(workspaces.name.path) where name is the name of the workspace. We'll see later how the workspace becomes associated with a storage volume.

Apply the file to your cluster to create the task.

kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/v1beta1/git/git-clone.yaml

4. Create a task to build an image and push it to a container registry

The next function that the pipeline needs is a task that builds a docker image and pushes it to a container registry. The catalog provides a kaniko task which does this using Google's kaniko tool. The task is described here.

The task is reproduced below.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: kaniko
spec:
params:
- name: IMAGE
description: Name (reference) of the image to build.
- name: DOCKERFILE
description: Path to the Dockerfile to build.
default: ./Dockerfile
- name: CONTEXT
description: The build context used by Kaniko.
default: ./
- name: EXTRA_ARGS
default: ""
- name: BUILDER_IMAGE
description: The image on which builds will run
default: gcr.io/kaniko-project/executor:latest
workspaces:
- name: source
results:
- name: IMAGE-DIGEST
description: Digest of the image just built.
steps:
- name: build-and-push
workingDir: $(workspaces.source.path)
image: $(params.BUILDER_IMAGE)
# specifying DOCKER_CONFIG is required to allow kaniko to detect docker credential
# https://github.com/tektoncd/pipeline/pull/706
env:
- name: DOCKER_CONFIG
value: /tekton/home/.docker
command:
- /kaniko/executor
- $(params.EXTRA_ARGS)
- --dockerfile=$(params.DOCKERFILE)
- --context=$(workspaces.source.path)/$(params.CONTEXT) # The user does not need to care the workspace and the source.
- --destination=$(params.IMAGE)
- --oci-layout-path=$(workspaces.source.path)/image-digest
securityContext:
runAsUser: 0
- name: write-digest
workingDir: $(workspaces.source.path)
image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/imagedigestexporter:v0.11.1
# output of imagedigestexport [{"key":"digest","value":"sha256:eed29..660","resourceRef":{"name":"myrepo/myimage"}}]
command: ["/ko-app/imagedigestexporter"]
args:
- -images=[{"name":"$(params.IMAGE)","type":"image","url":"$(params.IMAGE)","digest":"","OutputImageDir":"$(workspaces.source.path)/image-digest"}]
- -terminationMessagePath=image-digested
- name: digest-to-results
workingDir: $(workspaces.source.path)
image: stedolan/jq
script: |
cat image-digested | jq -j '.[0].value' | tee /tekton/results/IMAGE-DIGEST

You can see that this task needs a workspace as well. This workspace has the source to build. The pipeline will provide the same workspace that it used for the git-clone task.

The kaniko task also uses a feature called results. A result is a value produced by a task which can then be used as a parameter value to other tasks. This task declares a result named IMAGE-DIGEST which it sets to the digest of the built image. A task sets a result by writing it to a file named /tekton/results/name where name is the name of the result, in this case IMAGE-DIGEST. We will see later how the pipeline uses this result.

You may be wondering about how the task authenticates to the image repository for permission to push the image. This will be covered later on in the tutorial.

Apply the file to your cluster to create the task.

kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/v1beta1/kaniko/kaniko.yaml

5. Create a task to deploy an image to a Kubernetes cluster

The final function that the pipeline needs is a task that deploys a docker image to a Kubernetes cluster. Below is a Tekton task that does this. You can find this yaml file at tekton/tasks/deploy-using-kubectl.yaml.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: deploy-using-kubectl
spec:
workspaces:
- name: git-source
description: The git repo
params:
- name: pathToYamlFile
description: The path to the yaml file to deploy within the git source
- name: imageUrl
description: Image name including repository
- name: imageTag
description: Image tag
default: "latest"
- name: imageDigest
description: Digest of the image to be used.
steps:
- name: update-yaml
image: alpine
command: ["sed"]
args:
- "-i"
- "-e"
- "s;__IMAGE__;$(params.imageUrl):$(params.imageTag);g"
- "-e"
- "s;__DIGEST__;$(params.imageDigest);g"
- "$(workspaces.git-source.path)/$(params.pathToYamlFile)"
- name: run-kubectl
image: lachlanevenson/k8s-kubectl
command: ["kubectl"]
args:
- "apply"
- "-f"
- "$(workspaces.git-source.path)/$(params.pathToYamlFile)"

This task has two steps.

  1. The first step runs sed in an Alpine Linux container to update the yaml file used for deployment with the image that was built by the kaniko task. The step requires the yaml file to have two character strings, __IMAGE__ and __DIGEST__, which are substituted with parameter values.

  2. The second step runs kubectl using Lachlan Evenson's popular k8s-kubectl container image to apply the yaml file to the same cluster where the pipeline is running.

As was the case in the git-clone and kaniko tasks, this task makes use of parameters in order to make the task as reusable as possible. It also needs the workspace to get the deployment yaml file.

You may be wondering about how the task authenticates to the cluster for permission to apply the resource(s) in the yaml file. This will be covered later on in the tutorial.

Apply the file to your cluster to create the task.

kubectl apply -f tekton/tasks/deploy-using-kubectl.yaml

6. Create a pipeline

Below is a Tekton pipeline that runs the tasks we defined above. You can find this yaml file at tekton/pipeline/build-and-deploy-pipeline.yaml.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: build-and-deploy-pipeline
spec:
workspaces:
- name: git-source
description: The git repo
params:
- name: gitUrl
description: Git repository url
- name: pathToContext
description: The path to the build context, used by Kaniko - within the workspace
default: src
- name: pathToYamlFile
description: The path to the yaml file to deploy within the git source
- name: imageUrl
description: Image name including repository
- name: imageTag
description: Image tag
default: "latest"
tasks:
- name: clone-repo
taskRef:
name: git-clone
workspaces:
- name: output
workspace: git-source
params:
- name: url
value: "$(params.gitUrl)"
- name: subdirectory
value: "."
- name: deleteExisting
value: "true"
- name: source-to-image
taskRef:
name: kaniko
runAfter:
- clone-repo
workspaces:
- name: source
workspace: git-source
params:
- name: CONTEXT
value: $(params.pathToContext)
- name: IMAGE
value: $(params.imageUrl):$(params.imageTag)
- name: deploy-to-cluster
taskRef:
name: deploy-using-kubectl
workspaces:
- name: git-source
workspace: git-source
params:
- name: pathToYamlFile
value: $(params.pathToYamlFile)
- name: imageUrl
value: $(params.imageUrl)
- name: imageTag
value: $(params.imageTag)
- name: imageDigest
value: $(tasks.source-to-image.results.IMAGE-DIGEST)

A Pipeline resource contains a list of tasks to run. Each pipeline task is assigned a name within the pipeline; here they are clone-repo, source-to-image, and deploy-using-kubectl.

The pipeline configures each task via the task's parameters. You can choose whether to expose a task parameter as a pipeline parameter, set the value directly, or let the value default inside the task (if it's an optional parameter). For example this pipeline exposes the CONTEXT parameter from the kaniko task (under a different name, pathToContext) but does not expose the DOCKERFILE parameter and allows it to default inside the task.

This pipeline also shows how to take the result of one task and pass it to another task. We saw earlier that the kaniko task produces a result named IMAGE-DIGEST that holds the digest of the built image. The pipeline passes that value to the deploy-using-kubectl task by using the syntax $(tasks.source-to-image.results.IMAGE-DIGEST) where source-to-image is the name used in the pipeline to run the kaniko task.

By default Tekton assumes that pipeline tasks can be executed concurrently. In this pipeline each pipeline task depends on the previous one so they must be executed sequentially. One way that dependencies between pipeline tasks can be expressed is by using the runAfter key. It specifies that the task must run after the given list of tasks has completed. In this example, the pipeline specifies that the source-to-image pipeline task must run after the clone-repo pipeline task.

The deploy-using-kubectl pipeline task must run after the source-to-image pipeline task but it doesn't need to specify the runAfter key. This is because it references a task result from the source-to-image pipeline task and Tekton is smart enough to figure out that this means it must run after that task.

Apply the file to your cluster to create the pipeline.

kubectl apply -f tekton/pipeline/build-and-deploy-pipeline.yaml

7. Define a service account

Before running the pipeline, we need to set up a service account so that it can access protected resources. The service account ties together a couple of secrets containing credentials for authentication along with RBAC-related resources for permission to create and modify certain Kubernetes resources.

The 3rd command will create a secret that contains credentials for accessing the internal OpenShift image registry

Next, run the get routes command to view the registry endpoint. Copy it and insert it into the command after that replacing <Registry route>. Then, replace with the email address you used for the IBM Cloud account.

kubectl get routes -n openshift-image-registry
export REGISTRY=<Registry route>
export EMAIL=<your IBM ID>
kubectl create secret generic ibm-registry-secret --type="kubernetes.io/basic-auth" --from-literal=username=$(oc whoami) --from-literal=password=$(oc whoami -t)
kubectl create secret docker-registry pull-secret --docker-username=$(oc whoami) --docker-password=$(oc whoami -t) --docker-email=$EMAIL --docker-server=$REGISTRY
kubectl annotate secret ibm-registry-secret tekton.dev/docker-0=$REGISTRY

This secret will be used to both push and pull images from your registry.

Now you can create the service account using the following yaml. You can find this yaml file at tekton/pipeline-account.yaml.

apiVersion: v1
kind: ServiceAccount
metadata:
name: pipeline-account
secrets:
- name: ibm-registry-secret
---
apiVersion: v1
kind: Secret
metadata:
name: kube-api-secret
annotations:
kubernetes.io/service-account.name: pipeline-account
type: kubernetes.io/service-account-token
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: pipeline-role
rules:
- apiGroups: [""]
resources: ["services"]
verbs: ["get", "create", "update", "patch"]
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: pipeline-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: pipeline-role
subjects:
- kind: ServiceAccount
name: pipeline-account

This yaml creates the following Kubernetes resources:

  • A ServiceAccount named pipeline-account. The service account references the ibm-registry-secret secret so that the pipeline can authenticate to your private container registry when it pushes and pulls a container image.

  • A Secret named kube-api-secret which contains an API credential (generated by Kubernetes) for accessing the Kubernetes API. This allows the pipeline to use kubectl to talk to your cluster.

  • A Role named pipeline-role and a RoleBinding named pipeline-role-binding which provide the resource-based access control permissions needed for this pipeline to create and modify Kubernetes resources.

Apply the file to your cluster to create the service account and related resources.

kubectl apply -f tekton/pipeline-account.yaml

8. Create a PipelineRun

We've defined reusable Pipeline and Task resources for building and deploying an image. It is now time to look at how one runs the pipeline.

Below is a Tekton PipelineRun resource that runs the pipeline we defined above. You can find this yaml file at tekton/run/picalc-pipeline-run.yaml.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: picalc-pr-
spec:
pipelineRef:
name: build-and-deploy-pipeline
params:
- name: gitUrl
value: https://github.com/odrodrig/tekton-tutorial
- name: pathToYamlFile
value: kubernetes/picalc.yaml
- name: imageUrl
value: <REGISTRY>/default/picalc
- name: imageTag
value: "1.0"
serviceAccountName: pipeline-account
workspaces:
- name: git-source
persistentVolumeClaim:
claimName: picalc-source-pvc

Although this file is small there is a lot going on here. Let's break it down from top to bottom:

  • The PipelineRun does not have a fixed name. It uses generateName to generate a name each time it is created. This is because a particular PipelineRun resource executes the pipeline only once. If you want to run the pipeline again, you cannot modify an existing PipelineRun resource to request it to re-run -- you must create a new PipelineRun resource. While you could use name to assign a unique name to your PipelineRun each time you create one, it is much easier to use generateName.

  • The Pipeline resource is identified under the pipelineRef key.

  • Parameters exposed by the pipeline are set to specific values such as the Git repository to clone, the image to build, and the yaml file to deploy. This example builds a go program that calculates an approximation of pi. The source includes a Dockerfile which runs tests, compiles the code, and builds an image for execution.

    You must edit the picalc-pipeline-run.yaml file to substitute the values of <REGISTRY> with the information for your private container registry.

    • To find the value for <REGISTRY>, enter the command oc get routes -n openshift-image-registry.

  • The service account named pipeline-account which we created earlier is specified to provide the credentials needed for the pipeline to run successfully.

  • The workspace used by the pipeline to clone the Git repository is mapped to a persistent volume claim which is a request for a storage volume.

Before you run the pipeline for the first time, you must create the persistent volume claim for the workspace.

kubectl create -f tekton/picalc-pipeline-pvc.yaml

The persistent volume claim requests Kubernetes to obtain a storage volume. Since each PipelineRun references the same claim and thus the same volume, this means the PipelineRun can only be run serially to avoid conflicting use of the volume. There is funtionality coming in Tekton to allow each PipelineRun to create its own persistent volume claim and thus use its own volume.

Check that the persistent volume claim is bound before continuing.

$ kubectl get pvc picalc-source-pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
picalc-source-pvc Bound pvc-662946bc-57f2-4ba5-982c-b0fa9db1d065 20Gi RWO ibmc-file-bronze 2m

9. Run the pipeline

All the pieces are in place to run the pipeline.

$ kubectl create -f tekton/run/picalc-pipeline-run.yaml
pipelinerun.tekton.dev/picalc-pr-c7hsb created

Note that we're using kubectl create here instead of kubectl apply. As mentioned previously a given PipelineRun resource can run a pipeline only once so you need to create a new one each time you want to run the pipeline. kubectl will respond with the generated name of the PipelineRun resource.

Let's view the status of our pipeline run

kubectl get pipelinerun
NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME
picalc-pr-dqwqb Unknown Running 5s

The pipeline will be successful when SUCCEEDED is True.

NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME
picalc-pr-dqwqb True Succeeded 7m26s 4m49s

Check the status of the Kubernetes deployment. It should be ready.

$ kubectl get deploy picalc
NAME READY UP-TO-DATE AVAILABLE AGE
picalc 1/1 1 1 9m

You can curl the application using its NodePort service. First display the nodes and choose one of the node's external IP addresses. Then display the service to get its NodePort.

$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
10.221.22.11 Ready <none> 7d23h v1.16.8+IKS 10.221.22.11 150.238.236.26 Ubuntu 18.04.4 LTS 4.15.0-96-generic containerd://1.3.3
10.221.22.49 Ready <none> 7d23h v1.16.8+IKS 10.221.22.49 150.238.236.21 Ubuntu 18.04.4 LTS 4.15.0-96-generic containerd://1.3.3
$ kubectl get svc picalc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
picalc NodePort 172.21.199.71 <none> 8080:30925/TCP 9m
$ curl 150.238.236.26:30925?iterations=20000000
3.1415926036

Debugging a failed PipelineRun

Let's take a look at what a PipelineRun failure would look like. Edit the PipelineRun yaml and change the gitUrl parameter to a non-existent Git repository to force a failure. Then create a new PipelineRun and describe it after letting it run for a minute or two.

$ kubectl create -f tekton/picalc-pipeline-run.yaml
pipelinerun.tekton.dev/picalc-pr-sk7md created
$ tkn pipelinerun describe picalc-pr-sk7md
Name: picalc-pr-sk7md
Namespace: default
Pipeline Ref: build-and-deploy-pipeline
Service Account: pipeline-account
Status
STARTED DURATION STATUS
2 minutes ago 41 seconds Failed
Message
TaskRun picalc-pr-sk7md-clone-repo-8gs25 has failed (&#34;step-clone&#34; exited with code 1 (image: &#34;gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/[email protected]:bee98bfe6807e8f4e0a31b4e786fd1f7f459e653ed1a22b1a25999f33fa9134a&#34;); for logs run: kubectl -n default logs picalc-pr-sk7md-clone-repo-8gs25-pod-v7fg8 -c step-clone)
Resources
No resources
Params
NAME VALUE
gitUrl https://github.com/IBM/tekton-tutorial-not-there
pathToYamlFile kubernetes/picalc.yaml
imageUrl us.icr.io/gregd/picalc
imageTag 1.0
Taskruns
NAME TASK NAME STARTED DURATION STATUS
picalc-pr-sk7md-clone-repo-8gs25 clone-repo 2 minutes ago 41 seconds Failed

The output tells us that the clone-repo pipeline task failed. The Message also tells us how to get the logs from the pod which was used to run the task:

for logs run: kubectl -n default logs picalc-pr-sk7md-clone-repo-8gs25-pod-v7fg8 -c step-clone

If you run that kubectl logs command you will see that there is a failure trying to fetch the non-existing Git repository.

An even easier way to get the logs from a PipelineRun is to use the tkn CLI.

tkn pipelinerun logs picalc-pr-sk7md-clone-repo-8gs25-pod-v7fg8 -t clone-repo

If you omit the -t flag then the command will get the logs for all pipeline tasks that executed.

You can also get the logs for the last PipelineRun for a particular Pipeline using this command:

tkn pipeline logs build-and-deploy-pipeline -L

You should delete a PipelineRun when you no longer have a need to reference its logs. Deleting the PipelineRun deletes the pods that were used to run the pipeline tasks.

10. Check it out in OpenShift Console

OpenShift provides a nice UI for the pipelines and the applications deployed. From the OpenShift console, click developer in the upper left drop-down to get to the developer view. Then click Topology to view your running app.

OpenShift Topology

Click Pipelines to explore the pipline your created and explore the PipelineRuns

OpenShift Pipelines

Summary

Tekton provides simple, easy-to-learn features for constructing CI/CD pipelines that run on Kubernetes. This tutorial covered the basics to get you started building your own pipelines. There are more features available and many more planned for upcoming releases.