How to get started developing for the Pipelines-as-Code project #

Please read the Code of conduct #

It’s important: https://github.com/openshift-pipelines/pipelines-as-code/blob/main/code-of-conduct.md

Use the all in one install on kind to develop #

It uses kind under docker. You start it with:

make dev

When it finished you will have the following installed in your kind cluster:

Kind Cluster deployment
Internal registry to push to from ko
A ingress controller with nginx for routing.
Tekton and Dashboard installed with a ingress route.
Pipelines as code deployed from your repo with ko.
Gitea service running locally so you can run your E2E tests that targets gitea (the most comprehensive ones)

By default it will try to install from $GOPATH/src/github.com/openshift-pipelines/pipelines-as-code, if you want to override it you can set the PAC_DIRS environment variable.

It will deploy under the nip.io domain reflector, the URL will be :
- http://controller.paac-127-0-0-1.nip.io
- http://dashboard.paac-127-0-0-1.nip.io
You will need to create secret yourself, if you have the pass cli installed you can point to a folder which contains : github-application-id github-private-key webhook.secret As configured from your GitHub application. Configure PAC_PASS_SECRET_FOLDER environment variable to point to it. For example :
```
pass insert github-app/github-application-id
pass insert github-app/webhook.secret
pass insert -m github-app/github-private-key
```

If you need to redeploy your pac install (and only pac) you can do :

./hack/dev/kind/install.sh -p

make rdev

or you can do this directly with ko :

env KO_DOCKER_REPO=localhost:5000 ko apply -f ${1:-"config"} -B

more flags: -b to only do the kind creation+nginx+docker image, -r to install from latest stable release (override with env variable PAC_RELEASE) instead of ko. -c will only do the pac configuration (ie: creation of secrets/ingress etc..)
see the install.sh -h for all flags

Gitea #

Gitea is “unofficially” supported. You just need to configure Gitea the same way you do for other webhook methods with a token.

Here is an example of a Gitea NS/CRD/Secret (set to empty):

---
apiVersion: v1
kind: Namespace
metadata:
  name: gitea

---
apiVersion: "pipelinesascode.tekton.dev/v1alpha1"
kind: Repository
metadata:
  name: gitea
  namespace: gitea
spec:
  url: "https://gitea.my.com/owner/repo"
  git_provider:
    user: "git"
    url: "Your gitea installation URL, i.e: https://gitea.my.com/"
    secret:
      name: "secret"
      key: token
    webhook_secret:
      name: "secret"
      key: "webhook"
---
apiVersion: v1
kind: Secret
metadata:
  name: gitea-home-chmouel
  namespace: gitea
type: Opaque
stringData:
  token: "your token has generated in gitea"
  webhook: "" # make sure it's empty when you set this up on the interface and here

There is some gotchas with the webhook validation secret, Pipelines-as-Code detect a Gitea install and let the user set a empty webhook secret (by default it’s enforced).

The install.sh script will by default spin up a new instance of GITEA to play with and run the Gitea E2E tests.

You will need to create a Hook URL generated from https://hook.pipelinesascode.com/new into the environment variable TEST_GITEA_SMEEURL.

The defaults are :

URL: https://localhost:3000/
Admin Username: pac
Admin Password: pac

The E2E tests will automatically create repo using the admin username for each tests.

Debugging E2E #

As long you have the secrets setup you should be able to run the e2e tests properly. Gitea are the easiest to run (since they are self contained), for the other you will need to setup some environment variables.

See the e2e on kind workflow for all the variables set by provider.

By default the E2E tests cleanups after themselves if you want to keep the PR/MR opens and the namespace where the test has been created you can set the TEST_NOCLEANUP environment variable to true.

Debugging controller #

Create a hook URL and point your app/webhook to it. Use gosmee to forward the requests from GitHub to your locally installed controller (this can be either run on your debugger or inside kind).

An option of gosmee is to save the replay to a directory with --saveDir /tmp/save. If go to that directory a shell script will be created to replay the request that was sent directly to the controller without having to go through another push.

Use snazy to watch the logs, it support pac by adding some context like which GitHub provider.

snazy screenshot

Using the Makefile targets #

Several target in the Makefile is available, if you need to run them manually. You can list all the makefile targets with:

make help

For example to test and lint the go files :

make test lint-go

If you add a CLI command with help you will need to regenerate the golden files :

make update-golden

Configuring the Pre Push Git checks #

We are using several tools to verify that pipelines-as-code is up to a good coding and documentation standard. We use pre-commit tools to ensure before you send your PR that the commit is valid.

First you need to install pre-commit:

https://pre-commit.com/

It should be available as package on Fedora and Brew or install it with pip.

When you have it installed add the hook to your repo by doing :

pre-commit install

This will run several hooks on the files that has been changed before you push to your remote branch. If you need to skip the verification (for whatever reason), you can do :

git push --no-verify

or you can disable individual hook with the SKIP variable:

SKIP=lint-md git push

If you want to manually run on everything:

make pre-commit

Developing the Documentation #

Documentation is important to us, most of the time new features or change of behaviour needs to include documentation part of the Pull Request.

We use hugo, if you want to preview your change, you need to install hugo and do a :

make docs-dev

this will start a hugo server with live preview of the docs on :

https://localhost:1313

When we push the release, the docs get rebuilt by CloudFare.

By default the website https://pipelinesascode.com only contains the “stable” documentation. If you want to preview the dev documentation as from main you need to go to this URL:

https://main.pipelines-as-code.pages.dev

Documentation when we are doing the Release Process #

See here release-process

How to update all dependencies in Pipelines-as-Code #

Go Modules #

Unless if we that’s not possible we try to update all dependencies to the latest version as long it’s compatible with the Pipeline version as shipped by OpenShift Pipelines Operator (which should be conservative).

Every time you do a go modules update check if we can remove the replace clause that pins a dependency to a specific version/commit or match the replace to the tektoncd/pipeline version.

Update all go modules:
```
go get -u ./...
make vendor
```
Go to https://github.com/google/go-github and note the latest go version for example: v59
Open a file that use the go-github library (ie: pkg/provider/github/detect.go) and check the old version, for example: v56

Run this sed command:

find -name '*.go'|xargs sed -i 's,github.com/google/go-github/v56,github.com/google/go-github/v59,'

This will update everything, sometime the library ghinstallation is not updated with the new version, so you will need to keep the old version kept in there. For example you will get this kind of error:

pkg/provider/github/parse_payload.go:56:33: cannot use &github.InstallationTokenOptions{…} (value of type *"github.com/google/go-github/v59/github".InstallationTokenOptions) as *"github.com/google/go-github/v57/github".InstallationTokenOptions value in assignment

Check that everything compiles and tests are passing with this command:
```
make allbinaries test lint
```
Some structs needs to be updated, some of them are going to fail on deprecated, so you will need to figure how to update them. Don’t be lazy and avoid the update with a nolint or a pin to a dep you only delay the inevitable until the problem come back and hit you harder.

Go version #

Check that the go version is updated to the latest RHEL version:
```
docker pull golang
docker run golang go version
```
If this not the same as what we have in go.mod then you need to update the go.mod version. then you need to update for example here 1.20:
```
go mod tidy -go=1.20
```
Grep for the image go-toolset everywhere with:
```
git grep golang:
```
and change the old version to the new version

Update the pre-commit rules #

pre-commit autoupdate

Update the vale rules #

vale sync
make lint-md

Tools that are useful #

Several tools are used on CI and in pre-commit, the non exhaustive list you need to have on your system:

golangci-lint - For golang lint
yamllint - For YAML lint
shellcheck - For shell scripts linting
ruff - Python code formatter check
vale - For grammar check
markdownlint - For markdown lint
codespell - For code spelling
gitlint - For git commit messages lint
hugo - For documentation
ko - To rebuild and push change to kube cluster.
kind - For local devs
snazy - To parse json logs nicely
pre-commit - For checking commits before sending it to the outer loop.
pass - For getting/storing secrets
gosmee - For replaying webhooks