(Spoiler) In short, it appears that you currently can't use OpenShift Online as a
Kubernetes cluster for gitlab.com. I'd be happy to find out I'm wrong though.
I don't want to learn how to administer a Kubernetes cluster right now. I want
less infrastracture to manage, not more. With that in mind, Openshift
Online seems like the ideal Kubernetes backend to run my GitLab continuous
integration/deployment services. I already pay for and use the platform, it's
run by a company I trust and respect and provides a great web interface. Let's
set it up!
-
Given an account on gitlab.com, create a new project,
for example by forking
minimal-ruby-app.
-
Given either a "Starter" or "Pro" account on OpenShift
Online, create a project called
gitlab-managed-apps, create a service
account for
GitLab to use, and copy the service token:
$ oc login
$ oc new-project gitlab-managed-apps
$ oc create serviceaccount gitlab-robot
$ oc policy add-role-to-user admin system:serviceaccount:gitlab-managed-apps:gitlab-robot
$ oc serviceaccounts get-token gitlab-robot
If you log in to the OpenShift web console and visit the
gitlab-managed-apps project, you'll find our new service account under
"Resources, Membership".
Alternately you can create the project and the service account via the web
console, but I don't believe it's possible to view the service account
token without the oc command line.
-
Returning to GitLab, select "Settings, CI / CD" and expand the "Runners
settings". Click "Install Runner on Kubernetes". Click "Add Kubernetes
cluster", then click "Add an existing Kubernetes cluster". Set the cluster
name, API URL and token. For me these are something like:
Note that the "API URL" is the base URL from your OpenShift Online web
console, excluding the bit after the "/" and the token is the one we
exported above.
-
So far so good. On the following screen, under "Applications" click the
"Install" button next to "Helm Tiller". If it's not clickable, try refreshing
the page. If the service account is set up right, the indicator will spin for
a bit, then display "Installing" for about a minute, then issue an error:
Something went wrong while installing Helm Tiller
/bin/sh: can't create /etc/apk/repositories: Permission denied
If you're fast, you can briefly see a pod being spun up and deleted on the
OpenShift web console.
Unfortunately I wasn't able to overcome this issue, that's as far as I got.
There's no clear indication of what actual problem is and, as far as I know, no
way to dive in and find out what's going on. I suspect the issue may be related
to OpenShift Online's default policy of not allowing containers to run as
root, but I can't
say for sure.
The disappointing thing is that back in 2016 and 2017, there did seem to be some
sense of partnership between GitLab
and OpenShift. There are a number of official GitLab/OpenShift videos from
around this time, demonstrating the manual setup of GitLab Runner on OpenShift
(pre-GitLab Kubernetes integration). Unfortunately, there don't seem to be any
official and stable GitLab Runner templates for OpenShift. This
official-sounding
one is marked
EXPERIMENTAL and this unofficial
one
template I came across advises that it is not compatible with OpenShift Online
(only self-hosted OpenShift). The Review
Apps mention down the bottom that
docs for OpenShift/kubernetes are soon to be added.
The approach I have had some success with is using the .gitlab-ci.yml
template that GitLab
provide
for OpenShift via the "Set up CI/CD" button. This uses gitlab.com's shared
runners, but deploys to OpenShift. This
review-apps-openshift
looks promising too. But that's another blog post.
An aside: How is this meant to work?
In contrast, the GitLab CI/CD and Google Kubernetes Engine (GKE) integration
appears to be getting significantly more promotion and quite deep-running
integration from the GitLab team. While I don't use or recommend Google services
to any of my clients due to privacy concerns and lock-in from proprietary
network services, I gave this a brief trial to see how the GitLab/Kubernetes
integration is meant to work.
GitLab provide a custom setup process and documentation for GKE. This mostly
avoids the need to use the GKE web interface at all, aside from creating an
account, creating a project and enabling API permissions. That's a good thing,
since the GKE web interface seemed to be very buggy. Once Kubernetes was
configured, GitLab did the rest; launching a cluster and prompting me to install
Helm Tiller, GitLab Runner and create a wildcard DNS entry. Using this setup, I
was able to enable the Kubernetes GitLab Runner and use Auto Deploy to
successfully deploy "minimal-ruby-app" to the custom domain
The overall user experience with GKE wasn't actually significantly different to
that of using the 80 line OpenShift .gitlab-ci.yml mentioned above; just that
instead of using gitlab.com shared runners, the build/test/deploy work is done
from on Kubernetes. The major downside of the Kubernetes approach though (either
GKE or OpenShift if it worked), is that Auto Deploy magic is actually 800 lines
of shell script and YAML which, sooner or later, will need troubleshooting. So
while it makes for a slick webcast, I'm not sure this is a path I want to take
after all.
