In CentOS community pipeline service we run API server for fetching details from the service to UI. This API server runs in OKD cluster with having access to read data from project. It also reads data from Jenkins-persistent pod running in that OKD cluster.

Now for running the API pod with proper access to run oc  reads and jenkins reads needs for proper access tokens having access to both. We have jenkins service account which has access to both OKD cluster's project and jenkins-persistent pod. Containers running in OKD are not preferred to be with root user. So we run the container pod with the jenkins service account.

securityContext: {}
serviceAccount: "jenkins"
serviceAccountName: "jenkins"

by adding this to deployment template pod is started with service account jenkins. This is the complete deployment config we are using to deploy the API server to OKD template.

Now API server is deployed and pods are coming up. For getting the details about the projects and validate them we need git clone the container-index repository. And git clone is failing saying no user found for uid 1000040000.

For debugging this I went ahead and ran whoami

$ whoami
whoami: cannot find name for user ID 1000040000

So the user, the pod is running as is not found in /etc/passwd. After some debugging we found out the issue to be pod is running as 'service account jenkins' but it is not having any entry for sa-jenkins in '/etc/passwd`. Container image for the pod is built using this Dockerfile and it does add an user apirunner to run as to avoid running it as root user. So the main problem is container is not running as apirunner user even though it is specified.

This bolg on came in to rescue us. We figured out the sa-jenkins , we are using to run the pod as is not running it with user apirunner as no anyuid scc was added to the user.

$ oc adm policy add-scc-to-user anyuid system:serviceaccount:pipeline:jenkins

we ran added jenkins-sa to allow pods to run as anyuid and it did the trick.

Finally the API server is running as apirunner user and git is able to clone the repositories, as it is having entry in /etc/passwd. And we are happily able to use API end points :)