K8s Troubleshooting - "Err:28"
Error: Err:28
map[DriverName:filesystem Enclosed:map[Err:28 Op:mkdir Path:/var/lib/registry/docker/registry/v2/repositories/<project>/<repository>]]
Root cause: not enough space.
Verification: check disk space of the harbor registry pod:
$ kubectl -n HARBOR_NAMESPACE exec HARBOR_REGISTRY_POD_NAME -- df -ah | less
Solution: resize the disk size for the registry.
# Get the pod
POD=$(kubectl get pods -n HARBOR_NAMESPACE -l goharbor.io/operator-controller=registry -o name --kubeconfig=/path/to/kubeconfig)
# Set the new size
STORAGE_SIZE=400Gi
# Patch PVC
kubectl patch Persistentvolumeclaim/harbor-registry \
--kubeconfig=/path/to/kubeconfig \
-n harbor-system --type=merge \
-p '{"spec": {"resources": {"requests": {"storage": "'$STORAGE_SIZE'"}}}}'
# Wait until the storage capacity is changed
kubectl --kubeconfig=/path/to/kubeconfig -n HARBOR_NAMESPACE exec $POD -- df -ah | grep "/var/lib/registry"