Skip to content

Openshift - Management

Commands

Drain node (Remove pod from a node):

oc adm drain {node} --delete-local-data=true --grace-period=1

Cordon node (Unschedule pod from a node):

oc adm cordon {node}

Uncordon node (Reschedule pod from a node):

oc adm uncordon {node}

Run a console or command on a node:

oc debug node/{node} (chroot /host {command})
# Ex: oc debug node/worker-1
#     oc debug node/worker-1 chroot /host df -ThP

Procedures

Get system:admin kubeconfigs on OCP 4.6+

In all the masters, there is a /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/ folder that contains different system:admin kubeconfigs. They only differ in which API endpoint they target to:

The kubeconfigs are:

  • lb-ext.kubeconfig: It points to external API load balancer (api.{cluster-domain})
  • lb-int.kubeconfig: It points to internal API load balancer (api-int.{cluster-domain})
  • localhost.kubeconfig: It points to localhost. This one is useful in case of problems with the load balancers, for example.
  • localhost-recovery.kubeconfig: It points to localhost but sends the localhost-recovery SNI name, which in turns causes the special server certificates used by the auto-recovery logic to be used. This one may be useful if kube-apiserver serving certificates are expired and the auto-recovery logic has failed.

So if you can ssh to a master and become root, you can use one of these kubeconfigs to access the cluster like in this example (for lb-int.kubeconfig):

export KUBECONFIG=/etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/lb-int.kubeconfig

In addition, it is also possible to download all these kubeconfigs to your own machine by running this:

oc -n openshift-kube-apiserver extract secret/node-kubeconfigs

Source: https://access.redhat.com/solutions/6112601

Api browsing

Api Browsing is useful when troubleshooting api endpoints.

Supported methods

Openshift APi supports 2 auth methods : * Token * X509 Certificates

kubectl/oc mode Raw

using the RAW mode of kubectl remains the most convenient way

kubectl get --raw /apis
kubectl get --raw </API/ENDPOINT>

Curl API with Token

Retrieve a token and Curl Openshift api with token method 

# choose One export
export TOKEN=$(oc sa get-token -n openshift-apiserver default)
export TOKEN=$(oc whoami -t)

curl -k \
    -H "Authorization: Bearer $TOKEN" \
    -H 'Accept: application/json' \
    https://<API-ENDPOINT>:6443/oapi/v1/projects

Remember you can proxify your requests through kubectl or oc

oc proxy --port 8080    # also works with kubectl
curl http:localhost8080/<API_ENDPOINT>

Curl API with x509 certs ( Advanced )

To manually browse apiserver with curl extract the client.crt and client.key from your Admin Kubeconfig

export MYKUBECFG=lb-int.kubeconfig      # this is a kubeconfig with 'system:admin' Account . It contains X509 certs

curl --request GET \
     --url     https://<API-ENDPOINT>:6443/api/v1/ \
     --cert    <(cat $MYKUBECFG |yq '.users[0].user.client-certificate-data' |base64 -d) \
     --key     <(cat $MYKUBECFG |yq '.users[0].user.client-key-data' |base64 -d) \
     --header  'Content-Type: application/json' \
     --verbose

Sauce : * https://medium.com/@brucifi/working-with-tls-client-certificates-f67437a9aeb9 * https://miminar.fedorapeople.org/_preview/openshift-enterprise/registry-redeploy/rest_api/index.html

Back to top