Skip to content

Hosted Control Plane

https://docs.google.com/document/d/1EUaKD_0JGPPPAD7rAshfXUVfAOzTqmU5qx_60Hl_NE0/edit

https://github.com/gqlo/blogs/blob/main/hosted-control-plane-with-the-kubevirt-provider.md

Platform

Platform - KubeVirt

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
export PULL_SECRET=${HOME}/redhat-pullsecret-rh-ee-rbohne.json
export KUBEVIRT_CLUSTER_NAME=oat23
export TRUSTED_BUNDLE=${HOME}/Devel/gitlab.consulting.redhat.com/coe-lab/certificates/ca-bundle-v1.pem

hcp create cluster \
kubevirt \
  --name $KUBEVIRT_CLUSTER_NAME \
  --namespace rbohne-hcp \
  --node-pool-replicas=2 \
  --memory '16Gi' \
  --cores '8' \
  --generate-ssh \
  --root-volume-size 120 \
  --root-volume-storage-class 'coe-netapp-nas' \
  --pull-secret $PULL_SECRET \
  --etcd-storage-class ocs-storagecluster-ceph-rbd \
  --control-plane-availability-policy HighlyAvailable \
  --additional-trust-bundle $TRUSTED_BUNDLE \
  --auto-repair \
  --release-image=quay.io/openshift-release-dev/ocp-release:4.14.1-x86_64
  # --render
  # Optional - add --render to show yaml

Export kubeconfig

1
2
3
hcp create kubeconfig \
  --name $KUBEVIRT_CLUSTER_NAME \
  --namespace rbohne-hcp | sed "s/admin/$KUBEVIRT_CLUSTER_NAME/" > ~/.kube/clusters/${KUBEVIRT_CLUSTER_NAME}

Platform - None / BareMetal

Not tested, for a long time:

1
2
3
4
5
6
7
8
9
hcp create cluster \
none \
  --expose-through-load-balancer \
  --name $KUBEVIRT_CLUSTER_NAME \
  --control-plane-availability-policy HighlyAvailable \
  --etcd-storage-class ocs-storagecluster-ceph-rbd \
  --release-image=quay.io/openshift-release-dev/ocp-release:4.12.1-x86_64 \
  --pull-secret $PULL_SECRET
  # --render

Loadbalacner for ingress

Ingress is running on physical nodes, you have to provide an external load balancer.

Here a container solution bases on openshift-4-loadbalancer

  • Create new project
  • Create service account privileged : oc create sa privileged
  • Grant scc privileged to service account privileged
  • Download and edit line 35 & 37 with your BareMetal endpoints
1
2
3
curl -L -O  https://examples.openshift.pub/cluster-installation/hosted-control-plane/hosted-control-plane/openshift-4-loadbalancer-deployment.yaml
$EDITOR openshift-4-loadbalancer-deployment.yaml
oc apply -f openshift-4-loadbalancer-deployment.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: openshift-4-loadbalancer
    app.kubernetes.io/component: openshift-4-loadbalancer
    app.kubernetes.io/instance: openshift-4-loadbalancer
    app.kubernetes.io/name: openshift-4-loadbalancer
    app.kubernetes.io/part-of: openshift-4-loadbalancer
    app.openshift.io/runtime: haproxy
  name: openshift-4-loadbalancer
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  selector:
    matchLabels:
      app: openshift-4-loadbalancer
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: openshift-4-loadbalancer
        deployment: openshift-4-loadbalancer
    spec:
      containers:
        - image: quay.io/redhat-emea-ssa-team/openshift-4-loadbalancer:latest
          imagePullPolicy: Always
          name: openshift-4-loadbalancer
          env:
            - name: INGRESS_HTTP
              value: "ucs56-0=10.32.96.56:80,ucs57-0=10.32.96.57:80"
            - name: INGRESS_HTTPS
              value: "ucs56-0=10.32.96.56:443,ucs57-0=10.32.96.57:443"
          ports:
            - containerPort: 22623
              protocol: TCP
            - containerPort: 443
              protocol: TCP
            - containerPort: 6443
              protocol: TCP
            - containerPort: 80
              protocol: TCP
            - containerPort: 1984
              protocol: TCP
          resources: {}
          securityContext:
            privileged: true
            runAsUser: 0
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: privileged
      serviceAccountName: privileged
      terminationGracePeriodSeconds: 30
  • Apply service type load balancer
1
oc apply -f  https://examples.openshift.pub/cluster-installation/hosted-control-plane/hosted-control-plane/openshift-4-loadbalancer-service.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: v1
kind: Service
metadata:
  name: openshift-4-loadbalancer
spec:
  type: LoadBalancer
  selector:
    app: openshift-4-loadbalancer
    deployment: openshift-4-loadbalancer
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
      name: http
    - protocol: TCP
      port: 443
      targetPort: 443
      name: https
    - protocol: TCP
      port: 1984
      targetPort: 1984
      name: stats

Trouble shooting

https://hypershift-docs.netlify.app/how-to/troubleshooting/

1
2
3
4
5
6
7
8
9
export KUBEVIRT_CLUSTER_NAME=lenggries3
export CLUSTERNS="rbohne-hcp"

mkdir clusterDump-${CLUSTERNS}-${KUBEVIRT_CLUSTER_NAME}
hcp dump cluster \
    --name ${KUBEVIRT_CLUSTER_NAME} \
    --namespace ${CLUSTERNS} \
    --dump-guest-cluster \
    --artifact-dir clusterDump-${CLUSTERNS}-${KUBEVIRT_CLUSTER_NAME}
2023-12-17 2023-10-20 Contributors: Robert Bohne