Skip to content

OpenShift Virtualization (CNV/KubeVirt)

Example deployments

Tiny RHEL 9 VM with pod bridge network 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
  labels:
    app: rhel9-pod-bridge
    kubevirt.io/dynamic-credentials-support: "true"
  name: rhel9-pod-bridge
spec:
  dataVolumeTemplates:
    - apiVersion: cdi.kubevirt.io/v1beta1
      kind: DataVolume
      metadata:
        name: rhel9-pod-bridge
      spec:
        sourceRef:
          kind: DataSource
          name: rhel9
          namespace: openshift-virtualization-os-images
        storage:
          accessModes:
            - ReadWriteMany
          storageClassName: ocs-storagecluster-ceph-rbd-virtualization
          resources:
            requests:
              storage: 30Gi
  running: false
  template:
    metadata:
      annotations:
        vm.kubevirt.io/flavor: tiny
        vm.kubevirt.io/os: rhel9
        vm.kubevirt.io/workload: server
        kubevirt.io/allow-pod-bridge-network-live-migration: ""
      labels:
        kubevirt.io/domain: rhel9-pod-bridge
        kubevirt.io/size: tiny
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 1
          threads: 1
        devices:
          disks:
            - disk:
                bus: virtio
              name: rootdisk
            - disk:
                bus: virtio
              name: cloudinitdisk
          interfaces:
            - bridge: {}
              name: default
        machine:
          type: pc-q35-rhel9.2.0
        memory:
          guest: 1.5Gi
      networks:
        - name: default
          pod: {}
      terminationGracePeriodSeconds: 180
      volumes:
        - dataVolume:
            name: rhel9-pod-bridge
          name: rootdisk
        - cloudInitNoCloud:
            userData: |-
              #cloud-config
              user: cloud-user
              password: redhat
              chpasswd: { expire: False }
          name: cloudinitdisk

1
oc apply -f https://examples.openshift.pub/kubevirt/example/tiny-rhel-pod-bridge.yaml
Red Hat CoreOS with ignition & pod bridge network 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
  labels:
    app: rhcos-pod-bridge
    kubevirt.io/dynamic-credentials-support: "true"
  name: rhcos-pod-bridge
spec:
  dataVolumeTemplates:
    - apiVersion: cdi.kubevirt.io/v1beta1
      kind: DataVolume
      metadata:
        name: rhcos-pod-bridge
      spec:
        source:
          registry:
            pullMethod: node
            # openshift-install coreos print-stream-json | jq '.architectures.x86_64.images.kubevirt'
            url: docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ab118238b01765f103fe0739c0cd48ba10e745f25d5d1da202faf8c08b57fb58
        storage:
          accessModes:
            - ReadWriteMany
          storageClassName: ocs-storagecluster-ceph-rbd-virtualization
          resources:
            requests:
              storage: 30Gi
  running: false
  template:
    metadata:
      annotations:
        vm.kubevirt.io/flavor: tiny
        vm.kubevirt.io/os: rhcos
        vm.kubevirt.io/workload: server
        kubevirt.io/allow-pod-bridge-network-live-migration: ""
      labels:
        kubevirt.io/domain: rhcos-pod-bridge
        kubevirt.io/size: large
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 2
          threads: 1
        devices:
          disks:
            - disk:
                bus: virtio
              name: rootdisk
            - disk:
                bus: virtio
              name: cloudinitdisk
          interfaces:
            - bridge: {}
              name: default
        machine:
          type: pc-q35-rhel9.2.0
        memory:
          guest: 8Gi
      networks:
        - name: default
          pod: {}
      terminationGracePeriodSeconds: 180
      volumes:
        - dataVolume:
            name: rhcos-pod-bridge
          name: rootdisk
        - cloudInitConfigDrive:
            # Password hash
            #   podman run -ti --rm quay.io/coreos/mkpasswd --method=yescrypt
            # Password: redhat
            userData: |-
              {
                "ignition": {
                  "version": "3.2.0"
                },
                "passwd": {
                  "users": [
                    {
                      "name": "core",
                      "passwordHash": "$y$j9T$15cuONdoH5AKB62c9qTtD.$oOf4GqrwEnNzT7WuEFvkDuSOyv2xIx/z4EXzbQivdO0",
                      "sshAuthorizedKeys": [
                        "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAOfl+764UFbDkkxpsQYjET7ZAWoVApSf4I64L1KImoc rbohne@redhat.com"
                      ]
                    }
                  ]
                }
              }
          name: cloudinitdisk

1
oc apply -f https://examples.openshift.pub/kubevirt/example/rhcos-pod-bridge.yaml
Boot from ISO 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
  labels:
    app: beryllium
  name: beryllium
  namespace: demo-cluster-disco
spec:
  dataVolumeTemplates:
    - metadata:
        name: beryllium-root
      spec:
        storage:
          accessModes:
            - ReadWriteMany
          storageClassName: coe-netapp-nas
          resources:
            requests:
              storage: 80Gi
        source:
          blank: {}
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/domain: beryllium
    spec:
      volumes:
        - name: cdrom
          persistentVolumeClaim:
            claimName: beryllium-1-i386-hybrid
        - name: root
          dataVolume:
            name: beryllium-root
      networks:
        - name: coe
          multus:
            networkName: coe-bridge
        - name: disco
          multus:
            networkName: coe-br-vlan-69
      domain:
        cpu:
          cores: 4
        memory:
          guest: 8Gi
        resources:
          requests:
            memory: 8Gi
        devices:
          disks:
            - name: root
              bootOrder: 1
              disk:
                bus: virtio
            - name: cdrom
              bootOrder: 2
              cdrom:
                bus: sata
          interfaces:
            - bridge: {}
              # macAddress: 02:d8:6d:00:00:12
              model: virtio
              name: coe
            - bridge: {}
              # macAddress: 02:d8:6d:00:00:13
              model: virtio
              name: disco

1
oc apply -f https://examples.openshift.pub/kubevirt/example/boot-from-iso.yaml

Containerized Data Importer (CDI) / DataVolume

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: registry-image-datavolume
spec:
  pvc:
    accessModes:
    - ReadWriteMany
    resources:
      requests:
        storage: 5Gi
  source:
    registry:
      url: docker://image-registry.openshift-image-registry.svc:5000/cnv-demo/build-vm-image-container:latest
      certConfigMap: "tls-certs"

Source: cdi-examples

OpenShift Virtualization & Container Storage

Recommended storage settings:

1
2
3
4
5
6
7
8
$ oc edit cm kubevirt-storage-class-defaults -n openshift-cnv

accessMode: ReadWriteMany
ocs-storagecluster-ceph-rbd.accessMode: ReadWriteMany
ocs-storagecluster-ceph-rbd.volumeMode: Block
ocs-storagecluster-cephfs.accessMode: ReadWriteMany
ocs-storagecluster-cephfs.volumeMode: Filesystem
volumeMode: Block

Build container image with OS disk

1
2
3
oc new-build --name cirros \
    --build-arg image_url=http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img \
    https://github.com/openshift-examples/cnv-container-disk-build.git

Local IIS build in my lab

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
qemu-img convert -f raw -O qcow2 disk.img iis.qcow2

cat - > Dockerfile <<EOF
FROM scratch
LABEL maintainer="Robert Bohne <robert.bohne@redhat.com>"
ADD iis.qcow2 /disk/rhel.qcow2
EOF

oc create is iis -n cnv

export REGISTRY=$(oc get route default-route -n openshift-image-registry -o jsonpath='{.spec.host}')
export REGISTRY_TOKEN=$(oc whoami -t)
podman login -u $(oc whoami) -p $REGISTRY_TOKEN --tls-verify=false $HOST

podman build -t ${REGISTRY}/cnv/iis:latest .
podman push ${REGISTRY}/cnv/iis:latest

# Deploy template
oc apply -f https://raw.githubusercontent.com/openshift-examples/web/master/content/kubevirt/iis-template.yaml

Resources and useful articles

2024-01-04 2019-10-08 Contributors: Robert Bohne