feat(e2e): add e2e-test for raw block volume for device-localpv

Signed-off-by: w3aman <[email protected]>

e2e-tests/experiments/device-localpv-provisioner/device_localpv_sc.j2

@@ -30,4 +30,14 @@ allowVolumeExpansion: true
 parameters:
   fsType: "xfs"
   devname: "{{ device_name }}"
+provisioner: device.csi.openebs.io
+
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: device-sc-raw-block
+allowVolumeExpansion: true
+parameters:
+  devname: "{{ device_name }}"
 provisioner: device.csi.openebs.io

e2e-tests/experiments/functional/devicepv-raw-block-volume/README.md

@@ -0,0 +1,69 @@
+## About this experiment
+
+This experiment verifies the provision and deprovision of raw block volumes by device-localpv. There are some specialized applications that require direct access to a block device because, for example, the file system layer introduces unneeded overhead. The most common case is databases, which prefer to organize their data directly on the underlying storage. In this experiment we are not using any such application for testing, but using a simple busybox application to verify successful provisioning and deprovisioning of raw block volume.
+
+To provisione the Raw Block volume, we should create a storageclass without any fstype as Raw block volume does not have any fstype.
+
+```
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: device-sc-raw-block
+allowVolumeExpansion: true
+parameters:
+  devname: "{{ device_name }}"
+provisioner: device.csi.openebs.io
+```    
+Note: For running this experiment above storage-class should be present. This storage class will be created as a part of device-localpv provisioner experiment. If device-localpv components are not deployed using e2e-test script located at `openebs/device-localpv/e2e-tests/experiment/device-localpv-provisioiner` please make sure you create the storage class from above mentioned yaml.
+
+## Supported platforms:
+
+K8s : 1.18+
+
+OS : Ubuntu, CentOS
+
+device : 0.7, 0.8
+
+## Entry-Criteria
+
+- K8s cluster should be in healthy state including all desired nodes in ready state.
+- device-controller and node-agent daemonset pods should be in running state.
+- storage class without any fstype should be present.
+- a directory should be present on node with name `raw_block_volume`.
+
+## Steps performed
+
+- deploy the busybox application with given a devicePath.
+- verify that application pvc gets bound and application pod is in running state.
+- dump some data into raw block device and take the md5sum of data.
+- restart the application and verify the data consistency.
+- After that update the pvc with double value of previous pvc size, to validate resize support for raw block volumes.
+- when resize is successful, then dump some dummy data into application to use the resized space.
+- At last deprovision the application and check its successful deletion.
+
+## How to run
+
+- This experiment accepts the parameters in form of kubernetes job environmental variables.
+- For running this experiment of devicepv raw block volume creation, first clone openens/device-localpv[https://github.com/openebs/device-localpv] repo and then apply rbac and crds for e2e-framework.
+```
+kubectl apply -f device-localpv/e2e-tests/hack/rbac.yaml
+kubectl apply -f device-localpv/e2e-tests/hack/crds.yaml
+```
+then update the needed test specific values in run_e2e_test.yml file and create the kubernetes job.
+```
+kubectl create -f run_e2e_test.yml
+```
+All the env variables description is provided with the comments in the same file.
+After creating kubernetes job, when the job’s pod is instantiated, we can see the logs of that pod which is executing the test-case.
+
+```
+kubectl get pods -n e2e
+kubectl logs -f <devicepv-block-volume-xxxxx-xxxxx> -n e2e
+```
+To get the test-case result, get the corresponding e2e custom-resource `e2eresult` (short name: e2er ) and check its phase (Running or Completed) and result (Pass or Fail).
+
+```
+kubectl get e2er
+kubectl get e2er devicepv-raw-block-volume -n e2e --no-headers -o custom-columns=:.spec.testStatus.phase
+kubectl get e2er devicepv-raw-block-volume -n e2e --no-headers -o custom-columns=:.spec.testStatus.result
+```

e2e-tests/experiments/functional/devicepv-raw-block-volume/busybox.j2

@@ -0,0 +1,53 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: app-busybox
+  labels:
+    app: raw-block-vol
+spec:
+  selector:
+    matchLabels:
+      app: raw-block-vol
+  template:
+    metadata:
+      labels:
+        app: raw-block-vol
+    spec:
+      containers:
+      - name: app-busybox
+        imagePullPolicy: IfNotPresent
+        image: gcr.io/google-containers/busybox
+        command: ["/bin/sh"]
+        args: ["-c", "while true; do sleep 10;done"]
+        env:
+        volumeDevices:
+        - name: data-raw-block-vol
+          devicePath: /dev/sdc
+
+        volumeMounts:
+        - name: data-mount-vol
+          mountPath: /busybox
+
+      volumes:
+      - name: data-raw-block-vol
+        persistentVolumeClaim:
+          claimName: block-vol-pvc
+
+      - name: data-mount-vol
+        hostPath:
+          path: /raw_block_volume
+
+---
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: block-vol-pvc
+spec:
+  volumeMode: Block
+  storageClassName: {{ storage_class }}
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: {{ pvc_size }}

e2e-tests/experiments/functional/devicepv-raw-block-volume/run_e2e_test.yml

@@ -0,0 +1,43 @@
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  generateName: devicepv-block-volume-
+  namespace: e2e
+spec:
+  template:
+    metadata:
+      labels:
+        test: devicepv-block-volume
+    spec:
+      serviceAccountName: e2e
+      restartPolicy: Never
+
+      containers:
+      - name: ansibletest
+        image: openebs/device-localpv-e2e:ci
+        imagePullPolicy: IfNotPresent
+        env:
+          - name: ANSIBLE_STDOUT_CALLBACK
+            value: default
+
+            # This is the namespace where raw block volume consuming application will be deployed
+          - name: APP_NAMESPACE
+            value: 'block-ns'
+
+            # storage class name for raw block volume
+            # this storage class doesn't have any `fstype` parameter
+          - name: STORAGE_CLASS
+            value: 'device-sc-raw-block'
+
+            # size of the pvc for `volumeMode: Block`
+          - name: PVC_SIZE
+            value: '5Gi'
+
+            # This is the namespace where device-driver creates all its resources.
+            # By default it is `openebs` namespace
+          - name: DEVICE_OPERATOR_NAMESPACE
+            value: 'openebs'
+
+        command: ["/bin/bash"]
+        args: ["-c", "ansible-playbook ./e2e-tests/experiments/functional/devicepv-raw-block-volume/test.yml -i /etc/ansible/hosts -vv; exit 0"]

e2e-tests/experiments/functional/devicepv-raw-block-volume/test.yml

@@ -0,0 +1,170 @@
+- hosts: localhost
+  connection: local
+  gather_facts: False
+
+  vars_files:
+    - test_vars.yml
+
+  tasks:
+    - block:
+
+          ## Generating the testname for devicepv raw block volume test
+        - include_tasks: /e2e-tests/hack/create_testname.yml
+
+          ## Record SOT (start of test) in e2e result e2e-cr (e2e-custom-resource)
+        - include_tasks: /e2e-tests/hack/update_e2e_result_resource.yml
+          vars:
+            status: 'SOT'
+
+        - name: Create the namespace for block-volume of devicepv
+          shell: >
+            kubectl create ns {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Update the busybox application template with the test specific values
+          template:
+            src: busybox.j2
+            dest: busybox.yml
+
+        - name: Deploy the application using block volume pvc
+          shell: >
+            kubectl create -f busybox.yml -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Check if the block volume PVC is bound
+          shell: >
+            kubectl get pvc block-vol-pvc -n {{ app_ns }} 
+            --no-headers -o custom-columns=:.status.phase
+          args:
+            executable: /bin/bash
+          register: pvc_status
+          until: "'Bound' in pvc_status.stdout"
+          delay: 5
+          retries: 30
+
+        - name: Get the device volume name
+          shell: kubectl get pvc block-vol-pvc -n {{ app_ns }} -o custom-columns=:.spec.volumeName
+          args:
+            executable: /bin/bash
+          register: devicevol_name
+
+        - name: Get the application pod name
+          shell: >
+            kubectl get pod -n {{ app_ns }} -l app=raw-block-vol
+            --no-headers -o custom-columns=:.metadata.name
+          args:
+            executable: /bin/bash
+          register: app_pod_name
+
+        - name: Check if the application pod is in running state
+          shell: >
+            kubectl get pod {{ app_pod_name.stdout }} -n {{ app_ns }}
+            --no-headers -o custom-columns=:.status.phase
+          register: pod_status
+          until: "'Running' in pod_status.stdout"
+          delay: 5
+          retries: 50
+
+        - name: Create some test data into the raw block device and take the md5sum of data
+          shell: >
+            kubectl exec -ti {{ app_pod_name.stdout }} -n {{ app_ns }}
+            -- sh -c "{{ item }}"
+          args:
+            executable: /bin/bash
+          register: result
+          failed_when: "result.rc != 0"
+          with_items:
+            - "dd if=/dev/urandom of=/dev/sdc bs=4k count=262144"
+            - "md5sum /dev/sdc > /busybox/pre-md5"
+
+        - name: Restart the busybox application
+          shell: kubectl delete pod {{ app_pod_name.stdout }} -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Get the application pod name after restart
+          shell: >
+            kubectl get pod -n {{ app_ns }} -l app=raw-block-vol
+            --no-headers -o custom-columns=:.metadata.name 
+          args:
+            executable: /bin/bash
+          register: new_pod_name
+
+        - name: Check if the application pod is in running state
+          shell: >
+            kubectl get pod {{ new_pod_name.stdout }} -n {{ app_ns }}
+            --no-headers -o custom-columns=:.status.phase
+          register: pod_status
+          until: "'Running' in pod_status.stdout"
+          delay: 5
+          retries: 50
+
+        - name: Again take the md5sum of the data after restarting the application pod 
+          shell: >
+            kubectl exec -ti {{ new_pod_name.stdout }} -n {{ app_ns }}
+            -- sh -c "md5sum /dev/sdc > /busybox/post-md5"
+          args:
+            executable: /bin/bash
+          register: result
+          failed_when: "result.rc != 0"
+
+        - name: Verify whether data is consistence after restarting the application pod
+          shell: >
+            kubectl exec -ti {{ new_pod_name.stdout }} -n {{ app_ns }}
+            -- sh -c "diff /busybox/pre-md5 /busybox/post-md5"
+          args: 
+            executable: /bin/bash
+          register: result 
+          failed_when: "result.rc != 0 or result.stdout != ''"
+
+        - name: Deprovision the busybox application
+          shell: kubectl delete -f busybox.yml -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - name: Verify that busybox application is successfully deprovisioned
+          shell: kubectl get pods -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+          register: app_pod_status
+          until: "'new_app_pod.stdout' not in app_pod_status.stdout"
+          delay: 5
+          retries:
+
+        - name: Verify that pvc is deleted successfully
+          shell: kubectl get pvc -n {{ app_ns }}
+          args:
+            executable: /bin/bash
+          register: pvc_status
+          until: "'block-vol-pvc' not in pvc_status.stdout"
+          delay: 3
+          retries: 30
+
+        - name: Verify the device volume is deleted successfully
+          shell: kubectl get devicevol -n {{ device_operator_ns }}
+          args:
+            executable: /bin/bash
+          register: devicevol_list
+          until: "'devicevol_name.stdout' not in devicevol_list.stdout"
+          delay: 3
+          retries: 30
+
+        - name: Delete the application namespace
+          shell: kubectl delete ns {{ app_ns }}
+          args:
+            executable: /bin/bash
+
+        - set_fact:
+            flag: "Pass"
+
+      rescue:
+      - set_fact:
+          flag: "Fail"
+
+      always:
+          ## RECORD END-OF-TEST IN e2e RESULT CR
+        - include_tasks: /e2e-tests/hack/update_e2e_result_resource.yml
+          vars:
+            status: 'EOT'

e2e-tests/experiments/functional/devicepv-raw-block-volume/test_vars.yml

@@ -0,0 +1,9 @@
+test_name: devicepv-raw-block-volume
+
+app_ns: "{{ lookup('env','APP_NAMESPACE') }}"
+
+storage_class: "{{ lookup('env','STORAGE_CLASS') }}"
+
+pvc_size: "{{ lookup('env','PVC_SIZE') }}"
+
+device_operator_ns: "{{ lookup('env','DEVICE_OPERATOR_NAMESPACE') }}"