How to format and mount attached disk to a VM deployed by a cloud template using cloudConfig

[antonioaraujo]

Hi All,

I hope you are fine.

A cloud template requests a user to enter a number of disks with size and label that should be format and mounted as shown below:

  disks:
    type: array
    title: Enter disks
    description: Enter disks (optional and maximum 2)
    minItems: 0
    maxItems: 2
    items:
      type: object
      properties:
        size:
          type: integer
          title: Size
          default: 10
          minimum: 10
          maximum: 300
          description: Size in GB
        label:
          type: string
          title: Label
          maxLength: 25
          populateRequiredForNestedProperties: true

Then, cloudConfig section includes some commands in runcmd to format and mount each disk using the label entered by user as shown below:

      cloudConfig: | 
        #cloud-config
        runcmd:
          # Loop through each disk, assuming /dev/sdb onwards for the external disks
          - for i in $(seq 1 ${length(input.disks)}); do
              dev="/dev/sd$(echo $((97 + $i)) | awk '{printf "%c", $1}')"; 
              mkfs.xfs ${dev}; 
              mkdir -p /${input.disks[$i].label}; 
              mount ${dev} /${input.disks[$i].label};
              echo "${dev} /${input.disks[$i].label} xfs defaults 0 0" >> /etc/fstab;
            done

cloudConfig fails with the following message:

/var/lib/cloud/instance/scripts/runcmd: line 24: /${input.disks[$I].label}: bad substitution

The question is:

Is it possible to use variable substitution within an input property to get the label in:  ${input.disks[$i].label} using $i variable?

I appreciate any comment on this.

Best regards

Antonio

[LuluTchab]

First, variable substitution is working inside CloudConfig but in your case, only for :

${length(input.disks)}

for the code which is inside the loop, just remember that when your CloudConfig content is "created" (when submitting the request form), substitution will be done only for what exists in the inputs. Here, if you use :

${input.disks[$i].label}

the value for input.disks is existing in the input list, but you're accessing it through a loop, in a block of script that will be executed inside the VM and at this time, input.disks won't exists anymore.

What you need is to have the label list information when you execute the script inside the VM. I don't have a working possibility to offer you but only a suggestion. 
First, have a look at documentation about available functions in a CloudTemplate (maybe map_by or map_to_object could be useful).
Then, try to find how to "extract" the label information for each disk, maybe with:

map_to_object(resource.Disk[*].label, "label")

And you can for example use this to create a file inside the VM (using write_files possibility in CloudConfig) with the list of labels. 
At the end, the script you put inside runcmd will only need to read the file with the label list and loop through it to do what's needed.

And when everything has been created correctly for the disks, just add a rm command to the runcmd list to remove the file containing label list.

One last thing. To avoid any issue when mounting the disks inside the VM, I suggest you to add a RegEx for the value that user will enter in the request form for the disks labels, to ensure there is no space or special characters (ie: allow only [a-z0-9]+)

[LuluTchab]

Did some tests and found a solution. In my case, I have the following to add disks to the VM while requesting it:

disks:
    type: array
    title: Add disk
    description: Add new disks
    minItems: 0
    maxItems: 12
    items:
      type: object
      properties:
        name:
          type: string
          title: Volume name
          maxLength: 20
          minLength: 1
        size:
          type: integer
          title: Size in GB
          maxSize: 2048
          minSize: 1

Disks are specified like this:

resources:
  vm0Disk:
    type: Cloud.vSphere.Disk
    allocatePerInstance: true
    properties:
      count: ${length(input.disks)}
      capacityGb: ${input.disks[count.index].size}
      name: ${input.disks[count.index].name}
      provisioningType: thin

Then, in the CloudConfig part, I used files (with write_files) to store information (disk names and script to do the job) and it look like this (I've mounted the disks in /mnt)

      cloudConfig: |
        write_files:
          - path: /root/disk-names
            content: | 
              ${to_json(map_to_object(resource.vm0Disk[*].name, "name"))}
          - path: /root/mount-disks.sh
            content: |
              #!/bin/bash
              diskIndex=1;
              for row in $(cat /root/disk-names | jq -c '.[]')
              do
                diskName=$(echo $row | jq -r '.name')
                dev="/dev/sd$(echo $((97 + $diskIndex)) | awk '{printf "%c", $1}')"
                mkfs.xfs ${dev}
                mkdir -p /mnt/${diskName}
                mount ${dev} /mnt/${diskName}
                echo "${dev} /mnt/${diskName} xfs defaults 0 0" >> /etc/fstab
                diskIndex=$((diskIndex+1))
              done
              systemctl daemon-reload #added for Ubuntu, on which I did my tests

And finally, I've added what's needed to the runcmd section to execute the script and do some cleaning.

runcmd:
  - chmod u+x /root/mount-disks.sh; /root/mount-disks.sh; rm /root/mount-disks.sh

That's all. You may be able to put the script inside the runcmd command if you need

[antonioaraujo]

Dear @LuluTchab,

I hope you are doing fine.

Thank you very much for your two comments on this topic!

I have tested the latest and it formats and mounts partitions using the information entered by the user. However, there is one detail I observed: The order in which the devices were assigned does not seem to be correct.

User inputs are listed below:

disk-names file content is listed below:

File system disk space usage is listed below:

So, /dev/sdb is using 10gb and mounted in /disk13gb, and /dev/sdc is using 13gb and mounted in /disk10gb.

I am checking how to fix this detail.

Again, Thank you very much for your comments! 

Best regards

Antonio

[LuluTchab]

You're welcome 😉

Now that you figured out the "logic" behind the process, I'm sure you'll be able to find out how to fix the "order" issue 😎