Insufficient space on datastore

[rpayne1322]

Hi,

I've just started at this shop and I am completely new to VMware, and, Blade configurations. We are running HS22's Blades and have a DS3200 for our SAS storage. I've installed ESX 4.0 on the blades. We added the two 73 GB of internal drives (Mirrored) to the blades for ESX install to boot from because at this time it is not supported on the DS3200.

I'm trying to get the guest operating system installed (Windows server 2003 ENT), but when I create the new Virtual machine and map the LUN we are planning to use for the OS / SQL disk, it says I have available 135.5 GB. I'm not sure how much overheard VMware needs for its files though on the datastore.. I tried to make the datastore size 132 GB, but the VM would not start with the error "Insufficient disk space on datastore" I tried 130 (leaving 5.45 GB free), and had the same problem again. Is there a rule of thumb for how much space to leave on the LUN for VMware overhead? I read on a post here 20%? If that is the case we will probably have to order more drives in order to meet the space requirement configuration for the project we are working on. On this particular LUN they want two 36 GB Partitions and one 73 GB partition, and we are mirroring two 146 GB drives, after the Array creation (RAID 1) of the two 146 GB drives we only have 136.23, then down to 135.5 after VMware provisioned 500 some odd MB when adding the storage device to the System. Unfortunately, the project already has the 12 drive slots filled for the creation of the three VM's, so we would have to buy another DS3200 to add-on.

As I said, I'm completely new at this though..Maybe I'm creating the VM wrong? We have some training material on the way.. but this project is behind already. During the creation process, for the location of the datastore I'm using the LUN on the DS3200, I enter Datastore size of 130 GB (Available space says 135.5), and I click the radial for Support clustering features (FT). I was under the impression that this was needed in order for VMotion to be able to migrate VM's as needed? I do not have our vCenter Server running yet, waiting on some hardware to come in a couple days... not sure if having that up would help some of this setup or not? When I mapped the storage to the system in vSphere client I used the option of 256 GB max file size, and 1 MB block size, and leave the radial for maximize capacity clicked. None of our LUNs are over 146 GB.

When creating the VM, can I use the Internal disk to the blade to store the VM files, and then just map the SAS storage to the VM and install windows on those? But if I do that... will vCenter server be able to migrate the VM off of that host onto another blade? If so then this really isn't an issue.

I'm a little worried even if I pick 125 or 120 GB size for the datastore and it works to get the system up and running, is there really enough space for VMware files even though the system IS running now. I ask this because while I was waiting for some more hardware to come in on the blades, I did a test install and VM setup, using Thin format that time on the datastore. After I had the VM running.. i was testing some file copies to the system disk and as I got it down to around 4 GB free, the VM would crash.. I'd hate to run into that in production.

Thank you for any help you can pass along. I've learned a lot in the past few days, but I have a long ways to go..

-RP

[byoung111]

How much memory to you have assigned to the VM? You need the same amount of free disk space as you have memory allocated as when you power on the VM, ESX creates a swapfile that is equel to the amount of memory assigned to the VM.

[rpayne1322]

We have 36 GB on the Blade, during this VM creation I picked 16 GB. Thank you for that tip! I remember reading about the swap file but I don't recall it mentioning the size requirements based on memory, but I might of missed that. Starting to sound like our IBM resellers really didn't look over the project specs when selling us these drives.. I'm not looking forward to requesting another DS3200.

-RP

[byoung111]

Something to keep in mind. In the VM world, when you start dealing with many VM's, less is usually better. What I mean is only give the VM what it needs, especially where CPU is concerned. For example if you have a ESX server with 8 CPU's, and you have 4 VM's with 4 vCPU's each, you will see significant wait times with cpu usage. This is because when a VM make's a call for CPU time, each vCPU needs to attach to an HEC thread on the physical CPU. This means that 2 VM's will have all 8 CPU's tied up. If you drop the vCPU's down to 2 on each VM, there will always be free threads available to the physical CPU's. It not quite as bad to overcommit memory, but you'll want to watch that too. If you see a lot of ballooning (this is the process of the ESX server reclaiming memory from the VM) then you'll start seeing performance problems.

Hope that helps a little.

Oh one more tip about LUN sizing. I generally want 15GB of free space on my lun no matter how many VM's I'm running. Now mind you this is 15GB free after the VM's are turned on. I generally size like this.

Size of all my VMDK files (virtual disks) + size of the memory assigned to the VM + 1GB for general files associated + 15GB = Lun size

Now I don't add 15GB for each VM. I just add 15GB to whatever the total of all the VM's is. Now the size you pick also depends on if your going to use snapshots, and for how long, and for how many VM's. If you snapshot a VM you need at least enough free space on your lun that is equel to the amount of memory the VM you are snapshoting (this is on top of the normal virtual swap file that is created when the VM is turned on) + free space to hold all of the data changes.

I'm sure I'll think of more!

[rpayne1322]

Very helpful information, thank you.

To clarify - Does VMware classify a 'CPU' as one core on the processor? We have two quad core processors on each blade, so in essence I have 8 CPU's per blade according to VMware?

Initially I was using 4 vCPU's per VM but I'll drop that down to two.. and lower the memory quite a bit, probably 4 GB

[byoung111]

Very helpful information, thank you.

To clarify - Does VMware classify a 'CPU' as one core on the processor? We have two quad core processors on each blade, so in essence I have 8 CPU's per blade according to VMware?

Initially I was using 4 vCPU's per VM but I'll drop that down to two.. and lower the memory quite a bit, probably 4 GB

That depends on if you talking about licensing, or vCPU to pCPU. If your talking about licensing it's per socket, not per CPU. As far as just looking at the processors, ESX will see two quad core processors as 8 CPU's for use to the VM's.

[rpayne1322]

Great, thanks. That one had me worried for a minute, for now we are only licensed for 6 from VMware, but we are covered for our 3 Blades then till we add on others.

[byoung111]

As I said, I'm completely new at this though..Maybe I'm creating the VM wrong? We have some training material on the way.. but this project is behind already. During the creation process, for the location of the datastore I'm using the LUN on the DS3200, I enter Datastore size of 130 GB (Available space says 135.5), and I click the radial for Support clustering features (FT). I was under the impression that this was needed in order for VMotion to be able to migrate VM's as needed? I do not have our vCenter Server running yet, waiting on some hardware to come in a couple days... not sure if having that up would help some of this setup or not? When I mapped the storage to the system in vSphere client I used the option of 256 GB max file size, and 1 MB block size, and leave the radial for maximize capacity clicked. None of our LUNs are over 146 GB.

The FT option is not needed for vMotion. FT allows you to have a live secondary VM in case your production VM goes down. You do however have to have the Enterprise Plus license, shared storage, GB connections, and supported processors for that to work. When you create your VMFS volume, the 256 Max file size is not relevant to your lun. This means that the largest single VMDK file or Virtual disk you can have is 256GB.

[rpayne1322]

I'll probably have to look into the Licensing issue yet, but I do know that management wants the ability for migration incase of host failure.

So, I can choose Thin then on creation and not worry about FT it looks like, as we will not be having a live secondary machine. It could always be converted to thick later if need be..

I guess it might boil down to the one question though, if I make the VM datastore on the Internal drives, will VM migration to other hosts be available if all drives are on the shared storage? My guess is no.. it looks like trying to install windows on the Shared LUN still has to write some data to the 8 GB datastore I created on the internal drives to the Blade.

[byoung111]

I have never used thin provisioned disks, so I'm not sure about the process of converting to thick later. I'll have to do some reading on that.

It sounds like your VM might have a virtual disk on the local datastore. You should be able to right click you local datastore and click on browse datastore. This will let you see if you have any disks there. Your other option is to right click on your VM and click on Edit settings. Click on your hard disks and look in the upper right hand corner. This will tell you where your disks are located. Depending on your license, if you do have a disk on a local datastore, you should be able to migrate the disk to your shared storage.

Without shared storage you can still vMotion a VM to another server, however it has to be powered off. You can only live vMotion a VM to another server if all of the physical servers can see the same storage.

[rpayne1322]

I figured that was going to be the Key, having all of the physical servers be able to see the same storage. I'm scrapping the idea of using the internals for anything other than booting ESX and we'll have to just make do with what we have for space now, and add more later!

Everything posted has been very helpful, thank you for helping out this novice. I was a little overwhelmed at first but I'm looking forward to learning more about this setup and contributing here in the future!

[KiloMike]

RP-

I'm new also, but what I think you are doing is maxing out your storage area. I have found that if you create a new VM with a Win2K3 OS, you need about a 20GB drive size. I carved out about 300GB on our network storage and created a VMCTR folder which I point all my VM's to. eadch time I create a new VM, i start off with a 20GB drive and if I need more, I can simply add later. I would not use your blade for VM storage as I think it is better you have your files elsewhere (a location you can also backup).

I hope this answers your question.

K

[rpayne1322]

Thanks K,

I didn't think it would be a good idea to have the datastore on the Internals.. I'd like everything on the ds3200, but the specs for this project pretty much have that whole device used up :smileysad:

I -think- what I will do is use Thin provisioning for now and leave plenty of overheard on each datastore, and get them VM's up and running for testing. The spec called for 36 GB for the Win OS and SQL install, but I could lower that one by a bit. We had planned on another Ds3200 eventually for other projects.. I could always carve more room out of that to add to these VM's later I would imagine.

Thanks to you both for your quick and helpful responses.

-RP