Local Disk VMs vs. SAN for small business

[virtualeye]

Hey All.

I just bought VMware essentials and am wondering what cash strapped businesses (such as my own) do to start down the virtualization path. Having a budget of $6K max for hardware I am choosing between a relative powerhouse server (option #1 below) vs. a low end SAN and diskless server (option #2).

Background: I'm currently colocating 2X 1U Dell machines (old at this point, SC 1425 & 1435), bare metal OS (Linux) -- do about 50GB bandwidth per month for website traffic, basically load is minimal. Switching to VMware I am looking to run current web server in one VM, break out MySQL into another, and run development VMs for Java/Grails, Ruby on Rails, and some flavor of Windows Server.

My options are:

1) Dell PowerEdge 2970 -- 2X 6-core CPU 6MB; 8X 73GB 15K SCSI in RAID 10; 16 GB RAM; 2X Intel Pro Gigabit NICs

2)

Dell PowerEdge R310 -- 2X Intel CPU 12MB; 1X 160GB SATA (Dell forces that on you); 16GB RAM; Dual Port Broadcom Gigabit TOE/iSCSI NIC

+

SANS DIGITAL EliteRAID ER104I+ 1U 4 Bay SATA to iSCSI 2x GbE

+

4X WD 300GB 10K drives in RAID 10

From what I have read the real benefits of SAN-based setups are gained when using HA + VMotion (read: non-Essentials licensing) coupled with a cluster of diskless machines accessing a higher end SAN usually with a large number of disks.

Anyway, 2X R310 + low end SAN is an interesting proposition (I love the idea of diskless systems attaching to a local data repository), but relying on a low end SAN device, not so sure about that.

I am leaning toward the PE 2970 and keeping SC 1435 as backup machine (not an HA solution, but HA license $$ are not available...)

Thoughts? Would be great if data centers provided fiber SAN targets for colo customers, would take a massive expense out of the equation!

[virtualeye]

Not many small businesses around it appears, or at least ones with a limited budget -- I assume that everyone using ESX(i) is running off of SAN with diskless servers; in other words, an HA solution.

Low end SAN solution that is within my means won't really give me the HA benefits as licensing is too pricey for me at this point, not to mention adding a point of failure that local disk SCSI drives in RAID 10 will prevent (provided the RAID is kept track of via monitoring/alert tools).

Going local disk is an LA solution, but so is the current bare metal setup, no automatic fail over if production goes down.

Curious though, with $6K hardware budget and 3 to 4U rackspace, what is the best SAN + diskless server solution one could come up with?

[VMmatty]

If it were me and I were just starting out with that license I would choose the more powerful server and wait on the SAN. You can always add a SAN later as budget allows. There isn't that much value if you get a SAN but don't have the vSphere features that require one. SANs can do some powerful things like snapshot LUNs, replicate to other SANs, etc, but for what you're doing it doesn't sound like you need it.

I would focus on getting a powerful host that is capable of running lots of virtual machines and then just protect it with a good backup solution. I'd recommend Veeam Backup and Replication - it's pretty affordable and does a great job. Check out http://www.veeam.com for more info.

As for diskless servers, Dell's website doesn't let you order them for whatever reason. But we've been able to order the Dell R710 series (much higher end than what you're looking at) with ESXi embedded and no disks by speaking to our Dell rep directly rather than ordering on the website. I don't think it is the right move for your situation but something to think about for the future.

There are also tools out there that will let you turn the internal storage of a server into a SAN that cost nothing. Look at tools like OpenFiler (http://www.openfiler.com/) and see if you have an older server hanging around with lots of disk space. You could use that to create a SAN for no cost at all, although I'm not sure I would run it in production if you don't have a warranty on the old server (or a support contract for OpenFiler, which does cost money).

Hope this helps and best of luck on your journey into virtualization!

[virtualeye]

@VMmatty, great feedback, was doubting the decision to go for the PE 2970.

Obviously I'll need a solid backup solution, would be nice to minimize downtime if production server has issues; I'll check out Veeam.

Servers will live behind an ASA 5505 and I'll probably have IP tables running on Linux instances just to make sure.

It is risky putting all your eggs in one basket, but I've been doing that for years, downtime once in awhile is acceptable with my clients; they just want to know someone is on the case resolving the issue...

Would love to go high end SAN right now, fun, fun, HA and Vmotion, the new server RAID, what a concept.

We'll see what the future brings ;--)

[DSTAVERT]

A simple Linux install on your old servers with an NFS share can make an ideal backup destination datastore. Have a look at http://communities.vmware.com/docs/DOC-9843 for a simple and free solution to clone your VMs.

[DSTAVERT]

Forgot to mention. Since the NFS datastore is attached to your ESX(i) host it can be used directly to launch a cloned VM in case of emergency. Back up and running in a few minutes.

[virtualeye]

Excellent, so I can leverage existing hardware as a backup solution. Was considering buying another 1U server to freshen up the backup (i.e. the SC 1425), but it probably has a couple of years left on it, has never been under any load whatsoever, just rsync backups -- I'll buy a couple of new drives and create the NFS based on the new drive setup.

Thanks for the helpful tips!

[VMmatty]

If you're going to repurpose old hardware I would make sure that you still have a maintenance contract on the server. The last thing you need is a server that has no warranty and you lose a drive and your entire environment is down for a day or more as you try to get a new drive delivered. If this is for dev/test then I think whatever you have is fine but if you are planning on running production workloads on here I'd lean towards hardware with a warranty.

Best of luck to you! Post back here if you run into any issues.

[virtualeye]

Right, agreed, but the repurposing of old hardware will be for a backup server (to store VM snapshots and web server files, database, etc.) -- I'm going to put 2 new drives on this server as well.

If backup goes down, not a huge deal, I purchase NAS space from the data center as well, so at a minimum I have 24-hour production server state + get-up-and-running snapshots on the NAS.

If production goes down, that's another problem, quite a large one, hopefully the emergency NFS solution you're suggesting will do the trick in this scenario, need to do some research as to how to pull this off.

On another note, I cancelled the PE 2970 order, realized last minute that paying for 2U rack space in order to get 2 extra SCSI drives was a bit silly, 6X 15K 146GB SCSI drives will be plenty sufficient for the minimal load production will be handling.

In place of the PE 2970 I bought the R610 (a 1U server) with:

2X Intel 2.4ghz 12M 1066mhz CPUs

6X 15K 146GB SCSI drives

16GB 1066mhz RAM

2X dual Broadcom gigabit NICs

Should be just about as fast as the PE 2970 I had spec'd out, and will pay for itself by saving on rack space costs -- now I just have to crash-course get up & running from bare metal setup to virtualized, should be fun ;--)

[DSTAVERT]

You mentioned snapshots. I am hoping you are using the term in a NON VMware sense. Make sure you understand what is involved with a snapshot. Can save many tears. http://kb.vmware.com/kb/1015180

http://communities.vmware.com/community/vmtn/vsphere?view=overview is a good page to go through. Lots of webcast / document references to get you going.

[kchawk]

2cents - one advantage of vm is it can be most any hardware. If perf is not major you can easily reuse hardware. I like to setup two or three off lease servers bought on the cheap for load and recovery. N+1. Then if server goes down replace with another off lease box. NFS is your friend. Easy backups and can power up quickly from another host if major issue.

Don't much care about warranties, just replace box. I don't co-loc though so don't have to pay for rack space.

[FrostyatCBM]

Strategy I used at a previous employer was:

Step 1 -- buy a reasonably beefy server (16GB RAM, 1TB disk) and start with that as stand-alone ESX host

Step 2 -- purchase and deploy SAN, mopve VMs off local storage to SAN

Step 3 -- buy 2nd server (no disks) and move 1/2 the disks from server #1 to server #2 (so both have a bit of local storage, which sometimes comes in handy)

We had completed step #1 and had 6 VMs running and were about to consider step #2 when I moved on to my current employer. Not sure whether that is an optimal strategy, but it is lower cost and reasonably low risk, as it allows you to try out VMware and get into the whole virtualisation concept in a progressive way. If I had to go straight to step #3 from scratch it would have been a very steep learning curve.

Oh, yes, for a while we also used one of our physical Windows servers to serve up some disk space via NFS. It worked "okay" but didn't prove to be very reliable. Getting a little NFS storage device probably would have been a better idea with the benefits of hindsight!

[virtualeye]

Right, your approach has its merits.

For low budget, it seems local disks is the cheapest in a colo environment.

I'll have 2 servers, both 1U. Backup server is getting 2 new 1TB drives that will be setup in RAID 1. VM snapshots will go there and on a data center NAS. Production server (the new R610) will run ESXi on local disks along with the live VMs.

Server arrives Friday along with the ASA 5505. Will be a busy weekend getting setup. A bit nervous going virtual, bare metal setups are cake and completely reliable in my experience (with Linux that is), not so sure about the VM route, but would really like to host multiple OSs and applications, and that's not possible without going virtual.

Will be fun to find out ;--)

[DSTAVERT]

Good luck. let us know how it goes. You might want to consider installing ESXi rather than ESX and install to a USB stick (a supported install directly in the installer) rather than your local disk. All the features are identical and your licensing gives you the choice. The only thing missing is the console but you can install the vMA appliance and support all your hosts through the one.

[virtualeye]

From what I have read, in a local disk setup there is no benefit to installing ESXi on USB.

I've have SCSI drives in RAID 10 with alert monitoring -- why would anyone go USB in a local disk setup I am not sure. Obviously in a diskless setup it makes sense, but otherwise the only benefit I see is being able to mail a USB key to the data center so as to avoid having data center staff setup ESXi.

Anyway, I never see my servers, just order from Dell and ship to data center -- they'll do default ESXi install, then I'll take care of the VM setups, firewall config, etc. $50 to setup ESXi I can live with, maybe when I go full on diskless with a SAN will I consider the USB key idea, but until then if my VMs are on local disks, so will ESXi, marginal if any performance hit, correct me if I'm wrong...

[DSTAVERT]

Just my opinion but the reason I would like to have ESXi on USB with local disks is that the install or reinstall doesn't touch the datastore. You can have a spare ready to go. In your case you don't need to rely on someone being careful, knowledgeable etc should ESXi need to be reinstalled. In a heartbeat your datastore can be gone.

[virtualeye]

Right, just wondering why one cannot also backup the ESXi instance as well??

If you can snapshot a VM can you not also snapshot the ESXi install? A bit in the dark on that one, looking for reliability here. Again, I will never lay hands on this machine, so configuring ESXi on USB myself is not an option (assuming that ESXi install must occur on machine that you'll be running it on)

I suppose I could FedEx a USB key to the data center and have them install on that, have until Friday/Saturday before I start on the setup...

[DSTAVERT]

You can back up configuration but not the ESXi host. Installing to USB is a directly supported option from the installer CD. The disk must be 1GB or larger (only uses 1GB).

A snapshot is different than you are expecting. You should probably get familiar before you get too far down the road. Go through the "Free Online Training" http://communities.vmware.com/community/vmtn/vsphere/esxi?view=overview and http://kb.vmware.com/kb/20103585971 for snapshots

[DSTAVERT]

At this point you may be too far along to use USB or SD. Scrambling to get it up and going might be counter productive but . . .