iSCSI over WAN

[Vyatta]

Hello all,

First let me say this is more for training/to see if it can be setup than anything production. I've read plenty about iSCSI performance over WAN not being the best and acknowledge that. Yet, I still want to set it up.

A friend and myself have a point to point VPN tunnel between our networks (home). We both have ESXi 4's setup at each location. I also have a virtual ESX 4 setup as well. On my side, both servers connect fine to my iSCSI server running FreeNAS. My friend's system can see the target fine but cannot connect to the target's LUN. I've read that this is usually due to the iSCSI server presenting it's IP address back to the initiator but that shouldn't be a problem in our case. I can ping his ESXi's IP and he can ping my iSCSI by IP (our routing, firewalls, etc.) seem to be OK. His DNS server forwards my zones over to me and vice versa as well (did that to get his ESXi in my VCenter). Using the MS iSCSI initiator on his side, he can see the target and Windows can see the volume as well (can't access it due to the file system on it but that's understandable). What might we be missing? I greatly appreciate any comments.

[golddiggie]

If you're using CHAP, try turning that off and see if it flies. I've experienced some issues when using CHAP on iSCSI LUNs. Typically once that's removed from the mix things fly right. You could do it for a single LUN and see if the other cluster can see the storage.

BTW, it sounds like you're looking to allow your friend's cluster/server to use your storage. Can you configure the server, once you connect to his, to use the storage from your location? It does sound like you've got the domains trusting each other, but does he have an account on your domain (and vice versa) that's in the admin group on his ESX/ESXi server? Basically, have admin user accounts on both hosts/clusters/configurations that are from both domains.

Depending on the internet connections/pipe between the two locations, performance could either suck, or be within tolerance. That's something you'll need to decide.

Of course, you could be facing a limitation of the FreeNAS configuration. I looked at using that software briefly. Until it became clear, for me, that it wasn't really an option. That was more due to the hardware I was looking to use, but also partially due to limitations of the software. I'm now using a QNAP 5 drive array presenting iSCSI LUNs to my ESXi host, as well as shares to use for file storage (file backup location, as well as shares to move files between VM's and physical systems easily)... I do understand the allure of FreeNAS, it just wasn't a good fit for me.

[Vyatta]

CHAP is turned off completely. Yeah, I ran into some issues with it on my ESX/i's originally and just turned it off since I have my iSCSI server VLANed off to just my ESX/i's.

Correct. We're going to be using the storage in the other's location (there's going to be an iSCSI server in both locations). Both of us are logging in using admin accounts for the VCenter and ESXi's. We may consolidate into one admin account that's everywhere (AD and ESX/i's) so thanks for that suggestion.

We both think that it's going to suck but we're prepared for that. This is more a proof of concept/'Can I do it?' scenario than anything. We're seeing if we can setup a DR type scenario with a limited budget, etc.

We went with FreeNAS more from a budgetary and ease of use standpoint than anything. While I would love to buy a decent iSCSI physical SAN, it's just not in the cards. I looked at Openfiler but it was just way too complicated for what I needed at the time. I figure that when we get all our configurations done, we're going to tear them down and re-work them in a more efficient manner with things that we've learned. I appreciate the comments. Thanks.

[golddiggie]

How about using a B&R application to backup each other's VM's to the other's SAN/iSCSI target/drive? If you can mount the iSCSI LUN within the VM (MS iSCSI software) then you could just run B&R software to provide DR to each other's location. Since you're still extremely vulnerable with what I think you're trying to do. Basically, if there's any kind of issue, with the other location, your side goes dark. So a fault at one location brings both down.

You could do a trial of B&R software like Acronis to see if that will do a reasonable job. It could, it would just need to be tested to confirm that it does the job properly.

IMO, there's a reason why clusters have their main storage on location. For B&R, you can have that storage someplace else. 

While I do appreciate the thoughts behind what you're trying to do, I see some issues being created that could be avoided. IMO, you don't want to have a single point of failure, that will bring down both locations. As I think I understand what you're looking to do, you would have this (at least for the VM's)... You might be able to get some kind of WAN optimization virtual appliance, or set one up via open source solutions (I haven't looked for either, so I could be off there). That could help with the performance of the B&R solution. Or have it run during times when the servers won't be accessed actively. Like when people are sleeping...

If you're looking to do something like a poor man's SRM, I think you'll need to really alter the initial design concept. IMO, a good compromise would be running the VM's off of the local LUNs, but having the sites backup to LUNs/iSCSI targets on the other site's storage.

[Vyatta]

I think that might be the way we'll go. We'll get an iSCSI at each site this weekend or so. I don't think we'll have any issue getting that setup as two different entities (each ESXi having the local iSCSI connection). Even if we did that though, the initial question/issue stands. Connection to the iSCSI server for a datastore for the remote ESXi.  If it can't be done, that's cool. I just have read too much that seems to make it possible.

Don't get me wrong, our current setup is mutating as we learn about things. That's why I mentioned that at some point we'll be tearing everything down and rebuilding with more knowledge and understanding behind the new design. Right now, it's us learning about this in a hands on manner. Performance isn't an issue (not meaning that we have unlimited resources, just we know we'll have performance issues).  Single points of failure? Same. Not an issue (again because we know we're going to have them with this setup.).

However, we'll definately take a look at this Acronis to see if it's something that we can use in our final setup. We're more than happy to listen to good advise and that's what we're getting. :smileyhappy:

[Vyatta]

After some dedicated time to really watch our traffic, we finally figured out where our problem was. It was a mis-directed route headed out a wrong interface. While it's not setting any speed records, iSCSI access over the WAN isn't as horrible for us as we feared it would be. Thanks for the help and suggestions.