My server's CPU is quad core Xeon with 8GB RAM, plus a Gigbit switch so one 1TB disk is the bottleneck.
You cannot use the same machine for ESXi and for a NAS. You need an additional server which you have to setup for NFS or iSCSI.
On that server you may use software RAID.
>I know the hardware RAID is faster but expensive. Probably software RAID is my first try. BTW, for software RAID solution, which software has better performance, Openfiler, FreeNAS or something else? Thanks.
I don't know exact numbers for iSCSI and NFS, FreeNAS is often blamed for it's poor Samba/CIFS performance.
I don't need any gui additions, so I prefer a plain linux distro and use ssh for management.
In case of hardware controllers, I prefer Areca. They are building controllers with large caches (up to 4 GByte) and have direct ethernet access for RAID configuration and alerting (OS independant).