VMware vSphere

 View Only
  • 1.  vSphere + NSX + SAN - Design Guide

    Posted Jan 07, 2025 05:35 PM

    Hello,

    I have a project where small remote virtualized environment needs to be deployed  with following requirements:

    1) Environment need to be deployed with 3 Hypervisor hosts (ESXi).

    2) Environment need to use All-Flash SAN (Netapp Ontap storage) . Using vSAN is not an option.

    Currently there is an option to go with VCF licenses and deploy whole VMware stack vSphere/NSX/Aria Operations (without vSAN).

    What is best design guide that could be used for deployment (vSphere+NSX+SAN) considering above requirements ?

    Please note that this thread was already posted on VCF community VMware Cloud Foundation but I was adviced to repost it here.



  • 2.  RE: vSphere + NSX + SAN - Design Guide

    Posted Jan 08, 2025 04:54 AM

    You have to make your own architecture design. That's not rocket science, but you should start by collecting design factors (requirements, constraints, assumptions), then conceptual design, logical design, and physical design considering all your design factors.

    Known design factors based on the information you provided so far.

    Requirements:

    R1: Leverage VMware SDDC components included in VMware VCF (vSphere Ent+, NSX, Aria Operations, Aria Operations for Logs) 

    Constraints: 

    C1: 3 ESXi hosts

    C2: All-Flash NetApp Ontap storage (vSAN is not the option)

    C3: VCF Licenses available

    Assumptions:

    A1: ???

    Further design questions:

    Q1: Do you have any capacity requirements (vCPU, vRAM, Usable Strorage Capacity, Usable Storage Performance/IOPS, Network Capacity, etc.)

    Q2: Do you have any overbooking ratios you have to achieve? For example vCPU/pCPU ratio, vRAM/pRAM ratio, vNet/pNet overbooking, etc.

    Q3: Do you have any other requirements? For example OS Versions and OS licensing, Database licensing, recoverability "backup & recovery" requirements (RPO, RTO, retention), some manageability requirements (monitoring, RBAC, etc.), some security requirements (TPM/vTPM, secure boot, encryption, etc.), some other availability requirements (Fault-tolerance, OS clustering, etc.)

    Q4: Do you have some other constraints like budget, operational personnel constraints, etc.?

    Q5: What storage protocol do you want to use for NetApp storage (FC, iSCSI, NVMe-oF/FC, NVMe-oF/tcp)?

    Q6: Do you have any assumptions about YoY growth? This is important for capacity planning and hardware sizing.

    Q7: Do you want to leverage NSX software-defined networking for routing (NSX gateways) and switching (overlay segments)?

    Q8: Do you want to leverage NSX for advanced security (NSX distributed firewall, stateful firewall in NSX gateways)? 

     

    vSphere, physical computing, networking, and storage (SAN) Infrastructure Conceptual Design could be relatively easy.

    • SAN (FC, iSCSI, or NVMe-oF)
      • Design Decision (FC vs Ethernet):
        • Converged ethernet would probably make sense and would have a positive impact on cost and scalability
      • Design Decision (SCSI vs NVMe):
        • NVMe-oF (End-to-end NVMe) would have a positive impact on performance and it is a good decision for the future
      • Design Decision => Use NVMe-oF/tcp for SAN storage
      • Design Decision => NetApp ONTAP supports RDMA (Remote Direct Memory Access). Use RDMA to decrease CPU usage and keep CPU cycles for application workloads 
    • 3-node vSphere Ent+ Cluster
      • ESXi Specification depends on capacity planning based on capacity requirements and YoY growth assumption
      • If 4x25Gb ports
    • physical networking 
      • 2x 25 Gb datacenter ethernet switch supporting DCB and RoCE
    • Center Server Appliance placed on top of 3-node vSphere Cluster
    • NSX Manager Appliance placed on top of 3-node vSphere Cluster
    • If you want NSX software-defined networking for routing then
      • 2x NSX Edge Nodes for T0/T1 Gateways
    • Single Node Aria Operations Manager on top of 3-node vSphere Cluster for infrastructure monitoring
    • Single Node Aria Operations for Logs on top of 3-node vSphere Cluster for centralized log management

    NSX Conceptual Design depends on further networking and application requirements.

    So this is how you should start, collect more information (design factors) and evolve your own architecture design.

    I typically do between 5 and 9 design versions (iterations) based on design reviews with other stakeholders and implementing design improvements until the final architecture design is approved and ready for implementation.

    Happy designing!




  • 3.  RE: vSphere + NSX + SAN - Design Guide

    Posted 28 days ago

    Hi David,

    Thank you for your detailed and passioned answer. Sorry for not writing more info regarding hardware.

    Hardware is already known regarding CPU RAM capacity (96pCPU, 3TB RAM). SAN connection is FC. 

    NSX software-defined networking is going to be leveraged for routing (NSX gateways) and switching (overlay segments).

    NSX Distributed FW is not required for now.

    Question regarding design part I was referring more related to the VMware by Broadcom vSphere Software. 

    While Installation of vSphere software is not a rocket science (usually), for the price of licenses I was expecting updated "best-practice design guide" for small environment "3 host, vSphere+NSX+SAN" case. Like they done for "VCF with vSAN, 4 host case".




  • 4.  RE: vSphere + NSX + SAN - Design Guide

    Posted 28 days ago
    Edited by David Pasek 28 days ago

    Ok. So in your reply, you defined further technical requirements, constraints, and scope.

    Constraint:  The design system will use existing servers with CPU/RAM capacity (96pCPU, 3TB RAM)

    Technical requirements: 

    (1) NSX software-defined networking will be leveraged for routing (NSX gateways) and switching (overlay segments).

    (2) Fibre Channel protocol must be used for SAN.

    Scope: NSX Distributed FW is not required at this time.

    ========================================================================================

    So, here are the updated design factors

    Scope:

    • vSphere Design
    • NSX Design
    • Aria Operations
    • Aria Operations for Logs

    Out-of-Scope:

    • vSAN Design
    • Aria Automation

    Technical Requirements:

    TR1: Leverage VMware SDDC components included in VMware VCF (vSphere Ent+, NSX, Aria Operations, Aria Operations for Logs) 

    TR2: vSphere must be integrated with All-Flash NetApp Ontap storage (vSAN is not the option) over Fibre Channel protocol

    TR3: NSX software-defined networking is going to be leveraged for routing (NSX gateways) and switching (overlay segments)

    Non-Functional Requirement:

    NFR1: The design system will use 3 existing servers with CPU/RAM capacity (96pCPU, 3TB RAM)

    Constraints: 

    C1: Only 3 ESXi hosts are available.

    C2: Customer has a standardized storage vendor NetApp

    Assumptions:

    A1: Physical servers used for ESXi are at VMware HCL. https://compatibilityguide.broadcom.com/

    A2: VMware VCF Licenses are available

    A3: Operational staff are trained in VMware technologies (vSphere, NSX, Aria Operations, Aria Operations for Logs).

    Based on your design factors, Validated VCF Design cannot be used as the VCF Consolidated deployment model requires 4 ESXi VSAN Ready hosts at a minimum.

    That's the reason you have to design your environment by yourself. If you do not have technical design capabilities you can engage some VMware Designer to help you with the project.