Tintri / VMware

Proxmox and VMware in NFS Environments & Performance Testing

tapasmahanta124Leave a comment

Network File System (NFS) is a distributed file system protocol allowing a user on a client computer to access files over a computer network much like local storage is accessed. Both Proxmox VE and VMware vSphere, leading virtualization platforms, can leverage NFS for flexible and scalable storage solutions. This document outlines key features and use cases for Proxmox and VMware in NFS environments, and details how to approach NFS performance testing.

Proxmox VE with NFS

Proxmox Virtual Environment (VE) is an open-source server virtualization management platform. It integrates KVM hypervisor and LXC containers, software-defined storage, and networking functionality on a single platform.

Key Features of Proxmox VE

  • Open-source: No licensing fees, extensive community support.
  • Integrated KVM and LXC: Supports both full virtualization (virtual machines) and lightweight containerization.
  • Web-based management interface: Provides a centralized control panel for all management tasks.
  • Clustering and High Availability (HA): Allows for the creation of resilient infrastructure by grouping multiple Proxmox VE servers.
  • Live migration: Enables moving running virtual machines between physical hosts in a cluster without downtime.
  • Built-in backup and restore tools: Offers integrated solutions for data protection.
  • Support for various storage types: Including NFS, iSCSI, Ceph, ZFS, LVM, and local directories.

Use Cases for Proxmox VE

  • Small to medium-sized businesses (SMBs) seeking a cost-effective and powerful virtualization solution.
  • Home labs and development/testing environments due to its flexibility and lack of licensing costs.
  • Hosting a variety of workloads such as web servers, databases, application servers, and network services.
  • Implementing private clouds and virtualized infrastructure.

Configuring NFS with Proxmox VE

Proxmox VE can easily integrate with NFS shares for storing VM disk images, ISO files, container templates, and backups.

  1. To add NFS storage in Proxmox VE, navigate to the “Datacenter” section in the web UI, then select “Storage”.
  2. Click the “Add” button and choose “NFS” from the dropdown menu.
  3. In the dialog box, provide the following:
    • ID: A unique name for this storage in Proxmox.
    • Server: The IP address or hostname of your NFS server.
    • Export: The exported directory path from the NFS server (e.g., /exports/data).
    • Content: Select the types of data you want to store on this NFS share (e.g., Disk image, ISO image, Container template, Backups).
  4. Adjust advanced options like NFS version if necessary, then click “Add”.

VMware vSphere with NFS

VMware vSphere is a comprehensive suite of virtualization products, with ESXi as the hypervisor and vCenter Server for centralized management. It is a widely adopted, enterprise-grade virtualization platform known for its robustness and extensive feature set.

Key Features of VMware vSphere

  • Robust and mature hypervisor (ESXi): Provides a stable and high-performance virtualization layer.
  • Advanced features: Includes vMotion (live migration of VMs), Storage vMotion (live migration of VM storage), Distributed Resource Scheduler (DRS) for load balancing, High Availability (HA) for automatic VM restart, and Fault Tolerance (FT) for continuous availability.
  • Comprehensive management with vCenter Server: A centralized platform for managing all aspects of the vSphere environment.
  • Strong ecosystem and third-party integrations: Wide support from hardware vendors and software developers.
  • Wide range of supported guest operating systems and hardware.
  • Advanced networking (vSphere Distributed Switch, NSX) and security features.

Use Cases for VMware vSphere

  • Enterprise data centers and hosting mission-critical applications requiring high availability and performance.
  • Large-scale virtualization deployments managing hundreds or thousands of VMs.
  • Virtual Desktop Infrastructure (VDI) deployments.
  • Implementing robust disaster recovery and business continuity solutions.
  • Building private, public, and hybrid cloud computing environments.

Configuring NFS with VMware vSphere

vSphere supports NFS version 3 and 4.1 for creating datastores. NFS datastores can be used to store virtual machine files (VMDKs), templates, and ISO images.

  1. Ensure your ESXi hosts have a VMkernel port configured for NFS traffic (typically on the management network or a dedicated storage network).
  2. Using the vSphere Client connected to vCenter Server (or directly to an ESXi host):
    1. Navigate to the host or cluster where you want to add the datastore.
    2. Go to the “Configure” tab, then select “Datastores” under Storage, and click “New Datastore”.
    3. In the New Datastore wizard, select “NFS” as the type of datastore.
    4. Choose the NFS version (NFS 3 or NFS 4.1). NFS 4.1 offers enhancements like Kerberos security.
    5. Enter a name for the datastore.
    6. Provide the NFS server’s IP address or hostname and the folder/share path (e.g., /vol/datastore1).
    7. Choose whether to mount the NFS share as read-only or read/write (default).
    8. Review the settings and click “Finish”.

NFS Performance Testing

Testing the performance of your NFS storage is crucial to ensure it meets the demands of your virtualized workloads and to identify potential bottlenecks before they impact production.

Why test NFS performance?

  • To validate that the NFS storage solution can deliver the required IOPS (Input/Output Operations Per Second) and throughput for your virtual machines.
  • To identify bottlenecks in the storage infrastructure, network configuration (switches, NICs, cabling), or NFS server settings.
  • To establish a performance baseline before making changes (e.g., software upgrades, hardware changes, network modifications) and to verify improvements after changes.
  • To ensure a satisfactory user experience for applications running on VMs that rely on NFS storage.
  • For capacity planning and to understand storage limitations.

Common tools for NFS performance testing

  • fio (Flexible I/O Tester): A powerful and versatile open-source I/O benchmarking tool that can simulate various workload types (sequential, random, different block sizes, read/write mixes). Highly recommended.
  • iozone: Another popular filesystem benchmark tool that can test various aspects of file system performance.
  • dd: A basic Unix utility that can be used for simple sequential read/write tests, but it’s less comprehensive for detailed performance analysis.
  • VM-level tools: Guest OS specific tools (e.g., CrystalDiskMark on Windows, or `fio` within a Linux VM) can also be used from within a virtual machine accessing the NFS datastore to measure performance from the application’s perspective.

What the test does (explaining a generic NFS performance test)

A typical NFS performance test involves a client (e.g., a Proxmox host, an ESXi host, or a VM running on one of these platforms) generating I/O operations (reads and writes) of various sizes and patterns (sequential, random) to files located on the NFS share. The primary goal is to measure:

  • Throughput: The rate at which data can be transferred, usually measured in MB/s or GB/s. This is important for large file transfers or streaming workloads.
  • IOPS (Input/Output Operations Per Second): The number of read or write operations that can be performed per second. This is critical for transactional workloads like databases or applications with many small I/O requests.
  • Latency: The time taken for an I/O operation to complete, usually measured in milliseconds (ms) or microseconds (µs). Low latency is crucial for responsive applications.

The test simulates different workload profiles (e.g., mimicking a database server, web server, or file server) to understand how the NFS storage performs under conditions relevant to its intended use.

Key metrics to observe

  • Read/Write IOPS for various block sizes (e.g., 4KB, 8KB, 64KB, 1MB).
  • Read/Write throughput (bandwidth) for sequential and random operations.
  • Average, 95th percentile, and maximum latency for I/O operations.
  • CPU utilization on both the NFS client (hypervisor or VM) and the NFS server during the test.
  • Network utilization and potential congestion points (e.g., packet loss, retransmits).

Steps to run a (generic) NFS performance test

  1. Define Objectives and Scope: Clearly determine what you want to measure (e.g., maximum sequential throughput, random 4K IOPS, latency under specific load). Identify the specific NFS share and client(s) for testing.
  2. Prepare the Test Environment:
    • Ensure the NFS share is correctly mounted on the test client(s).
    • Minimize other activities on the NFS server, client, and network during the test to get clean results.
    • Verify network connectivity and configuration (e.g., jumbo frames if used, correct VLANs).
  3. Choose and Install a Benchmarking Tool: For example, install `fio` on the Linux-based hypervisor (Proxmox VE) or a Linux VM.
  4. Configure Test Parameters in the Tool:
    • Test file size: Should be significantly larger than the NFS server’s cache and the client’s RAM to avoid misleading results due to caching (e.g., 2-3 times the RAM of the NFS server).
    • Block size (bs): Vary this to match expected workloads (e.g., bs=4k for database-like random I/O, bs=1M for sequential streaming).
    • Read/Write mix (rw): Examples: read (100% read), write (100% write), randread, randwrite, rw (50/50 read/write), randrw (50/50 random read/write), or specific mixes like rwmixread=70 (70% read, 30% write).
    • Workload type: Sequential (rw=read or rw=write) or random (rw=randread or rw=randwrite).
    • Number of threads/jobs (numjobs): To simulate concurrent access from multiple applications or VMs.
    • I/O depth (iodepth): Number of outstanding I/O operations, simulating queue depth.
    • Duration of the test (runtime): Run long enough to reach a steady state (e.g., 5-15 minutes per test case).
    • Target directory: Point to a directory on the mounted NFS share.
  5. Execute the Test: Run the benchmark tool from the client machine, targeting a file or directory on the NFS share.

Example fio command (conceptual for a random read/write test):

  1. (Note: /mnt/nfs_share_mountpoint should be replaced with the actual mount point of your NFS share. Parameters like size, numjobs, iodepth should be adjusted based on specific needs, available resources, and the NFS server’s capabilities. direct=1 attempts to bypass client-side caching.)
  2. Collect and Analyze Results: Gather the output from the tool (IOPS, throughput, latency figures). Also, monitor CPU, memory, and network utilization on both the client and the NFS server during the test using tools like top, htop, vmstat, iostat, nfsstat, sar, or platform-specific monitoring tools (Proxmox VE dashboard, ESXTOP).
  3. Document and Iterate: Record the test configuration and results. If performance is not as expected, investigate potential bottlenecks (NFS server tuning, network, client settings), make adjustments, and re-test to measure the impact of changes. Repeat with different test parameters to cover various workload profiles.

Conclusion

Both Proxmox VE and VMware vSphere offer robust support for NFS, providing flexible and scalable storage solutions for virtual environments. Understanding their respective key features, use cases, and configuration methods helps in architecting efficient virtualized infrastructures. Regardless of the chosen virtualization platform, performing diligent and methodical NFS performance testing is essential. It allows you to validate your storage design, ensure optimal operation, proactively identify and resolve bottlenecks, and ultimately guarantee that your storage infrastructure can effectively support the demands of your virtualized workloads and applications.


fio --name=nfs_randrw_test \
--directory=/mnt/nfs_share_mountpoint \
--ioengine=libaio \
--direct=1 \
--rw=randrw \
--rwmixread=70 \
--bs=4k \
--size=20G \
--numjobs=8 \
--iodepth=32 \
--runtime=300 \
--group_reporting \
--output=nfs_test_results.txt

Leave a comment