I/O filters are software components that can be installed on ESXi hosts and can offer additional data services to virtual machines. The filters process I/O requests, which move between the guest operating system of a virtual machine and virtual disks.

The I/O filters can be offered by VMware or created by third parties through vSphere APIs for I/O Filtering (VAIO).

I/O filters can gain direct access to the virtual machine I/O path. You can enable the I/O filter for an individual virtual disk level. The I/O filters are independent of the storage topology.

VMware offers certain categories of I/O filters. In addition, third-party vendors can create the I/O filters. Typically, they are distributed as packages that provide an installer to deploy the filter components on vCenter Server and ESXi host clusters.

After the I/O filters are deployed, vCenter Server configures and registers an I/O filter storage provider, also called a VASA provider, for each host in the cluster. The storage providers communicate with vCenter Server and make data services offered by the I/O filter visible in the VM Storage Policies interface. You can reference these data services when defining common rules for a VM policy. After you associate virtual disks with this policy, the I/O filters are enabled on the virtual disks.

Datastore Support

I/O filters can support all datastore types including the following:

• VMFS
• NFS 3
• NFS 4.1
• Virtual Volumes (VVol)
• vSAN

Types of I/O Filters

VMware provides certain categories of I/O filters that are installed on your ESXi hosts. In addition, VMware partners can create the I/O filters through the vSphere APIs for I/O Filtering (VAIO) developer program. The I/O filters can serve multiple purposes.

The supported types of filters include the following:

• Replication. Replicates all write I/O operations to an external target location, such as another host or cluster.
• Encryption. Offered by VMware. Provides encryption mechanisms for virtual machines. For more information, see the vSphere Securitydocumentation.
• Caching. Implements a cache for virtual disk data. The filter can use a local flash storage device to cache the data and increase the IOPS and hardware utilization rates for the virtual disk. If you use the caching filter, you might need to configure a Virtual Flash Resource.
• Storage I/O control. Offered by VMware. Throttles the I/O load towards a datastore and controls the amount of storage I/O that is allocated to virtual machines during periods of I/O congestion. For more information, see the vSphere Resource Management documentation.

I/O Filtering Components:

+++++++++++++++++++++

VAIO Filter Framework

A combination of user world and VMkernel infrastructure provided by ESXi. With the framework, you can add filter plug-ins to the I/O path to and from virtual disks. The infrastructure includes an I/O filter storage provider (VASA provider). The provider integrates with the Storage Policy Based Management (SPBM) system and exports filter capabilities to vCenter Server.

I/O Filter Plug-In

A software component provided by VMware or developed by VMware partners that intercepts and filters I/O data in transit between virtual disks and guest operating systems. If VMware partners develop the I/O filter, the filter might include additional optional components that help in its configuration and management.

Each Virtual Machine Executable (VMX) component of a virtual machine contains a Filter Framework that manages the I/O filter plug-ins attached to the virtual disk. The Filter Framework invokes filters when the I/O requests move between the guest operating system and the virtual disk. Also, the filter intercepts any I/O access towards the virtual disk that happens outside of a running VM.

The filters run sequentially in a specific order. For example, a replication filter executes before a cache filter. More than one filter can operate on the virtual disk, but only one for each category.

Once all filters for the particular disk verify the I/O request, the request moves to its destination, either the VM or the virtual disk.

Because the filters run in user space, any filter failures impact only the VM, but do not affect the ESXi host.

Storage Providers for I/O Filters

When I/O filters are installed on ESXi hosts, the I/O filter framework configures and registers a storage provider, also called a VASA provider, for each host in the cluster.

Storage providers for I/O filtering are software components that are offered by vSphere. They integrate with I/O filters and report data service capabilities that I/O filters support to vCenter Server.

The capabilities populate the VM Storage Policies interface and can be referenced in a VM storage policy. You then apply this policy to virtual disks, so that the I/O filters can process I/O for the disks.

System Requirements for I/O Filters

To be able to use I/O filters in your environment, you must follow specific requirements.

The following requirements apply.

• Use the latest version of ESXi and vCenter Server compatible with I/O filters. Older versions might not support I/O filters, or provide only partial support.
• Check for any additional requirements that individual partner solutions might have. In specific cases, your environment might need flash devices, extra physical memory, or network connectivity and bandwidth. For information, contact your vendor or your VMware representative.
• Web server to host partner packages for filter installation. The server must remain available after initial installation. When a new host joins the cluster, the server pushes appropriate I/O filter components to the host.

Enable I/O Filter Data Services on Virtual Disks

Enabling data services that I/O filters provide is a two-step process. You create a virtual machine policy based on data services that the I/O filters provide, and then attach this policy to a virtual machine.

Prerequisites

For the caching I/O filters, configure the virtual flash resource on your ESXi host.

Procedure

1. Define a VM policy based on I/O filter services.You must first create a virtual machine policy that lists data services provided by the I/O filters.
2. Assign the I/O filter policy to a virtual machine.To activate data services that the I/O filter provides, associate the I/O filter policy with virtual disks. You can assign the policy when you provision the virtual machine.

Assign the I/O Filter Policy to Virtual Machines

To activate data services that I/O filters provide, associate the I/O filter policy with virtual disks. You can assign the policy when you create or edit a virtual machine.

You can assign the I/O filter policy during an initial deployment of a virtual machine. This topic describes how to assign the policy when you create a new virtual machine. For information about other deployment methods, see the vSphere Virtual Machine Administration documentation.

Note:

You cannot change or assign the I/O filter policy when migrating or cloning a virtual machine.

Prerequisites

Verify that the I/O filter is installed on the ESXi host where the virtual machine runs.

Procedure

1. Start the virtual machine provisioning process and follow the appropriate steps.
2. Assign the same storage policy to all virtual machine files and disks.
   • On the Select storage page, select a storage policy from the VM Storage Policy drop-down menu.
   • Select the datastore from the list of compatible datastores and click Next.The datastore becomes the destination storage resource for the virtual machine configuration file and all virtual disks. The policy also activates I/O filter services for the virtual disks.
3. Change the VM storage policy for the virtual disk.Use this option to enable I/O filters just for your virtual disks.
   • On the Customize hardware page, expand the New hard disk pane.
   • From the VM storage policy drop-down menu, select the storage policy to assign to the virtual disk.
   • (Optional) Change the storage location of the virtual disk.Use this option to store the virtual disk on a datastore other than the datastore where the VM configuration file resides.
4. Complete the virtual machine provisioning process.

Results

After you create the virtual machine, the Summary tab displays the assigned storage policies and their compliance status.

ATS-Only Mechanism

For storage devices that support T10 standard-based VAAI specifications, VMFS provides ATS locking, also called hardware-assisted locking. The ATS algorithm supports discrete locking per disk sector. All newly formatted VMFS5 and VMFS6 datastores use the ATS-only mechanism if the underlying storage supports it, and never use SCSI reservations.

When you create a multi-extent datastore where ATS is used, vCenter Server filters out non-ATS devices. This filtering allows you to use only those devices that support the ATS primitive.

In certain cases, you might need to turn off the ATS-only setting for a VMFS5 or VMFS6 datastore.

ATS+SCSI Mechanism

A VMFS datastore that supports the ATS+SCSI mechanism is configured to use ATS and attempts to use it when possible. If ATS fails, the VMFS datastore reverts to SCSI reservations. In contrast with the ATS locking, the SCSI reservations lock an entire storage device while an operation that requires metadata protection is performed. After the operation completes, VMFS releases the reservation and other operations can continue.

Datastores that use the ATS+SCSI mechanism include VMFS5 datastores that were upgraded from VMFS3. In addition, new VMFS5 or VMFS6 datastores on storage devices that do not support ATS use the ATS+SCSI mechanism.

Configure SAN Components and Storage System :

Before you set up your ESXi host to boot from a SAN LUN, configure SAN components and a storage system.

Because configuring the SAN components is vendor specificǰ refer to the product documentation for each item.

Procedure

>> Connect network cable, referring to any cabling guide that applies to your setup. Check the switch wiring, if there is any.

>> Configure the storage array.

      ++ From the SAN storage array, make the ESXi host visible to the SAN. This process is often referred to as creating an object.

      ++ From the SAN storage array, set up the host to have the WWPNs of the host’s adapters as port names or node names.

      ++ Create LUNs.

      ++ Assign LUNs.

      ++ Record the IP addresses of the switches and storage arrays.

      ++ Record the WWPN for each SP.

Configure Storage Adapter to Boot from SAN :

When you set up your host to boot from SAN, you enable the boot adapter in the host BIOS. You then configure the boot adapter to initiate a primitive connection to the target boot LUN.

Prerequisites

Determine the WWPN for the storage adapter.

Procedure

Configure the storage adapter to boot from SAN. Because configuring boot adapters is vendor specificǰ refer to your vendor documentation.

Set Up Your System to Boot from Installation Media

>>When seĴing up your host to boot from SAN, you first boot the host from the VMware installation media. To achieve this, you need to change the system boot sequence in the BIOS setup. Because changing the boot sequence in the BIOS is vendor specificǰ refer to vendor documentation for instructions.

The following procedure explains how to change the boot sequence on an Dell host.

Procedure

>> During your system power up, enter the system BIOS ConfigurationȦSetup Utility.

>> Select Startup Options and press Enter.

>> Select Startup Sequence Options and press Enter.

>> Change the First Startup Device to [CD-ROM].

You can now install ESXi

For further reference please follow : https://docs.vmware.com/en/VMware-vSphere/6.5/vsphere-esxi-vcenter-server-65-storage-guide.pdf

Connecting to a vPostgres Database With pgAdmin:

The pgAdmin client tool is not necessary for connecting to vPostgres databases. The Data Director UI includes everything you need to manage your vPostgres databases graphically. The instructions for using pgAdmin are included for completeness.

If you do not have pgAdmin installed, download the pgAdmin appropriate for your platform from the Postgres site and install it. See the Postgres site for information.

To use the Postgres pgAdmin tool to connect to vPostgres databases, pgAdmin must use the Data Director libpq.

The Windows pgAdmin tool works only with Windows 32-bit systems. If you run Windows 64-bit, install the 32-bit versions of the vPostgres client tools. The installer places the 32-bit tools in the C:\Program Files (x86) tree.

Before you begin, obtain the following information.

■        The vPostgres connection string. See Connecting to vPostgres Databases.

■        The Data Director DB Name Service IP address. You can get the IP address from the Data Director       vApp in vSphere Client. Contact your administrator if you need help.

>>      Login to vSphere Client as an administrator.

>>      Locate your Data Director vApp in the Hosts and Clusters list, and expand the vApp.

>>      Click DB Name Server to select it, and click the Summary tab.

>>      The IP address is listed in the General pane.

Ask your system administrator for help if you do not have access to the Data Director Web UI or to the vSphere Client.

1        Ensure that pgAdmin uses the vPostgres libpq on Windows.

>>      Copy the C:\Program Files\pgAdmin III\<version> directory to C:\Program         Files\pgAdmin III\<version>-vPostgres, where <version> is the pgAdmin version number:

>>      Copy all the files in your C:\Program Files\VMware\vPostgres\1.0\bin directory to C:\Program Files\pgAdmin III\<version>-vPostgres.

2        Start pgAdmin from the C:\Program Files\pgAdmin III\<version>-vPostgres directory.

3        Select File > Add Server.

4        Enter values for the following properties.

>>      Name. Enter a meaningful name for the server, such as Data Director.

>>      Host. Enter the {UUID}.[DB Name Service IP or FQDN] part of the vPostgres connection string.

>>      Leave the default values for Port, Service, and Maintenance DB.

>>      Username. Enter the database user name. This is usually the database owner user ID.

>>      Password. Enter the database user name’s password. You can optionally store the password, but note that pgAdmin stores the password in plain text format.

>>      (Optional) Enter a color to denote this server in the object browser and in diagrams.

>>      (Optional) Enter a server group in which to place this server, or accept the default.

5        Click the Advanced tab, and type the DB Name Service IP address in the Host text box.

6        Click OK.

PgAdmin connects to the vPostgres database:

Connect to a vPostgres Database with JDBC

The JDBC connection string has the following format.

jdbc:postgresql://{UUID}.<host name>/<RDB name>?user=<user name>

The curly brackets, {}, are part of the connection string and denote the UUID.

For example, suppose that your vPostgres database, myDB, is deployed on the host w1-devtest-22.dev.mycorp.com. If theUUID is d35f7ab1-d70e-4d98-c121-122f68e4ab60 and the user name is dbowner, the JDBC connection string is as follows.

jdbc:postgresql://{d35f7ab1-d70e-4d98-c121-122f68e4ab60}.w1-devtest-22.dev.mycorp.com/mydb?user=dbowner

Connect to a vPostgres Database With psql:

The psql connection string has the following format.

psql -h {UUID}.<DB Name Service IP> -p 5432 -d <DB name> -U <db user name>

You connect to a database using psql. The database has the UUID 1234-5678-9012-3456, the DB Name Service port is 5432, the IP address for the DB Name Server is 10.0.0.1, the database name is myDB, and the database user name is dbuser. The psql command is as follows.

$ psql -h {1234-5678-9012-3456}.10.0.0.1 -p 5432 -d myDB -U dbuser

Psql connects to myDB and prompts for the password, and logs you in. You can enter psql commands as usual.

Back up the embedded vCenter Server Appliance database to protect the data stored in your vPostgres database.

Prerequisite:

Create the folder in which you want to create the backup file and verify that you have read and write permissions on this folder.

Procedure:

Caution: This procedure cannot be stopped. Stopping the script will cause inconsistencies in the vCenter Server appliance database and can prevent vCenter Server appliance from starting.

1. Log in to the vCenter Server Appliance Linux console as root.
2. Download the Linux backup and restore package 2091961_linux_backup_restore.zip attached to this Knowledge Base article and extract it on the Linux machine.
3. Make a backup_lin.py executable.

   For example to save the file as /tmp/backup_lin.py , run this command:

   chmod 700 /tmp/backup_lin.py

4. Run the backup_lin.py file and provide the location for the backup file.

   For example, if you want to save the file as /tmp/backup_VCDB.bak, run this command:

   python /tmp/backup_lin.py -f /tmp/backup_VCDB.bak

When the backup completes, you see a message that the backup completed successfully.

Restore the vCenter Server Appliance vPostgres Database

It may be required to copy the database to the new vCenter Server Appliance or Windows installed vCenter Server. After you back up the embedded vPostgres database, you can restore it from the backup file.

Note: Using WinSCP on the vCenter Server Appliance may fail. For more information, see Error when uploading files to vCenter Server Appliance using WinSCP (2107727).

Prerequisite:

Back up the vCenter Server Appliance embedded vPostgres database.

Procedure:

Caution: This procedure cannot be stopped. Stopping the script will cause inconsistencies in the vCenter Server appliance database and can prevent vCenter Server appliance from starting.

1. Log in to the vCenter Server Appliance Linux console as root.
2. Download the Linux backup and restore package 2091961_linux_backup_restore.zip attached to this Knowledge Base article and extract it on the Linux machine.
3. Make a restore_lin.py executable, for example /tmp/restore_lin.pychmod 700 /tmp/restore_lin.py
4. Stop the vmware-vpxd and vmware-vdcs services, by running these commands depending on vCenter Server version:

5. Run the restore_lin.py file and provide the location for the backup file.

   For example, if the backup file is saved to /tmp/backup_VCDB.bak, run this command:

   python /tmp/restore_lin.py -f /tmp/backup_VCDB.bak

   When the restore completes, you see a message that the restore completed successfully.

6. Start the vmware-vpxd and vmware-vdcs services, by running these commands depending on vCenter Server version:

vSAN can perform data at rest encryption. Data is encrypted after all other processing, such as deduplication, is performed. Data at rest encryption protects data on storage devices, in case a device is removed from the cluster.

Using encryption on your vSAN cluster requires some preparation. After your environment is set up, you can enable encryption on your vSAN cluster.

vSAN encryption requires an external Key Management Server (KMS), the vCenter Server system, and your ESXi hosts. vCenter Server requests encryption keys from an external KMS. The KMS generates and stores the keys, and vCenter Server obtains the key IDs from the KMS and distributes them to the ESXihosts.

vCenter Server does not store the KMS keys, but keeps a list of key IDs.

Design

Consider these guidelines when working with vSAN encryption.

• Do not deploy your KMS server on the same vSAN datastore that you plan to encrypt.
• Encryption is CPU intensive. AES-NI significantly improves encryption performance. Enable AES-NI in your BIOS.
• The witness host in a stretched cluster does not participate in vSAN encryption. Only metadata is stored on the witness host.
• Establish a policy regarding core dumps. Core dumps are encrypted because they can contain sensitive information such as keys. If you decrypt a core dump, carefully handle its sensitive information. ESXi core dumps might contain keys for the ESXi host and for the data on it.
  • Always use a password when you collect a vm-support bundle. You can specify the password when you generate the support bundle from the vSphere Client or using the vm-support command.
    The password recrypts core dumps that use internal keys to use keys that are based on the password. You can later use the password to decrypt any encrypted core dumps that might be included in the support bundle. Unencrypted core dumps or logs are not affected.
  • The password that you specify during vm-support bundle creation is not persisted in vSphere components. You are responsible for keeping track of passwords for support bundles.

Enable Encryption on a New vSAN Cluster

Prerequisites

• Required privileges:
  • Host > Inventory > EditCluster
  • Cryptographer > ManageEncryptionPolicy
  • Cryptographer > ManageKMS
  • Cryptographer > ManageKeys
• You must have set up a KMS cluster and established a trusted connection between vCenter Server and the KMS.

Procedure

1. Navigate to an existing cluster.
2. Click the Configure tab.
3. Under vSAN, select Services and click the Encryption Edit button.
4. On the vSAN Services dialog, enable Encryption, and select a KMS cluster.Note:Make sure the Erase disks before use check box is deselected, unless you want to wipe existing data from the storage devices as they are encrypted.
5. Complete your cluster configuration.

Results

Encryption of data at rest is enabled on the vSAN cluster. vSAN encrypts all data added to the vSAN datastore.

Enable vSAN Encryption on Existing vSAN Cluster

You can enable encryption by editing the configuration parameters of an existing vSAN cluster.

Prerequisites

• Required privileges:
  • Host > Inventory > EditCluster
  • Cryptographer > ManageEncryptionPolicy
  • Cryptographer > ManageKMS
  • Cryptographer > ManageKeys
• You must have set up a KMS cluster and established a trusted connection between vCenter Server and the KMS.
• The cluster’s disk-claiming mode must be set to manual.

Procedure

1. Navigate to the vSAN host cluster.
2. Click the Configure tab.
3. Under vSAN, select Services.
4. Click the Encryption Edit button.
5. On the vSAN Services dialog, enable Encryption, and select a KMS cluster.
6. (Optional) If the storage devices in your cluster contain sensitive data, select Erase Disks Before Use.This setting directs vSAN to wipe existing data from the storage devices as they are encrypted. This option can increase the time to process each disk, so do not choose it unless you have unwanted data on the disks.
7. Click Apply.

Results

A rolling reformat of all disk groups takes places as vSAN encrypts all data in the vSAN datastore.

When the virtual machines attempt a snapshot consolidation there are underlying vSAN components which undergo a change of node-ownership within the cluster. When these events happen simultaneously, various behaviors may occur:

• An error stating Generic Configuration Fault Error occurs when a backup is taken.
• An error stating the virtual machine cannot continue to run due to a problem with quiescing the guest operation system, and it might crash.

Error message = “An error occurred while saving the snapshot: Failed to quiesce the virtual machine”

abdicateDomOwnership

public java.lang.String[] abdicateDomOwnership(java.lang.String[] uuids)

                                        throws RuntimeFault,

                                               java.rmi.RemoteException

Abdicate ownership of DOM objects. The objects must be currently owned by this host. Which host has ownership of an object at a given point in time can be queried from QueryVsanObjects() or QueryCmmds() APIs. Abdicating ownership tears down DOM owner in-memory state. Hosts in the cluster will then compete to become the new owner of the object, similar to a host failure event. There is a short interruption of IO flow while the owner re-election is going on, but it is transparent to any consumers of the object. This API is meant as a troubleshooting and debugging tool. It is internal at this point and can be used to resolve issues where DOM owner gets “stuck”.

Parameters:

uuids – List of VSAN/DOM object UUIDs.

Returns:

String[]

Throws:

java.rmi.RemoteException

RuntimeFault ——>https://jar-download.com/javaDoc/com.toastcoders/yavijava/6.0.03/com/vmware/vim25/RuntimeFault.html

Since:

vSphere API 6.0

Please contact VMware support for further understanding on Owner Abdication on VSAN as it is an internal process.

VMware ESXi and ESX provide advanced configuration options that affect the behavior of various components. This article provides steps to review and set new advanced configuration options using several methods. VMware recommends that you set these configuration options under the direction of VMware Technical Support or a VMware Knowledge Base article.

Numeric options have limited ranges (for example, 0-10). String options accept any value. Option values are not checked for validity beyond being in the proper range. Confirm all changes before applying them.

Advanced configuration settings can be reviewed and modified on an ESXi/ESX host using the vSphere Web Client, vSphere Client, PowerCLI, Command-Line Interface, or local console.

All options are grouped into sections. A given method may visually group the sections or separate the section and option names using a forward slash or period. Options are usually documented using the form.SectionName.OptionName

When deploying virtual machines located on a non-shared storage to other ESXi hosts, the deployment occurs across the network (using NFC/hostd). This may have an impact on the file system buffer cache and exhaust the number of buffers, reporting a significant delay in deployment time.

Advanced Options:

> Login to the host console using ‘root’ credentials

>Click on Manage

>From System Tab please click on ‘Advanced Settings’

To prevent this issue, relocate the virtual machine templates to a shared storage that is presented to all ESXi hosts.

1. To increase the number of buffers from 2048 to 4096, run this command:# esxcfg-advcfg -s 32768 /BufferCache/MaxCapacity
   

   Note: If the above change does not result in a significant improvement, try reducing the flush interval.

2. To reduce the Buffer Cache Flush interval from 30 seconds to 20 seconds, run this command:# esxcfg-advcfg -s 20000 /BufferCache/FlushInterval
   

A host reboot is not required to increase the buffers, but to decrease the buffers to a lower value or to the default value, a host reboot is necessary for the changes to take effect.