VMware

VMKPing “Invalid Argument” While Testing vMotion Network

Tapas Kumar MahantaLeave a comment

When testing vMotion network connectivity from an ESXi host, vmkping can return Unknown interface 'vmkX': Invalid argument even though the VMkernel adapter exists and works for vMotion.​

This behavior is almost always related to the vMotion TCP/IP stack or another non‑default stack (vxlan, provisioning) being used on the VMkernel interface.​

Scope and prerequisites

This document applies to:

  • ESXi hosts using dedicated vMotion VMkernel adapters (often on the vMotion TCP/IP stack).​
  • Environments where vmkping from ESXi fails with an “Invalid argument” or “Unknown interface” error when specifying -I vmkX.​

Prerequisites:

  • Shell/SSH access to the ESXi host (Tech Support Mode / SSH enabled).​
  • vSphere Client access to verify VMkernel and TCP/IP stack configuration.​

Problem description and symptoms

Typical error messages

When running vmkping from ESXi to test vMotion connectivity, you might see:​

  • Unknown interface 'vmk1': Invalid argument
  • vmkping: sendto() failed: Invalid argument

This usually occurs with commands like:​

  • vmkping -I vmk1 <peer_vmotion_IP>
  • vmkping <peer_vmotion_IP>

Functional impact

Despite the error:

  • vMotion may still work successfully because the vMotion TCP/IP stack is functioning correctly.​
  • Standard ping from ESXi or from external devices to the vMotion IPs may fail because the vMotion stack is L2‑only or has no gateway.​

Root cause

vMotion TCP/IP stack behavior

VMkernel adapters can be attached to different TCP/IP stacks:​

  • defaultTcpipStack (usually Management, vMotion, vSAN in simple setups)
  • vmotion (dedicated vMotion TCP/IP stack)
  • vxlan, vSphereProvisioning, etc.

Key points:​

  • When a VMkernel adapter is created on the vMotion stack, the gateway option disappears in the UI because this stack is designed as an L2 network for vMotion traffic.
  • vmkping uses the VMkernel’s TCP/IP stack, not the host’s management stack, and requires explicit stack selection for non‑default stacks.​

If you call vmkping without telling it which TCP/IP stack to use, it assumes defaultTcpipStack.​
When the interface is actually on vmotion, this mismatch causes the Unknown interface or Invalid argument error.​

Identification: confirm VMkernel and TCP/IP stack

Perform these checks from an ESXi shell:

1. List VMkernel interfaces and stacks

esxcli network ip interface list
  • Look at Name (vmk0, vmk1, …) and Netstack Instance columns to see which stack each VMkernel uses (e.g., defaultTcpipStack, vmotion, vxlan).​

Alternative older command:​

esxcfg-vmknic -l
  • Check the NetStack column to identify which stack is bound to each vmk.​

2. List available TCP/IP stacks

esxcli network ip netstack list
  • Confirm valid stack names such as defaultTcpipStack, vmotion, vxlan, vSphereProvisioning.​

3. Validate vMotion tagging and IPs

In vSphere Client:​

  • For each host, open Configure → Networking → VMkernel adapters.
  • Verify:
    • Which vmk is enabled for vMotion.
    • Whether it uses the vMotion TCP/IP stack or the default stack.
    • IP address, VLAN, and port group settings.

Correct test methods for vMotion network

Option 1 – esxcli (recommended for vMotion stack)

Use esxcli network diag ping and specify the VMkernel and the netstack:

esxcli network diag ping -I vmk1 --netstack=vmotion -H <target_vmotion_vmk_IP>
  • Replace vmk1 with the local vMotion VMkernel name.​
  • Replace <target_vmotion_vmk_IP> with the vMotion VMkernel IP of the peer ESXi host.​

This method works even when the vMotion stack is L2‑only with no gateway.​

Jumbo frame / MTU test

For MTU 9000 testing, include payload size and do‑not‑fragment options:

esxcli network diag ping -I vmk1 --netstack=vmotion -H <target_vmotion_vmk_IP> -s 8972 -d
  • For MTU 9000, payload 8972 bytes plus headers approximates the full frame and validates end‑to‑end jumbo support.​

Option 2 – vmkping with netstack parameter

If you prefer vmkping directly:​

vmkping -S vmotion -I vmk1 <target_vmotion_vmk_IP>

or

vmkping ++netstack=vmotion -I vmk1 <target_vmotion_vmk_IP>

Key notes:​

  • -S vmotion or ++netstack=vmotion selects the vMotion TCP/IP stack.
  • Stack name is case‑sensitive and must match the value from esxcli network ip netstack list.

For MTU testing:

vmkping ++netstack=vmotion -I vmk1 -s 8972 -d <target_vmotion_vmk_IP>

Step‑by‑step troubleshooting workflow

Use this procedure when you see Invalid argument while testing vMotion.

Step 1 – Confirm VMkernel stack and role

  1. On each host, run:​bashesxcli network ip interface list
  2. Identify:
    • Which vmk has vMotion enabled.
    • Its Netstack Instance (e.g., vmotion).
  3. In vSphere Client, verify that vMotion is enabled on that vmk and that IPs are in the correct vMotion VLAN/subnet.​

Step 2 – Reproduce the error with standard vmkping

From the ESXi shell:​

vmkping -I vmk1 <target_vmotion_vmk_IP>
  • If you get Unknown interface 'vmk1': Invalid argument, this confirms the mismatch between vmkping’s default stack and the interface’s actual stack.​

Step 3 – Test with correct netstack

Run:​

vmkping -S vmotion -I vmk1 <target_vmotion_vmk_IP>
# or
esxcli network diag ping -I vmk1 --netstack=vmotion -H <target_vmotion_vmk_IP>
  • Successful responses (ICMP replies) indicate L2 connectivity on the vMotion network.​

Step 4 – Validate MTU and physical network

If connectivity works but jumbo frame tests fail:

  1. Run a jumbo ping:​bashvmkping -S vmotion -I vmk1 -s 8972 -d <target_vmotion_vmk_IP>
  2. If it fails:
    • Check vSwitch / vDS MTU configuration.
    • Check physical NIC MTU.
    • Check upstream physical switch ports and VLAN MTU.​

Step 5 – Check routing/gateway expectations

  • For vMotion TCP/IP stack, Layer 3 routing typically requires configuring a default gateway for that stack in Networking → TCP/IP configuration on the host.​
  • Without a gateway or static route, vMotion stack pings to other subnets will fail even if same‑subnet pings work.​

Example command set (copy/paste)

Adjust interface names and IPs before using.

# 1. Inventory VMkernel interfaces and stacks
esxcli network ip interface list
esxcfg-vmknic -l
esxcli network ip netstack list

# 2. Test vMotion vmk using vMotion stack (esxcli)
esxcli network diag ping -I vmk1 --netstack=vmotion -H 10.10.20.12

# 3. Test vMotion vmk using vmkping with netstack
vmkping -S vmotion -I vmk1 10.10.20.12
vmkping ++netstack=vmotion -I vmk1 10.10.20.12

# 4. Jumbo frame tests (MTU 9000)
esxcli network diag ping -I vmk1 --netstack=vmotion -H 10.10.20.12 -s 8972 -d
vmkping -S vmotion -I vmk1 -s 8972 -d 10.10.20.12

Leave a comment