HEXDUMP on VMFS and VMX

hexdump on a VMFS (Virtual Machine File System) volume to analyze its data structures and content, it usually involves accessing the raw device representing the datastore in ESXi or another hypervisor that supports VMFS.

Warning:

This kind of operation is very risky, can lead to data corruption, and should generally be avoided, especially on production systems. Typically, only VMware Support or experienced system administrators would do this kind of operation, and mostly on a system that’s isolated from production, using a copy of the actual data.

Sample Process:

Identify the VMFS Device SSH into your ESXi host and identify the storage device representing the VMFS volume you are interested in, usually represented as /vmfs/devices/disks/naa.XXXXXXXXXXXXXXXXXXXXXXXXXXXX:.

esxcli storage vmfs extent list

Use hexdump on the Device Once you have identified the correct device, you could then use hexdump to analyze the device content.

hexdump -C /vmfs/devices/disks/naa.XXXXXXXXXXXXXXXXXXXXXXXXXXXX:
  • -C is used to display the output in “canonical” hex+ASCII display.

Example Output:

When using hexdump on a raw device, you would typically see hexadecimal representations of the data in the left columns and the ASCII representation (where possible) on the right. Non-printable characters will usually be displayed as dots ..

00000000  fa 31 c0 8e d8 8e d0 bc  00 7c fb 68 c0 07 1f 1e  |.1.......|.h...|
00000010  68 66 00 cb 88 16 0e 00  66 81 3e 03 00 4e 54 46  |hf.....f.>..NTF|
00000020  53 75 15 b4 41 bb aa 55  cd 13 72 0c 81 fb 55 aa  |Su..A..U..r...U.|
00000030  75 06 f7 c1 01 00 75 03  e9 dd 00 1e 83 ec 18 68  |u.....u........h|

Risks and Precautions:

  • Data Corruption: Incorrectly using hexdump can corrupt the data.
  • Data Sensitivity: Be mindful of sensitive information that might be exposed.
  • Read-Only Analysis: Ensure any analysis is read-only to prevent accidental data modifications.
  • Use Copies: If possible, use copies of the actual data or isolated environments to perform such analysis.

Hypothetical Example 1: VMFS Superblock

If you were to run hexdump on the device where VMFS is located, you might see the contents of the VMFS superblock, which contains metadata about the VMFS filesystem. It would look like a mix of readable ASCII characters and hexadecimal representations of binary data.

# hexdump -C /vmfs/devices/disks/naa.xxxxxxxx
00000000  4d 56 4d 46 53 2d 35 2e  30 39 00 00 00 00 00 00  |VMFS-5.09......|
...

Hypothetical Example 2: VMFS Heartbeat Region

The heartbeat region is where VMFS stores lock information and metadata updates. You may encounter sequences representing heartbeat information. This information is critical for maintaining the consistency of the VMFS filesystem in a multi-host environment.

# hexdump -C /vmfs/devices/disks/naa.xxxxxxxx
00002000  48 42 54 00 00 00 00 00  01 00 00 00 00 00 00 00  |HBT............|
...

Implications of such hypothetical examples:

  • Analysis Purpose: These examples might be used for analysis or diagnostics purposes, especially when investigating corruption or storage subsystem failures.
  • Risk of Data Corruption: Given the sensitive nature of the data in these regions, performing write operations here could lead to irrecoverable data loss.
  • Complexity of Interpretation: Interpreting such data requires in-depth knowledge of VMFS internal structures and is usually reserved for VMware developers or support engineers.
  • Need for Caution: Any attempt to read the VMFS structure directly should be approached with extreme caution.

Recommended Approach:

For normal VMFS troubleshooting and recovery:

  1. Use VMware-Supported Tools: Use built-in tools like VOMA to check VMFS metadata integrity.
  2. Consult VMware Documentation: Refer to official VMware documentation for troubleshooting steps.
  3. Engage VMware Support: If needed, involve VMware support to resolve complex VMFS issues or to interpret low-level VMFS data.
  4. Backup Data: Always have recent backups of your VMs before performing advanced troubleshooting or recovery operations.

Conclusion:

The hexdump -C examples given here are strictly hypothetical and illustrate how low-level VMFS data might appear. In real-world situations, direct examination of VMFS data structures should be performed with caution and preferably under the guidance of VMware support professionals.

You might use hexdump to examine a .vmx file, and what it might look like. Given that .vmx files are text-based, using -C with hexdump makes it more readable by showing the ASCII representation along with the hex dump.

Command to run hexdump on a .vmx file:

hexdump -C /vmfs/volumes/datastore_name/vm_name/vm_name.vmx

Example:

A .vmx file hexdump might look like this:

00000000  2e 65 6e 63 6f 64 69 6e  67 20 3d 20 22 55 54 46  |.encoding = "UTF|
00000010  2d 38 22 0a 63 6f 6e 66  69 67 2e 76 65 72 73 69  |-8".config.versi|
00000020  6f 6e 20 3d 20 22 38 22  0a 76 69 72 74 75 61 6c  |on = "8".virtual|
00000030  48 57 2e 76 65 72 73 69  6f 6e 20 3d 20 22 37 22  |HW.version = "7"|

Explanation:

  • The -C option is showing the ASCII representation of the .vmx file’s contents along with their hexadecimal values.
  • This hypothetical output represents readable ASCII characters because .vmx files are plain text files.

Steps to view .vmx files more conveniently:

  1. SSH into the ESXi host or access the ESXi Shell.
  2. Navigate to the directory containing the .vmx file, usually in /vmfs/volumes/[DatastoreName]/[VMName]/.
  3. Use a text viewer or editor like vi to read or modify it:
vi /vmfs/volumes/datastore_name/vm_name/vm_name.vmx

Important Note:

When modifying .vmx files, ensure you know the implications of the changes being made, as incorrect configurations can lead to issues with VM operation. Always back up the original .vmx file before making any changes to it. And typically, modifications to .vmx files are usually done with the VM powered off to avoid conflicts and ensure the changes are recognized when the VM is powered on next.

Leave a comment