Resolving ESXi Fatal Error 8 (Device Error) After Moving to NFS

The situation

I run ESXi as a virtual machine in my lab, so there are two distinct layers involved: the physical ESXi host running the nested ESXi VM, and the nested ESXi instance running workloads such as vCenter Server Appliance (VCSA).

During some networking tests, I had moved a VM to another host because I was troubleshooting connectivity to my lab. I eventually found that the network problem was on my Windows workstation: I had two routes for the same lab destination. One route pointed to my normal default gateway, while the correct route pointed to the lab virtual router that also handles DNS for the environment.

The original connectivity issue was routing, not compute capacity.
Once I corrected the duplicate route, the network path through my lab virtual router worked normally again.

At about the same time, however, I made two additional changes. I moved the nested ESXi VM from local storage to a newly provisioned NFS datastore, and I increased the nested host's virtual CPU and memory because I thought resource pressure might also be contributing to the symptoms.

The fatal boot error

On the first reboot after those changes, the nested ESXi VM stopped during its early boot sequence. The console loaded several boot modules, reached k.b00, and then displayed:

Fatal error: 8 (Device error)

VMware ESXi console showing Fatal error: 8 Device error during boot — The nested ESXi boot loader stopped after loading `k.b00` and reported `Fatal error: 8 (Device error)`.

Because this happened immediately after I added vCPU and memory, I initially associated the failure with the virtual hardware change. The timing seemed convincing, but it was only a coincidence.

Why the first diagnosis was misleading

Visible change I increased vCPU and memory immediately before rebooting the nested ESXi VM.

Visible symptom ESXi stopped early in its boot sequence with a device error.

Actual differentiator The VM's backing storage had also changed from local storage to NFS.

I tried upgrading or reinstalling ESXi, but the same error remained. That was an important clue. If changing the ESXi installation did not alter the failure, I needed to look below the guest operating system and at the infrastructure presenting its virtual disk.

The error occurred while the ESXi boot loader was reading modules from its boot device. In this incident, the nested ESXi guest saw a device-read failure, but the relevant “device” was ultimately backed by the outer hypervisor's NFS datastore.

Important distinction: this result does not mean nested ESXi can never run from NFS. It means this NFS-backed placement or its I/O path was the failing variable in my incident. Storage availability, latency, timeouts, server performance, or the backing datastore implementation can all affect a VM before its guest OS is fully running.

How I isolated the cause

I worked backward through the changes instead of continuing to modify the ESXi installation.

I separated the incidents.
The Windows routing problem explained the earlier connectivity failure. It did not explain why the nested ESXi boot loader could no longer read its boot modules.

I stopped treating the resource change as proven cause.
The added vCPU and memory happened near the failure, but there was no evidence in the console message that CPU or RAM allocation was the failing component.

I considered the storage migration.
The nested ESXi VM had also moved from known-good local storage to a new NFS datastore. That change affected every boot read made by the VM.

I moved the nested ESXi VM back to local storage.
This restored the previous, known-good backing storage path.

The VM booted correctly.
No ESXi reinstall was required, and the additional vCPU and memory were not the cause.

The resolution

I relocated the nested ESXi VM from the newly provisioned NFS datastore back to local storage on the physical ESXi host. After returning it to local storage, the nested ESXi instance booted normally with the added CPU and memory still configured.

Root cause confirmed by rollback: the boot failure followed the NFS-backed placement, not the virtual hardware expansion. Returning the same VM to local storage restored normal operation.

That comparison was much stronger evidence than the timing of the first failed reboot. The resource change and storage migration had been made close together, but only reversing the storage change resolved the error.

What I would check before using the NFS datastore again

Moving the VM back to local storage was the immediate recovery. Before placing critical or nested infrastructure on that NFS datastore again, I would validate the complete path:

NFS availability Confirm the NFS service remains running, the export stays mounted, and the server is not hosted on storage that depends on its own export.

I/O and latency Check the physical host and NFS server for latency, timeouts, disk contention, queueing, packet loss, and filesystem capacity.

Network consistency Verify MTU, routing, link stability, and NIC error counters across the ESXi-to-NFS path.

I would also test the datastore with a noncritical VM through multiple cold boots and sustained I/O before trusting it with management appliances or nested hypervisors.

Lessons from the incident

The biggest lesson was not specific to ESXi or NFS: when several changes happen in the same window, the change made closest to the failure is not automatically the cause.

Change one major variable at a time.
Storage placement and virtual hardware should ideally be changed and validated separately.

Read the failure at the correct layer.
A guest “device error” can originate from the hypervisor or backing datastore rather than from the guest's virtual hardware configuration.

Use rollback as a diagnostic test.
Returning to known-good local storage provided a clean comparison and quickly isolated the failing variable.

Keep dependencies out of circular storage designs.
An NFS server VM should not live on the datastore it exports, and critical management services need a storage path that remains available during recovery.

In the end, the additional compute resources were fine. The ESXi image was fine. The problem was the new NFS-backed placement, and moving the nested ESXi VM back to local storage brought it online immediately.