When designing a virtualization infrastructure, most attention usually goes to the hosts’ CPU and RAM, but the fate of your VMs depends more than anything on storage. If storage is slow, every server is slow, and if it fails, they all stop together. The right choice depends on the number of hosts, your need for high availability and your budget.
Local storage
The disks sit inside the host itself, usually with hardware RAID or, in Proxmox, with ZFS.
- Pros: Simple, inexpensive and very fast with SSDs; no dependence on the network.
- Limitations: If the host fails, its VMs are unavailable. Moving VMs between hosts is slower because the entire disk has to be transferred.
- Best for: One or two hosts in small companies, combined with regular backups or replication to a second host.
NAS (file-based network storage)
VM disks are stored as files on network storage, using NFS or, in Hyper-V, SMB 3.
- Pros: Simple management, all hosts accessing one shared storage system, and support for live migration and HA.
- Limitations: Performance depends on the network and the device’s capabilities. Many entry-level NAS devices are built for backup and file sharing, not for running dozens of VMs at once.
SAN (block storage)
Storage is presented to the hosts as block disks over iSCSI or Fibre Channel.
- Pros: High performance and reliability, dual controllers on enterprise-grade arrays and broad support across all platforms.
- Limitations: Higher cost and a need for expertise. An array with a single controller or a single network path is itself a single point of failure for every VM.
Ceph and distributed storage
With Ceph, the disks inside the hosts together form shared, distributed storage, and data is replicated across several hosts. Proxmox supports it natively. VMware and Microsoft have their own similar solutions, whose licensing terms need to be reviewed separately.
- Pros: No separate storage appliance, tolerance for disk and even host failures, and growth by adding hosts.
- Limitations: It requires at least three hosts, a fast dedicated network and careful design. Ceph manages disks directly, so don’t put them behind hardware RAID. Troubleshooting it also requires its own specialized knowledge.
Whatever storage you choose, backups must not live on the same device. Shared storage is also a shared point of failure.
Design tips that matter for every option
- Measure speed and latency, not just capacity: For running VMs, SSD has become almost the standard.
- Choose the right RAID level: For running VMs, RAID 10 usually performs better than RAID 5; rebuilding a RAID 5 array with large disks is also long and risky.
- Separate the storage network: Put iSCSI, NFS or Ceph traffic on a dedicated network or VLAN, and if you enable jumbo frames, make sure they are consistent along the entire path.
- Have a second path: Use two network cards and two switches for storage connectivity so a single cable failure doesn’t stop everything.
Checklist
- The storage type matches the number of hosts and the HA requirements.
- Shared storage has no single point of failure.
- Disk performance has been measured under the real VM workload.
- The storage network is separate and has a second path.
- Disk health and free space are monitored.
- Backups are kept on a different device.
Choose your storage right once, and you’ll avoid years of trouble. ITPlus724 engineers can design and implement the right storage for your company based on your services and budget.