Pulling back from the archives this is a repost of a previous blog post. This time ripped from a guest spot at The Nubby Admin, a fantastic blog from a fellow tech nerd.
Let's face it. NFS is a magical thing. It allows you to centralize your storage, share volumes across systems, and all while maintaining sane permissions and ownership. Unfortunately, it can also be a bit of a fickle beast. Let's say you just had your volume configured and you set up the mounts. You go and run this command:
mount -t nfs 10.10.10.1:/vol1/fs1 /data
Works like a champ, you now have your data partition mounted over NFS. So you
add this line to your
/etc/fstab and make it mount automagically.
10.10.10.1:/vol1/fs1 /data nfs defaults 0 0
A few weeks go by and you apply a kernel update. No big deal, you apply the updates and during your next maintenance window reboot to apply the new kernel. Then you start to see applications failing and notice the volume isn't actually mounted. This is an unfortunate result of the automounter subsystem.
It's like this. At boot time the root partition gets mounted, automounter reads
/etc/fstab file, and boots any filesystem that doesn't have
noauto as a
mount option. Since we're still very early in the boot process the network isn't
up yet, so naturally any network filesystems fail. The real problem here is that
at no point does automounter go back and attempt to remount those systems. So
your NFS mount points fail because there is no network, and done is done.
The developers were nice enough to provide a fix for this. There exists a mount
_netdev. If we quote directly from the man page (sourced from
_netdev The filesystem resides on a device that requires network access (used to prevent the system from attempting to mount these filesystems until the network has been enabled on the system).
This is awesome, and exactly what we want. So you modify your entry in fstab to look like this:
10.10.10.1:/vol1/fs1 /data nfs defaults,_netdev 0 0
You've been bitten by NFS mounting in the past so you throw this in your test
environment and reboot immediately. After the system comes up you notice a
problem. Your NFS volumes are still unmounted. You see, there's a bit of a
hitch. Automounter followed the same procedure that it did before, except this
time it didn't even attempt to mount
_netdev option doesn't tell
the system to mount the filesystem when network comes up, it says don't attempt
to mount it at all if the network isn't up. There is still a missing piece to
the puzzle. If you look at your init scripts there is a service called
If you read the script you can see in the chkconfig header this description:
# description: Mounts and unmounts all Network File System (NFS), \ # CIFS (Lan Manager/Windows), and NCP (NetWare) mount points.
This is exactly what you need. It is a service whose sole purpose is to read
/etc/fstab and mount network filesystems. All you have to do is enable it
chkconfig netfs on
and watch the magic happen. Now your mount boot process should look something like this:
- Automounter reads
/datasince it has
- Mounts all other filesystems
- Finishes mount jobs and allows system to continue booting
- Network comes up
- netfs reads
/etc/fstaband finds an nfs filesystem
- netfs mounts
What's funny is that while I was researching this problem I never stumbled across netfs as a service. I had even gone so far as to start planning out my own custom init script that would do exactly this, except specifically for my mount points instead of generalizing. It's nice to see that I was on the right track, but even better that the tools already existed.