nfs4, ldap authentication, and idmapd

We’re trying not to use ‘old stuff’ as we’re building out our new cluster, but we have a big need for nfs or some other ad-hoc shared filesystem designed for high i/o on content servers. We’d been using ocfs2, but it’s slower than molasses and doesn’t scale n-ward as you increase the number of systems attached to a filesystem (due to the need for a journal for each node), whereas we can mount as many as we can support if our nfs server’s hardware will tolerate it.

Anyway, so the preference against ‘old stuff’ means that we’d shy away from the nfs v2 and v3 that are well-documented and stable on linux, and towards the hairy, thorny wilds of nfs 4. There’s a multitude of websites about nfs4, but they all seem to be incomplete or to apply to Solaris’s implementation, which is thorough and well-documented.

And don’t mistake me, nfs4 does run. It runs with TCP, it runs quickly, and we haven’t run into any issues save one — mapping users between servers. With nfsv3, between two servers it just works. With nfs4, you have to have a shared user authentication system and idmapd has to be running and configured correctly.

Idmapd is essentially undocumented on linux. Or, if there is documentation, I have not been able to find it. There is a man page giving basic options for the daemon. It *seems* that a configuration file syntax guide is living in /usr/share/doc/packages/nfsidmap/README, but I can’t verify that. The configuration file man page states that only Nobody-User and Nobody-Group are permitted in the [Mapping] area.

For what it’s worth, the following configuration is working for me on SLES11.

1. Get some sort of shared authentication working between the servers. Since we’re a Novell shop, we’re doing it on eDirectory with the ‘linux user’ option enabled on the accounts, which assigns a uid and gid to the user.
2. Set the idmapd.conf to have the same domain on each server. (ours, predictably, is ‘tamu.edu’).
3. Add the mount point to /etc/exports on the server. Don’t forget that if you’re using nfs4 you need to bind the mount point on the server inside of the pseudofilesystem, and then set the pseudofs in the /etc/exports file as fsid=0. Start the server, and make sure that idmapd runs.
4. On the client, set the domain in the idmapd to your domain, add the mount point to the fstab and then start the nfs service. Double check to make sure that idmapd is running.

Notes:

• I have root squashing enabled, but don’t set the nobody-user or nobody-group on the server. I don’t know what effect it will have if I did… haven’t tried. Need to move on.
• SLES10sp2 doesn’t start idmapd when you start the nfs service; you need to set it to start manually.
• You could probably manually create users and manually set their uid/gid specifically. Again, I did not try this since we already had a solution in place to manage it. We just run our web servers and other clients as specific users that are defined in our ldap tree but disallowed logins. As a side bonus, our Novell logging infrastructure logs attempted logins/accesses for those user IDs.

Write a file as a user to the nfs mount on the client, and then check it on the server (and vice versa). It should show up as the same uid and gid — if you see an exceptionally long one, or get ‘nobody’, it’s not working for you. I’m sorry, but I don’t have time right now to hack around and try different ways to get it to work!