Group sharing on a FreeBSD home server

My partner and I share a home server for our storage needs, running on FreeBSD 12 with ZFS. We have our own users, delan and aria, and a group (delanria) that in theory we can use for common areas, like our software collection. Services like torrent clients and media libraries have their own users too, and all of these users need to write to things others have created. This was easier said than done.

• Unix permissions 101
  • Ownership and creation
• Shared storage
• Login classes
• Fixing sudo(8)
• Fixing services
• Fixing samba(8)
• Funny execute bits
• What did we learn?

Unix permissions 101

Files are owned by a user (the owner) and a group (the group). Users belong to groups, and each group they join grants them access to any files owned by that group. Processes run as a user (euid) and a group (egid), more or less.

Traditional permissions for files consist of twelve bits: nine for whether the owning user (u), owning group (g), and others (o) can read (r), write (w), or execute (x); three for controlling execution behaviour. Of the latter three, setuid (u+s) makes the file run with its owning user as euid, setgid (g+s) does that for owning group and egid, and sticky (t) was mostly only used historically.

Directories repurpose the execute bits for “search”, which essentially means whether you can blindly access children (subject to the children’s permissions). This is distinct from the read bits, which control whether you can list children, and write bits, for creating and deleting children. They also repurpose the sticky bit for “restricted deletion”, where children can only be deleted by their owners, rather than anyone who can write to the directory (useful for /tmp).

This permission model frankly sucks. It’s clear that these twelve bits are a messy and leaky abstraction over filesystem access rights. But I’ve never used ACL:s outside of Windows, and I don’t intend to change that any time soon. I know that ls(1) indicates ACL:s with a plus (drwxrwxrwx+), that there are apparently “POSIX” and “NFS” flavours of ACL:s, and… that’s pretty much it.

Ownership and creation

When you create a file, it’s owned by you (euid), but the owning group depends. Unfortunately, sometimes it’s your “current” group (egid), which is controlled by newgrp(1) and defaults to your “login” group, which is usually something like “users” or “staff” or a group with the same name as your user.

FreeBSD makes group ownership of directories easy, because new files always inherit the owning group, so that “sometimes” is never! But on Linux, they’re only inherited when the parent is setgid, and otherwise take their owning group from you (egid).

Unix requires the nine main permission bits upfront when creating a file or directory, and convention is for programs that don’t know or care about these bits to give 666₈ (u+rw, g+rw, o+rw) or 777₈ (u+rwx, g+rwx, o+rwx) respectively.

// create a directory (u+rwx, g+rwx, o+rwx)
mkdir("path/to/foo", 0777);
// create an executable file (u+rwx, g+rwx, o+rwx)
creat("path/to/bar", 0777);
// create a non-executable file (u+rw, g+rw, o+rw)
creat("path/to/baz", 0666);

The bits are then filtered by the umask, an environmental setting that almost always defaults to 022₈ (g-w, o-w). This is where the common permissions of 644₈ (u+rw, g+r, o+r) for files and 755₈ (u+rwx, g+rx, o+rx) for directories comes from.

$ ls -l path/to
drwxr-xr-x  [...]  foo
drwxr-xr-x  [...]  bar
drw-r--r--  [...]  baz

Shared storage

Let’s say we have some groups.

$ rg delan,aria /etc/group
wheel:*:0:root,delan,aria
delanria:*:1003:delan,aria
sonarr:*:351:delan,aria
radarr:*:352:delan,aria
_sabnzbd:*:350:delan,aria,sonarr,radarr
qbittorrent:*:850:delan,aria,sonarr,radarr

Let’s also say I have a notes directory and a software directory, both of which are shared with my partner including write permissions (g+w).

$ ls -la /ocean/notes
drwxrwxr-x  [...]  delan  delanria  [...]  .

$ ls -la /ocean/software
drwxrwxr-x  [...]  delan  delanria  [...]  .
drwxrwxr-x  [...]  delan  delanria  [...]  accounting
drwxrwxr-x  [...]  delan  delanria  [...]  benchmarks
drwxrwxr-x  [...]  delan  delanria  [...]  drivers
                                           ...

When adding a new note, or adding a category of software, the immediate problem we run into is that a umask of 022₈ (g-w, o-w) makes the new directory group-read-only (g+rx).

delan@storage$ echo bread > /ocean/notes/buy
delan@storage$ mkdir /ocean/software/fonts
aria@storage$ echo soul > /ocean/notes/sell
aria@storage$ mkdir /ocean/software/games

$ ls -ld /ocean/notes/{,buy,sell}
drwxrwxr-x  [...]  delan  delanria  [...]  .
-rw-r--r--  [...]  delan  delanria  [...]  buy
-rw-r--r--  [...]  aria   delanria  [...]  sell

$ ls -ld /ocean/software/{,fonts,games}
drwxrwxr-x  [...]  delan  delanria  [...]  .
drwxr-xr-x  [...]  delan  delanria  [...]  fonts
drwxr-xr-x  [...]  aria   delanria  [...]  games

This prevents the other partner from changing notes or adding software, unless we periodically “fix up” the permissions in common areas.

delan@storage$ echo out >> /ocean/notes/sell
zsh: permission denied: /ocean/notes/sell

aria@storage$ cd /ocean/software/fonts
aria@storage$ curl -sSO https://fonts.example.com/iosevka.otf
curl: (23) Failure writing output to destination

$ chmod -R g+w /ocean/{notes,software}

Login classes

We can avoid these problems by setting our umask to 002₈ (o-w), which is controlled by each user’s login class. In my login.conf(5), many settings are defined only in the “default” login class, including the umask, which are directly inherited by the other classes.

$ cat /etc/login.conf
default:\
	:passwd_format=sha512:\
	:...:\
	:umask=022:
standard:\
	:tc=default:
daemon:\
	:...:\
	:tc=default:
...

Put a pin in the line with “passwd_format”, we’ll need that later.

If we change the umask setting and rebuild, logging in yields the expected umask, fixing the scenarios above… except when switching users with sudo(8). Switching users with su(1) via sudo(8) works as expected, so what gives?

# vim /etc/login.conf
# cap_mkdb /etc/login.conf
# su -l delan
delan@storage$ umask
002

delan@storage$ sudo -iu aria umask
022

delan@storage$ sudo su -l aria -c 'umask'
002

Fixing sudo(8)

At first, the only way we can get the expected umask when switching users with sudo(8) is to explicitly ask for a login class, such as the user’s default class:

delan@storage$ sudo -iu aria -c - umask
002

This is because of defaults for three sudoers(5) settings: use_loginclass, umask_override, and umask. These settings mean that sudo(8) forms the new umask as follows.

use_loginclass is off, so we start by taking our umask from the environment in which sudo was invoked, in this case, 002₈. umask_override is off, so our next step will do a bitwise OR, rather than replacing our umask entirely. The umask setting is 022₈, so our final umask is 002₈ OR 022₈, which is… well… 022₈.

To fix this, we can turn use_loginclass on, which usually¹ takes our initial umask from the user’s default login class, or turn the umask setting off, which tells sudo(8) not to modify that initial umask.

Defaults use_loginclass  # option 1: use_loginclass on
Defaults !umask          # option 2: umask off

# option 3: umask off, but in a way that makes no sense
# (seriously, sudo authors, why did you add this case?
#  this just makes it impossible to actually set 0777!)
Defaults umask=0777

Fixing services

According to the rc.subr(8) manual, services ostensibly use the “daemon” login class by default. But despite setting the umask in all of our login classes to 002₈ (o-w), services like qbittorrent and sabnzbd continue to create things group-read-only. Once again, what gives?

As the manual says, the ${name}_login_class is used with ${name}_limits. The former points to a login class containing our initial set of resource limits, and the latter overrides those limits, by way of limits(1). Indeed, if we look under the hood, the login class is only ever used in the arguments passed to limits(1).

$ rg _login_class /etc/rc.subr
786:#   ${name}_login_class n   Login class to use, else "daemon".
969:        _prepend=\$${name}_prepend  _login_class=\${${name}_login_class:-daemon} \
1124:                   _doit="$_cd limits -C $_login_class $_limits $_doit"

But resource limits aren’t the only settings defined by login classes, which as we saw earlier, also says things like “passwd_format is sha512”! So this begs the question: is umask considered a resource limit for the purposes of limits(1)?

root@storage$ umask 000; ulimit 69
root@storage$ limits sh -c 'umask; ulimit'
0000
69

root@storage$ limits -C daemon sh -c 'umask; ulimit'
0000
unlimited

No. In fact, not only is the umask of the “daemon” login class not consulted when running a service, but rc(8) and init(8) themselves don’t even run in a login class. You can see the former for yourself by adding a couple of lines to /etc/rc and rebooting.

$ head -2 /etc/rc
#!/bin/sh
umask=$(umask)          # add this

$ tail -2 /etc/rc
echo "umask is $umask"  # add this
exit 0

As for the latter, you can write a script that spews out the umask, then reboot and tell loader(8) to tell init(8) to immediately exec that script.

$ sudo chmod +x /root/umask
$ cat /root/umask
#!/bin/sh
while :; do umask; done

loader> set init_exec=/root/umask
loader> boot

In both cases, the umask is 022₈ (g-w, o-w). This is because login classes aren’t magic, nor are they omnipotent! The only processes subject to them are those spawned by login(1), or things like login(1)², such as su(1), sudo(8), or sshd(8).

All other processes ultimately inherit their umask from the “kernel” process (pid 0), whose umask is hardcoded to, you guessed it, 022₈ (g-w, o-w).

// sys/kern/init_main.c
static void
proc0_init(void *dummy __unused)
{
	struct proc *p;
	// [...]
	p->p_pd = pdinit(NULL, false);
	// [...]
}

// sys/kern/kern_descrip.c
struct pwddesc *
pdinit(struct pwddesc *pdp, bool keeplock)
{
	// [...]
	newpdp->pd_cmask = CMASK;
	// [...]
}

// sys/sys/param.h
#define	CMASK	022		/* default file mask: S_IWGRP|S_IWOTH */

So getting back on task, how do we run our services with another umask? One way might be to add a line setting the umask to the beginning of /etc/rc, but this is rather drastic, and the security of this… smells questionable.

The solution I’ve settled on is to sneak a umask command into the per-service rc.conf(5) for specific services. This works because in rc.subr(5), load_rc_config executes /etc/rc.conf “if it has not yet been read in” (whatever that means), then executes /etc/rc.conf.d/foo if it exists. Most of the time, these files contain variables only, but they’re just shell scripts. It’s shell scripts all the way down.

$ ls -l /etc/rc.conf.d
-rw-r--r--  [...]  _umask
lrwxr-xr-x  [...]  qbittorrent -> _umask
lrwxr-xr-x  [...]  radarr -> _umask
lrwxr-xr-x  [...]  sabnzbd -> _umask
lrwxr-xr-x  [...]  sonarr -> _umask

$ cat /etc/rc.conf.d/_umask
umask 002

Fixing samba(8)

That rc.conf(5) hack doesn’t work for samba(8), where the umask that applies to things created by clients is controlled by internal configuration, just like sudo(8). In this case, “create mode” and “directory mode” are the settings to change.

[global]
create mode = 0775     # like umask 002
directory mode = 0775  # like umask 002

While we’re at it, if you want to set execute bits on all new files, you can use “force create mode”. Aria likes this, but I’m not so sure.

[global]
# force create mode = 0111  # like ugo+x

Funny execute bits

At this point, it looked like we were done, but something caught my eye when Aria created some files for testing. The files had owning user execute (u+x), but not the other two execute bits. For the last bloody time, what gives?

$ ls -l
-rwxrw-r--  [...]  aria  delanria  [...]  foo.txt

Turns out there’s an old samba(8) feature that repurposes the three execute bits for the three legacy DOS attributes respectively: archive, system, hidden. The setting for the first execute bit (“map archive”) is on by default, and Windows had created the file with the archive bit on, hence owning user execute (u+x)!

Nowadays extended attributes are a better way to store those attributes, which is on by default as “store dos attributes”. But I was worried that I would need to do something messy like vfs_streams_xattr(8), remembering that FreeBSD and ZFS don’t support xattrs, at least not until FreeBSD 13. After all, ZFS says that they’re not supported!

root@storage$ zfs set xattr=on ocean
property 'xattr' not supported on FreeBSD: permission denied

Turns out that error is misleading, and xattrs more or less work fine.

root@storage$ lsextattr user foo.txt
foo.txt DOSATTRIB

(archive bit only)
root@storage$ getextattr -x user DOSATTRIB foo.txt
foo.txt 00 00 04 00 04 00 00 00 51 00 00 00 20
> 00 00 00 44 a8 2c de 2d b1 d7 01 44 a8 2c de
> 2d b1 d7 01

(archive + system + hidden)
root@storage$ getextattr -x user DOSATTRIB foo.txt
foo.txt 00 00 04 00 04 00 00 00 51 00 00 00 23
> 00 00 00 44 a8 2c de 2d b1 d7 01 44 a8 2c de
> 2d b1 d7 01

(none of those attributes)
root@storage$ getextattr -x user DOSATTRIB foo.txt
foo.txt 00 00 04 00 04 00 00 00 51 00 00 00 00
> 00 00 00 44 a8 2c de 2d b1 d7 01 44 a8 2c de
> 2d b1 d7 01

To top it all off, the smb.conf(5) manual says that “store dos attributes” should have automatically disabled the execute-bit-based attribute mapping, but they actually don’t. I guess the manual was wrong.

[global]
map archive = no
map system = no
map hidden = no

What did we learn?

FreeBSD gives us owning group inheritance for free, but inheriting the group-writable bit requires changing the umask. This can be done for human use by way of the login classes in login.conf(5), with special tweaks needed for sudo(8), but services only use login classes for resource limits, not the umask, which has a hardcoded default of 022₈ (g-w, o-w).

Most services can have the desired umask set imperatively in /etc/rc.conf.d, but samba(8) needs to be configured with its own “create mask” and “directory mask” settings.

When in doubt, read the source code.