Project Sharing

This guide provides step-by-step instructions to configure two independent Flame Family workstations, without shared storage, for collaboration. Configuration differs slightly depending on the operating system of your workstations. This demonstration uses a macOS system and a Linux system to cover both cases. In these instructions, the two systems act as both servers exporting projects and clients importing projects.

For setting up a server-only system, such as a Project Server, follow Prerequisites, Setting the Hostname, Storage Configuration, and Export via NFS.

For setting up a client-only system, such as a Burn node or a remotely connected Flare, ensure it is connected to the same network and subnet as the server(s), and follow Setting the Hostname, Import via NFS, and Configure the Default Project Paths.



Prerequisites

To enable project sharing, ensure the following:



Setting the Hostname

Assigning easily identifiable hostnames to your systems is good practice, even if not strictly necessary.



Storage Configuration

By default, a project's metadata and media cache are stored under the project's home directory which resides on the system drive. For better performance and organization, it is recommended to store project metadata and media caches on separate storage volumes. Media Caches benefit from fast, high-capacity storage.

A key constraint is that shared project metadata and Media Caches must be accessible using the same paths on all hosts. We will thus use /mnt/<hostname>/projects for metadata and /mnt/<hostname>/media for media caches. Using such paths instead of going through the default automounted /hosts/<host>/... paths has the added benefit of bypassing NFS for local access on macOS. Linux always bypasses NFS when the mounted path is local.

Warning:

The file system on which projects are created requires hard links capabilities and subsecond timestamp accuracy. To ensure smooth project management workflows, the storage used by the application should not be mounted over CIFS/smbfs.



Linux Storage Setup

  1. Create a Striped Storage Volume:

    This example creates a 2TB striped logical volume over 4 500GB nvme devices. Adjust the steps for the number and size of devices you want to use.

    • Locate available storage devices:

      $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:32   0 476.9G  0 disk
├─sda1        8:33   0   200M  0 part /boot/efi
├─sda2        8:34   0     1G  0 part /boot
└─sda3        8:35   0 475.7G  0 part
 ├─rl-root 253:0    0 144.4G  0 lvm  /
 ├─rl-swap 253:1    0  31.3G  0 lvm  [SWAP]
 └─rl-home 253:3    0   300G  0 lvm  /home
nvme0n1         259:0    0  500G  0 disk
nvme1n1         259:1    0  500G  0 disk
nvme2n1         259:2    0  500G  0 disk
nvme3n1         259:3    0  500G  0 disk

    • Initialize physical volumes with pvcreate:

      $ sudo pvcreate /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1

    • Create a volume group named vg0 with vgcreate:

      $ sudo vgcreate vg0 /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1

    • Retrieve the total number of physical extents with vgdisplay:

      $ sudo vgdisplay vg0 | grep -i 'total pe'
Total PE              511996

    • Create logical volume vg0-media with lvcreate -l <#PE> -i <#PV> -I <#KB/PV/stripe>. This command creates a logical volume over the 4 physical volumes with stripes of 32KB per physical volume:

      $ sudo lvcreate -l 511996 -i 4 -I 32 -n media vg0

  2. Create an XFS File System:

    Format the logical volume with the XFS file system:

    $ sudo mkfs.xfs /dev/vg0/media
    Important:

    The file system must support timestamps with subsecond accuracy to preserve metadata integrity and ensure optimal NFS performance. This is particularly critical for NFSv3 (used by macOS servers), which relies on timestamps to update its cache. NFSv4 (Linux default) is less dependent on timestamps but still benefits from subsecond accuracy.

  3. Mount the Media Cache Storage

    • Create the mount point with mkdir:

      $ sudo mkdir -p /mnt/penguin/media

    • Add the following line to /etc/fstab to make the mount point persistent:

      /dev/vg0/media   /mnt/penguin/media   xfs   rw,noatime,inode64,nofail

    • Reload systemd and mount the volume"

      $ sudo systemctl daemon-reload
$ sudo mount /mnt/penguin/media

  4. Set Permissions

    Make the mount point accessible to all users (refer to File Security for more information on secure workflows):

    $ sudo chmod 777 /mnt/penguin/media

  5. Create the Project Metadata Folder

    Create the projects folder with open permissions (refer to File Security for more information on secure workflows):

    $ sudo mkdir -m 777 /mnt/penguin/projects


macOS Storage Setup

  1. Add a Storage Volume

    Connect a storage device to the macOS and use the "Disk Utility" to configure a volume on it. Name the volume "Media HD" and format it with a case-sensitive APFS file system (recommended). The volume will be mounted under /Volumes/"Media HD".

    Important:

    The file system must support timestamps with subsecond accuracy to preserve metadata integrity and ensure optimal NFS performance. This is particularly critical for NFSv3 (used by macOS servers), which relies on timestamps to update its cache. NFSv4 (Linux default) is less dependent on timestamps but still benefits from subsecond accuracy. Consequently, HFS+ cannot be used.

    Note: macOS is restrictive when it comes to mounting file systems in specific locations1. The following steps work around these restrictions.

  2. Add mnt to synthetic.conf

    macOS, with System Integrity Protection enabled, does not allow creating a folder at the root of the file system. To work around this, create a mnt directory under /System/Volumes/Data:

    $ sudo mkdir /System/Volumes/Data/mnt

    and add the following line to /etc/synthetic.conf. It will create a symbolic link from /mnt to /System/Volumes/Data/mnt.

    mnt     System/Volumes/Data/mnt
    Important: Use a tab character to separate the two fields.
    Note: Changes to synthetic.conf take effect after a system reboot.

  3. Create Sub-directory and Link

    • Create the <hostname> sub-directory:

      $ sudo mkdir /System/Volumes/Data/mnt/joker

    • Create a symbolic link for the media folder:

      $ sudo ln -s "../../Volumes/Media HD" /System/Volumes/Data/mnt/joker/media

  4. Create the Project Metadata Folder

    Create the projects folder with open permissions (refer to File Security for more information on secure workflows):

    $ sudo mkdir -m 777 /System/Volumes/Data/mnt/joker/projects

  5. Reboot

    Reboot the system to apply changes made to synthetic.conf:

    $ sudo reboot

  6. Verify the Mount Point

    Create a dummy folder under /mnt/joker/media and confirm that it exists on the new volume. Remove the folder when done:

    $ mkdir /mnt/joker/media/dummy
$ ls -ld /Volumes/Media\ HD/dummy
drwxr-xr-x  2  andreg   staff  64  7 avr 13:26 /Volumes/Media HD/dummy
$ rmdir /mnt/joker/media/dummy

  7. Set Permissions

    Set the mount point permissions to make it accessible to Flame users on the network. For simplicity, make it accessible to all users (Refer to File Security for more information on secure workflows):

    $ sudo chmod 777 /mnt/joker/media

1: The fstab documentation states that you must use the volume label or UUID to mount an APFS file system under a specific directory at boot. However, our tests have shown that local file system entries in /etc/fstab are often ineffective. While diskutil allows you to mount the file system in a specific location, the configuration is lost after a reboot. Using a symbolic link has proven to be a more reliable solution.



Export via NFS

To enable project sharing, the Project Home and Media Cache folders must be exported to make them accessible to other hosts in your workgroup. This section provides instructions for configuring NFS exports on both Linux and macOS. Refer to the exports(5) man page for additional information that may be applicable to your environment.

NFS Version Considerations: Linux uses NFSv4 by default, while macOS NFS servers are limited to NFSv3. In mixed environments (Linux and macOS), connections to macOS servers will use NFSv3, as the server dictates the protocol version. Connections to Linux servers can use NFSv4 regardless of the client platform.



Linux

  1. Edit /etc/exports

    Configure the export paths for the Project Homes and Media Caches. Use the async option for Media Caches to improve performance. The sync option is required for Project Homes to ensure data integrity.

    /mnt/penguin/projects 192.168.0.0/24(rw,sync)
/mnt/penguin/media 192.168.0.0/24(rw,async)

  2. Apply the Changes

    Run the following command to apply the changes:

    $ sudo exportfs -vra


macOS

  1. Enable Editing of /etc/exports

    Note: /etc/exports is often protected to prevent macOS updates from resetting its contents.

    Use chflags to make the file editable:

    $ sudo chflags nosimmutable /etc/exports

  2. Edit /etc/exports

    Add the following lines to export the Project Homes and Media Caches:

    /System/Volumes/Data/mnt/joker/projects /mnt/joker/projects -network 192.168.0.0 -mask 255.255.255.0
/System/Volumes/Data/Volumes/Media\ HD /mnt/joker/media -network 192.168.0.0 -mask 255.255.255.0

  3. Verify Your Changes

    Run nfsd checkexports to validate the exports:

    $ nfsd checkexports
exports:1: path contains non-directory or non-existent components: /mnt/joker/projects
exports:2: path contains non-directory or non-existent components: /mnt/joker/media
    Note: The above messages can be ignored. They appear because /mnt is a symbolic link. They do not prevent the exports.

    If you see nfsd is not running message, enable NFS:

    $ sudo nfsd enable

    Repeat the export verification afterwards.

  4. Apply the Changes

    Update the NFS configuration:

    $ sudo nfsd update

  5. Make /etc/exports Immutable Again

    Protect the file to prevent accidental changes:

    $ sudo chflags simmutable /etc/exports


Import via NFS

To enable project sharing, each host in the workgroup must import the Project Homes and Media Caches of other workgroup hosts. This ensures that all hosts can interact with shared projects seamlessly.

NFS Version in Use: When connecting to a Linux server, NFSv4 will be used. When connecting to a macOS server, NFSv3 will be used, as macOS NFS servers are limited to NFSv3.



Linux

  1. Configure Direct Mounts

    Add the following line to /etc/auto.master with your favourite text editor:

    /-  auto.direct

    Then, edit /etc/auto.direct and add these lines:

    /mnt/joker/projects -lookupcache=pos  192.168.0.21:/mnt/joker/projects
/mnt/joker/media                      192.168.0.21:/mnt/joker/media
    Important:

    Project metadata mounts must include the lookupcache=pos option to preserve metadata integrity. The NFS client cache has difficulty staying in sync without this option whenever a file is hard-linked or a file/directory is renamed. lookupcache=pos is preferred on Linux because it has minimal impact on performance as opposed to using acdirmin=0 and acdirmax=0.

    Note: If the remote host has a dedicated IP address for media transfers, you can specify it in this file. For example, if the media transfer IP address of the remote host is 192.168.0.22, you can use:

    /mnt/joker/media 192.168.0.22:/mnt/joker/media

    This ensures that media transfers are routed through the dedicated IP address, improving performance and isolating traffic.

  2. Apply the Changes

    Restart the autofs service:

    $ sudo systemctl restart autofs

  3. Validate the Mount Options

    Execute the following command and verify the lookupcache=pos option is active for the Project Home mount:

    $ nfsstat -m

  4. Work Around the macOS NFS Mount Complications

    macOS NFS servers will stop responding to Linux clients if they are idle for 5 minutes or more unless specific conditions are met. For instance, having the same macOS storage mounted on a second macOS system eliminates the issue. A robust, albeit inelegant, solution to this problem is to access the mount point more than once every 5 minutes. This can be achieved with a scheduled job.

    Use cron to schedule a task every 4 minutes, regardless of user login status. cron runs system-wide and does not require a user to be logged in.

    Example: To run a command every 4 minutes, add a similar line to the system crontab (/etc/crontab). Replace /mnt/joker/media and /mnt/joker/projects with your own paths:

    */4 * * * * root /bin/stat /mnt/joker/media /mnt/joker/projects > /dev/null
    Note: Refer to the crontab(5) man page for details about the syntax of the crontab file:
    $ man 5 crontab


macOS

  1. Enable Editing of /etc/auto_master

    Note: /etc/auto_master is often protected to prevent macOS updates from resetting its contents.

    Use chflags to make the file editable:

    $ sudo chflags nosimmutable /etc/auto_master

  2. Configure Direct Mounts

    On macOS 26 and later, add the following line to /etc/auto_master:

    /-      auto_direct     -nobrowse
    Note:

    The resvport option is required to import from Linux on macOS. On macOS 15 and earlier, set resvport in auto_master, not in the direct mapping. 

    /-      auto_direct     -nobrowse,resvport

    Then, edit /etc/auto_direct and add these lines (if on macOS 15 or earlier, see this Important note below, if on macOS 14 or earlier, see this additional Important note below):

    /mnt/penguin/projects -acdirmin=0,acdirmax=0,resvport 192.168.0.11:/mnt/penguin/projects
/mnt/penguin/media    -resvport                       192.168.0.11:/mnt/penguin/media
    Important:

    Project metadata mounts must include the acdirmin=0 and acdirmax=0 options to preserve metadata integrity. The NFS client cache has difficulty staying in sync without these options whenever a file is hard-linked or a file/directory is renamed.

    Important:

    (macOS 15 or earlier) Set resvport in auto_master, and omit it from auto_direct entries: 

    /mnt/penguin/projects -acdirmin=0,acdirmax=0 192.168.0.11:/mnt/penguin/projects
/mnt/penguin/media                           192.168.0.11:/mnt/penguin/media

    Important:

    (macOS 14 or earlier) automount does not support synthetic symbolic links on these OS versions. To work around this limitation and make /mnt/penguin/projects and /mnt/penguin/media locally accessible, specify the physical paths instead: 

    /System/Volumes/Data/mnt/penguin/projects -acdirmin=0,acdirmax=0 192.168.0.11:/mnt/penguin/projects
/System/Volumes/Data/mnt/penguin/media                           192.168.0.11:/mnt/penguin/media

    Note: If the remote host has a dedicated IP address for media transfers, you can specify it in this file. For example, if the media transfer IP address of the remote host is 192.168.0.12, you can use:

    /mnt/penguin/media 192.168.0.12:/mnt/penguin/media

    This ensures that media transfers are routed through the dedicated IP address, improving performance and isolating traffic.

  3. Apply the Changes

    Refresh the automount configuration:

    $ sudo automount -c

  4. Validate the Mount Options

    Execute the following command and verify the acdirmin=0 and acdirmax=0 options are active for the Project Home mount:

    $ nfsstat -m

  5. Make /etc/auto_master Immutable Again

    Protect the file to prevent accidental changes:

    $ sudo chflags simmutable /etc/auto_master


Configure the Default Project Paths

To simplify project sharing, configure the default project paths in the sysconfig.cfg file. This ensures consistent paths for project metadata and media caches across all hosts in the workgroup. Refer to Centralised Components for additional details.

Under configuration/settings/project_folders, set the following values:

...
         "project_folders": {
            "default_home": "/mnt/<hostname>/projects/<project name>",
            "default_remote_home": "/mnt/<hostname>/projects/<project name>",
            "default_setups_dir": "<project home>/setups",
            "default_media_cache": "/mnt/<hostname>/media/<project name>"
         }
...