Share

Project Sharing

This guide provides step-by-step instructions to configure two independent Flame Family workstations, without shared storage, for collaboration. Configuration differs slightly depending on the operating system of your workstations. This demonstration uses a macOS system and a Linux system to cover both cases.



Pre-Requisites

To enable project sharing, ensure the following:

  • Both systems must be connected to the same network and be on the same subnet for automatic service discovery to work. Since both systems will act as servers to each other, each should have a network interface with a static IP address.

    For this example, the two systems are configured as follows:

    Name OS IP Address Subnet Mask
    penguin Rocky 9 192.168.0.11 192.168.0.0/24 255.255.255.0
    joker macOS 15 192.168.0.21 192.168.0.0/24 255.255.255.0

  • You must have administrative access on both systems.



Setting the Host Name

Assigning easily identifiable host names to your systems is good practice, even if not strictly necessary. If your local network does not have a Domain Name System (DNS), you may also want to add the other hosts to the local /etc/hosts file. This allows you to refer to the systems by name instead of IP address.

macOS

sudo scutil --set HostName joker

Rocky Linux

sudo hostnamectl hostname penguin


Storage Configuration

By default, a project's metadata and media cache are stored under the project's home directory which resides on the system drive. For better performance and organization, it is recommended to store project metadata and media caches on separate storage volumes. Media Caches benefit from fast, high-capacity storage.

A key constraint is that shared project metadata and Media Caches must be accessible using the same paths on all hosts. We will thus use /mnt/<host name>/projects for metadata and /mnt/<host name>/media for media caches. Using such paths instead of going through the default automounted /hosts/<host>/... paths has the added benefit of by-passing NFS for local access on macOS. Linux always by-passes NFS when the mounted path is local.

Warning: The file system on which projects are created requires hard links capabilities and sub-1 second timestamp granularity. To ensure smooth project management workflows, the storage used by the application should not be mounted over CIFS/smbfs.

Linux Storage Setup

  1. Create a Striped Storage Volume: This example creates a 2TB striped logical volume over 4 500GB nvme devices. Adjust the steps for the number and size of devices you want to use.

    • Locate available storage devices:

      $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:32   0 476.9G  0 disk
├─sda1        8:33   0   200M  0 part /boot/efi
├─sda2        8:34   0     1G  0 part /boot
└─sda3        8:35   0 475.7G  0 part
 ├─rl-root 253:0    0 144.4G  0 lvm  /
 ├─rl-swap 253:1    0  31.3G  0 lvm  [SWAP]
 └─rl-home 253:3    0   300G  0 lvm  /home
nvme0n1         259:0    0  500G  0 disk
nvme1n1         259:1    0  500G  0 disk
nvme2n1         259:2    0  500G  0 disk
nvme3n1         259:3    0  500G  0 disk

    • Initialize physical volumes with pvcreate:

      $ sudo pvcreate /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1

    • Create a volume group named vg0 with vgcreate:

      $ sudo vgcreate vg0 /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1

    • Retrieve the total number of physical extents with vgdisplay:

      $ sudo vgdisplay vg0 | grep -i 'total pe'
Total PE              511996

    • Create logical volume vg0-media with lvcreate -l <#PE> -i <#PV> -I <#KB/PV/stripe>. This command creates a logical volume over the 4 physical volumes with stripes of 32KB per physical volume:

      $ sudo lvcreate -l 511996 -i 4 -I 32 -n media vg0

  2. Create an XFS File System:

    Format the logical volume with the XFS file system:

    $ sudo mkfs.xfs /dev/vg0/media

  3. Mount the Media Cache Storage

    • Create the mount point with mkdir:

      $ sudo mkdir -p /mnt/penguin/media

    • Add the following line to /etc/fstab to make the mount point persistent:

      /dev/vg0/media   /mnt/penguin/media   xfs   rw,noatime,inode64,nofail

    • Reload systemd and mount the volume"

      $ sudo systemctl daemon-reload
$ sudo mount /mnt/penguin/media

  4. Set Permissions

    Make the mount point accessible to all users (refer to File Security for more information on secure workflows):

    $ sudo chmod 777 /mnt/penguin/media

  5. Create the Project Metadata Folder

    Create the projects folder with open permissions (refer to File Security for more information on secure workflows):

    $ sudo mkdir -m 777 /mnt/penguin/projects


macOS Storage Setup

  1. Add a Storage Volume

    Connect a storage device to the Mac and use the "Disk Utility" to configure a volume on it. Name the volume "Media HD" and format it with a case sensitive APFS file system. The volume will be mounted under /Volumes/"Media HD".

    Note: macOS is restrictive when it comes to mounting file systems in specific locations1. The following steps work around these restrictions.

  2. Add mnt to synthetic.conf

    macOS, with System Integrity Protection enabled, does not allow creating a folder at the root of the file system. To work around this, create a mnt directory under /System/Volumes/Data:

    $ sudo mkdir /System/Volumes/Data/mnt

    and add the following line to /etc/synthetic.conf. It will create a symbolic link from /mnt to /System/Volumes/Data/mnt.

    mnt     System/Volumes/Data/mnt
    Important: Use a tab character to separate the two fields.
    Note: Changes to synthetic.conf take effect after a system reboot.

  3. Create Sub-directory and Link

    • Create the <host name> sub-directory:

      $ sudo mkdir /System/Volumes/Data/mnt/joker

    • Create a symbolic link for the media folder:

      $ sudo ln -s "../../Volumes/Media HD" /System/Volumes/Data/mnt/joker/media

  4. Create the Project Metadata Folder

    Create the projects folder with open permissions (refer to File Security for more information on secure workflows):

    $ sudo mkdir -m 777 /System/Volumes/Data/mnt/joker/projects

  5. Reboot

    Reboot the system to apply changes made to synthetic.conf:

    $ sudo reboot

  6. Verify the Mount Point

    Create a dummy folder under /mnt/joker/media and confirm that it exists on the new volume. Remove the folder when done:

    $ mkdir /mnt/joker/media/dummy
$ ls -ld /Volumes/Media\ HD/dummy
drwxr-xr-x  2  andreg   staff  64  7 avr 13:26 /Volumes/Media HD/dummy
$ rmdir /mnt/joker/media/dummy

  7. Set Permissions

    Set the mount point permissions to make it accessible to Flame users on the network. For simplicity, make it accessible to all users (Refer to File Security for more information on secure workflows):

    $ sudo chmod 777 /mnt/joker/media

1: The fstab documentation states that you must use the volume label or UUID to mount an APFS file system under a specific directory at boot. However, our tests have shown that local file system entries in /etc/fstab are often ineffective. While diskutil allows you to mount the file system in a specific location, the configuration is lost after a reboot. Using a symbolic link has proven to be a more reliable solution.



Export via NFS

To enable project sharing, the Project Home and Media Cache folders must be exported to make them accessible to other hosts in your workgroup. This section provides instructions for configuring NFS exports on both Linux and macOS. It is recommended to use NFSv4 servers, or at minimum, NFSv3. At the time of writing, NFSv4 is the default for Linux, while macOS is limited to NFSv3, which is also its default.



Linux

  1. Edit /etc/exports

    Configure the export paths for the Project Homes and Media Caches. Use the async option for Media Caches to improve performance. The sync option is required for Project Homes to ensure data integrity.

    /mnt/penguin/projects 192.168.0.0/24(rw,sync)
/mnt/penguin/media 192.168.0.0/24(rw,async)

  2. Apply the Changes

    Run the following command to apply the changes:

    $ sudo exportfs -va


macOS

  1. Enable Editing of /etc/exports

    Note: /etc/exports is often protected to prevent macOS updates from resetting its contents.

    Use chflags to make the file editable:

    $ sudo chflags nosimmutable /etc/exports

  2. Edit /etc/exports

    Add the following lines to export the Project Homes and Media Caches:

    /System/Volumes/Data/mnt/joker/projects /mnt/joker/projects -network 192.168.0.0 -mask 255.255.255.0
/System/Volumes/Data/Volumes/Media\ HD /mnt/joker/media -network 192.168.0.0 -mask 255.255.255.0

  3. Verify Your Changes

    Run nfsd checkexports to validate the exports:

    $ nfsd checkexports
exports:1: path contains non-directory or non-existent components: /mnt/joker/projects
exports:2: path contains non-directory or non-existent components: /mnt/joker/media
    Note: The above messages can be ignored. They appear because /mnt is a symbolic link. They do not prevent the exports.

    If you see nfsd is not running message, enable NFS:

    $ sudo nfsd enable

    Repeat the export verification afterwards.

  4. Apply the Changes

    Update the NFS configuration:

    $ sudo nfsd update

  5. Make /etc/exports Immutable Again

    Protect the file to prevent accidental changes:

    $ sudo chflags simmutable /etc/exports


Import via NFS

To enable project sharing, each host in the workgroup must import the Project Homes and Media Caches of other workgroup hosts. This ensures that all hosts can interact with shared projects seamlessly. It is recommended to use NFSv4 for server connections. However, when connecting to a macOS server, NFSv3 will be used.



Linux

  1. Configure Direct Mounts

    Add the following line to /etc/auto.master with your favourite text editor:

    /-  auto.direct

    Then, edit /etc/auto.direct and add these lines:

    /mnt/joker/projects 192.168.0.21:/mnt/joker/projects
/mnt/joker/media    192.168.0.21:/mnt/joker/media
    Note: If the remote host has a dedicated IP address for media transfers, you can specify it in this file. For example, if the media transfer IP address of the remote host is 192.168.0.22, you can use:

    /mnt/joker/media 192.168.0.22:/mnt/joker/media

    This ensures that media transfers are routed through the dedicated IP address, improving performance and isolating traffic.

  2. Apply the Changes

    Restart the autofs service:

    $ sudo systemctl restart autofs

  3. Work around the macOS NFS mount complications

    macOS NFS servers will stop responding to Linux clients if they are idle for 5 minutes or more unless specific conditions are met. For instance, having the same macOS storage mounted on a second macOS system eliminates the issue. A robust, albeit inelegant, solution to this problem is to access the mount point more than once every 5 minutes. This can be achieved with a scheduled job.

    Use cron to schedule a task every 4 minutes, regardless of user login status. cron runs system-wide and does not require a user to be logged in.

    Example: To run a command every 4 minutes, add a similar line to the system crontab (/etc/crontab). Replace /mnt/joker/media and /mnt/joker/projects with your own paths:

    */4 * * * * root /bin/stat /mnt/joker/media /mnt/joker/projects > /dev/null
    Note: Refer to the crontab(5) man page for details about the syntax of the crontab file:
    $ man 5 crontab


macOS

  1. Enable Editing of /etc/auto_master

    Note: /etc/auto_master is often protected to prevent macOS updates from resetting its contents.

    Use chflags to make the file editable:

    $ sudo chflags nosimmutable /etc/auto_master

  2. Configure Direct Mounts

    Add the following line to /etc/auto_master:

    /-      auto_direct     -nobrowse,resvport
    Note: The resvport option is required to import from Linux on macOS.

    Then, edit /etc/auto_direct and add these lines:

    /mnt/penguin/projects 192.168.0.11:/mnt/penguin/projects
/mnt/penguin/media    192.168.0.11:/mnt/penguin/media
    Note: If the remote host has a dedicated IP address for media transfers, you can specify it in this file. For example, if the media transfer IP address of the remote host is 192.168.0.12, you can use:

    /mnt/penguin/media 192.168.0.12:/mnt/penguin/media

    This ensures that media transfers are routed through the dedicated IP address, improving performance and isolating traffic.

  3. Apply the Changes

    Refresh the automount configuration:

    $ sudo automount -c

  4. Make /etc/auto_master Immutable Again

    Protect the file to prevent accidental changes:

    $ sudo chflags simmutable /etc/auto_master


Configure the Default Project Paths

To simplify project sharing, configure the default project paths in the sysconfig.cfg file. This ensures consistent paths for project metadata and media caches across all hosts in the workgroup. Refer to Centralised Components for additional details.

Under configuration/settings/project_folders, set the following values:

...
         "project_folders": {
            "default_home": "/mnt/<host name>/projects/<project name>",
            "default_remote_home": "/mnt/<host name>/projects/<project name>",
            "default_setups_dir": "<project home>/setups",
            "default_media_cache": "/mnt/<host name>/media/<project name>"
         }
...

Was this information helpful?