Stone and Wire

Stone and Wire is a layer that sits between the application and the file system. It is required and installed with Creative Finishing applications, except Lustre. It handles local management of media and the transfer of media between workstations. Wire enables the transfer of uncompressed video, film and audio between workstations over networks.

Five processes that must be running for Stone and Wire to work: sw_serverd, sw_probed, sw_dbd, sw_bwmgr, ifffsWiretapServer.

Use the Service Monitor to restart Stone and Wire or check the services status.

From the command line, enter ServiceMonitor or access the Service Monitor from:

• On Linux: Launcher > Autodesk > Service Monitor
• On macOS: Applications > Autodesk > Service Monitor

Stone+Wire Server Does Not Respond

Connection Refused

A connection refused error happens when Stone and Wire cannot connect to the specified port on the target host. There are two possible causes for this error:

The firewall rules forbid the inbound connection. There is no process running to acknowledge the connection.

Check that the services is running correctly by checking the service state in the Service Monitor.

Change the Stone+Wire service port /opt/Autodesk/sw/cfg/sw_serverd.cfg.

See Network Ports section of Network Settings for a full list of network ports used by Flame and its services.

API Request timed out (Port conflict)

An API request often times out because another process uses the same network port. Some applications use the same ports by default and conflict with API requests. On macOS, AirPlay is usually the application that causes conflicts.

To check for port conflicts, you can run the Network Ports diagnostic under the Diagnostics tab of the Services Monitor.

To solve the port conflict, try either of the following:

• If you can, disable AirPlay in System Preferences > Sharing > AirPlay Receiver.

• If you cannot disable AirPlay, you must change the network port used by Stone+Wire.

  As root, edit /opt/Autodesk/sw/cfg/sw_serverd.cfg and change the port token in the in [Server] section. You must also change the port token on all hosts part of the same network to this new value.

See Network Ports section of Network Settings for a full list of network ports used by Flame and its services.

Logging and errors

Every Stone and Wire process has a log file at /opt/Autodesk/sw/log.

The current log file is named <process>.log, where <process> is the name of the Stone and Wire process or daemon. The next time Stone and Wire creates a log file for the process, it renames the previous log file by adding a number to the file name. For example, the sw_served process log file is named sw_served.log. The next time the process is launched, the first log file is renamed to sw_served.log.1. Each time Stone and Wire creates a new log file, the old log file is renamed by incrementing the number that appears at the end of the filename.

Use the terminal to list the most recent version of the log files: ls -altr

On Linux, you can get system logs with: journalctl -u “adsk_*”

Stone and Wire configuration

The configuration file is /opt/Autodesk/sw/cfg/stone+wire.cfg. In the case of software upgrades, if stone+wire.cfg exists, then it is not overwritten. However a new stone+wire.cfg.sample file is written which includes any new functionality added in that release, including for example support for new file formats.

sw_framestore_map

/opt/Autodesk/sw/cfg/sw_framestore_map is only available after the installation of the Autodesk product. It is configured when the application is installed. Normally, it should not be necessary to edit this file.

It has two sections

• [FRAMESTORES]: contains information on the media storage volume. If the workstation uses DHCP to get its IP address, use localhost as the value for the HADDR parameter.
• [INTERFACES]: keywords that define the network interfaces on the system that are available for Wire transfers.

Media pre-allocation

By default, disk space for video files is allocated on-the-fly as the files are rendered or written. Concurrent I/O operations from multiple clients or background processes can generate frames with poor locality of reference, which leads to clip fragmentation and decreases playback performance.

To prevent this problem, Stone and Wire provides media file pre-allocation mechanisms that reserve the necessary space on the disk before the actual writing of the media files takes place. Pre-allocation reserves as many available contiguous disk sectors as needed for the file sequence to be written.

There are two media pre-allocation methods:

FALLOCATE

The disk space needed for the file is allocated and filled with zeroes. On most filesystems, the performance impact of this operation is as serious as when writing the actual file.

RESERVE

The disk space needed for the file is reserved, but not written to. The apparent size of the frame remains zero. This operation is much faster than FALLOCATE, but is supported only on certain filesystems, such as XFS.

By default, pre-allocation is turned off in Stone and Wire (the method is set to NONE). To enable it, modify a parameter in stone+wire.cfg. This is not a global setting. Pre-allocation must be individually enabled for each partition. On an XFS system on which unwritten extent flagging was disabled when creating the filesystem, the RESERVE pre-allocation method is not available for it.

To enable media pre-allocation for a partition edit the Preallocation parameter in /opt/Autodesk/sw/cfg/stone+wire.cfg Save the configuration file and restart Stone and Wire.

sw_io_perf_tool

Stone and Wire comes with a command line tool to measure filesystem performance called sw_io_perf_tool. This tool simulates I/O requests (audio and/or video) that would be issued by the Creative Finishing application, and reports the throughput and maximum latency of all the requested I/O operations. Use sw_io_perf_tool to measure read and write performance of a volume declared and mounted by Stone and Wire. Several options are available to simulate many file formats and writing/reading patterns. Launching the command will display a list of available options.

To list all options available for sw_io_perf_tool, open a terminal and type /opt/Autodesk/sw/tools/sw_io_perf_tool -h .

To test a standard filesystem partition's performance for a specific resolution, Run /opt/Autodesk/sw/tools/sw_io_perf_tool <-p #> <options> where <-p #> specifies a partition ID (0 to 7), and<options> can be a combination of several media format attributes.

sw_io_perf_tool with custom resolutions may be used. For example, to test read performances for frames with a width of 1600 pixels and a height of 900 pixels: Max Latency: 335.284 ms

The command can be run while other operations are ongoing to simulate how the application will behave under these conditions. The media used by sw_io_perf_tool is allocated and freed automatically. sw_io_perf_tool will not reserve any bandwidth on the storage device. Partitions/volumes must be properly configured in the Stone and Wire configuration file.

network.cfg

/opt/Autodesk/cfg/network.cfg is the main configuration file used by network interfaces. It is used for Stone+Wire and WireTap based services.

wiretapgateway

Imports compressed media into Stone and Wire partitions. Decodes compressed media in order to be used or imported by Stone and Wiretap servers, or by remote applications. Its configuration file is: /opt/Autodesk/wiretapgateway/cfg/wiretapgateway.cfg.

sw_probed

Uses a self-discovery mechanism to locate all other Wire hosts and their framestores on the Wire network. Stores the configuration required for communication between two hosts in memory. Its configuration files are: /opt/Autodesk/sw/cfg/sw_probed.cfg and /opt/Autodesk/cfg/network.cfg. To enable multicasting:

1. On the render node, open /opt/Autodesk/sw/cfg/sw_probed.cfg for editing.
2. Set SelfDiscovery to Yes. Now sw_probed runs in self-discovery mode and it will automatically probe the network for other systems. This is set to Yes by default when Stone and Wire is installed on the render node.
3. Save and close sw_probed.cfg
4. On the render node, open /opt/Autodesk/cfg/network.cfg for editing.
5. The Scope parameter in the [SelfDiscovery] section defines the scope for the multicast. Ask your network administrator for the right value to use.

For a network with one subnet, Link Local ( 224.0.0.1 ) should be used, with TTL = 1.

For a network with more than one subnet, use a scope appropriate to your network requirements and router configuration; you also need to edit the "TTL" value (see next step).

Note for AWS Transit Gateway multicast users: 224.0.0.1 does not work with IGMPv2 automatic discovery, preventing members from joining the multicast group. For multicast to work in this context, use instead an address in the 239.0.0.0/8 range such as 239.0.0.1.

The "scope" value must be the same for every Flame Family workstation on your network or workgroup.

1. If the workstations and nodes in the facility are on separate networks and connected through routers, you must specify the number of router hops for a multicast. For this you use the TTL parameter in the [SelfDiscovery] section.

1. Save and close network.cfg, then restart the sw_probed daemon with /opt/Autodesk/sw/sw_restart all.

To be usable for multicasting, the network interface must have the MULTICAST flag enabled at the OS level. This can be validated and changed using the ifconfig or ip link commands. Enabling the 'MULTICAST' flag on the 'loopback' interface is recommended to help ensure discoverability of local services.

sw_serverd

The Wire server daemon. Controls the transactions between the Stone and Wire filesystem and remote applications. Its configuration file is: /opt/Autodesk/sw/cfg/sw_framestore_map.

sw_dbd

Responsible for mapping frame IDs from the entries in the swdb database to the location of the media on disk. Database entries are created for all managed media and soft-imported media. One Standard FS media database file exists for each Standard FS partition. Its configuration file is: /opt/Autodesk/sw/cfg/sw_dbd.cfg. Logs are located in: /opt/Autodesk/sw/log/

Configure the Stone and Wire database to automatically shut off when there is too little free space:

1. As root, edit /opt/Autodesk/sw/cfg/sw_dbd.cfg, and in the [Disk Check] section uncomment the Enabled and Minimum keywords.
2. Set the value of Minimum to the minimum allowed amount of free space on the disk (in Megabytes). If the amount of free space on the disk falls below this threshold, the Stone and Wire database daemon automatically shuts off.
3. Optionally uncomment the Timeout keyword, and set how often the Stone and Wire daemon should check the available disk space. The default value is 30 seconds.
4. Save and close the configuration file, the restart Stone and Wire: /opt/Autodesk/sw/sw_restart all.

Bandwidth reservation: sw_bwmgr

Reserves storage bandwidth for Autodesk applications and tools based on its configuration file /opt/Autodesk/sw/cfg/sw_bwmgr.cfg. Documentation is in /opt/Autodesk/sw/cfg/sw_bwmgr.cfg.sample.

The Stone and Wire bandwidth reservation mechanism tries dynamically to improve bandwidth distribution to Autodesk applications connecting to a workstation’s storage. It cannot protect the storage from I/O requests from non-Autodesk applications and tools, since it doesn't handle them.

In low bandwidth situations, instead of reducing all reservations proportionally towards zero, it first reduces the bandwidth of each process towards the low-bandwidth value specified for that process. If the bandwidth continues to degrade, it starts reducing bandwidth proportionally towards zero.

Configure bandwidth reservation

Additional standard filesystem mount points (such as mount points to SAN volumes or to any other standard filesystem volume that is not present in /opt/Autodesk/sw/cfg/stone+wire.cfg) can be managed by the Bandwidth Manager. Each partition managed by the Bandwidth Manager must have a [Device] section associated to it in /opt/Autodesk/sw/cfg/sw_bwmgr.cfg. To add a new device to the Bandwidth Manager:

1. Open /opt/Autodesk/sw/cfg/sw_bwmgr.cfg and add a new [Device] section at the end of the file, as follows:

   • <device_number>: the number of the [Device] section, e.g. [Device0].
   • Path: the partition's mount point. Since a partition can have several paths, <path_number> is the number of the current path, starting at 0 for each device, e.g.: Path0=/mnt/XYZ.
   • TotalAvailableReadBandwidth: the estimated total reading bandwidth of the device, in megabytes per second. Based on this value, and on the amount of active reservations for the current partition, the Bandwidth Manager decides whether new reservation requests are granted or rejected.
   • TotalAvailableWriteBandwidth: the maximum estimated writing bandwidth of the device, in megabytes per second. If unspecified, its value defaults to the value of TotalAvailableReadBandwidth. The total bandwidth parameters are estimates of the theoretical maximum bandwidth of the partition. The actual bandwidth is affected by several factors, including multiple applications trying to concurrently read or write to it. The Bandwidth Manager continuously measures partition performance and dynamically maintains the actual total available bandwidth for each partition.
   • DefaultReservation: the bandwidth assigned by default to applications that are denied a reservation request or that don't have individual or group reservations associated to them. The value is expressed in megabytes per second, e.g.: DefaultReservation=10

2. Define reservations for the current partition.

3. Save and close the configuration file, and restart Stone and Wire: /opt/Autodesk/sw/sw_restart all

Define groups of applications

Bandwidth reservation can be done by groups of applications.

1. In /opt/Autodesk/sw/cfg/sw_bwmgr.cfg, [Groups] section, add a line for each group of applications to be defined, as in this example: <group_name>=<application_name> <application_name> where:

   • <group_name> is the custom name of the group. The group name must not contain spaces and must not be the same as one of the predefined application names.
   • Each <application_name> represents the name of an application in the group. This parameter can take one of the following predefined values: “inferno”, “flame”, “flint”, “smoke”, “stonifiseTool”, “publishTool”, “S+W Server”, “IFFFS Wiretap Server”, or “IFFFSTool” for other Creative Finishing command-line tools.

2. In the [Device] section add a Reservation line for the new group. The syntax is the same as when adding a reservation for a single application. For example (using the “IFFFSTools” group from the previous step): Reservation1=IFFFSTools 50 50

3. Save and close the configuration file, and restart Stone and Wire: /opt/Autodesk/sw/sw_restart all

For more information see the comments inside the configuration file.

Remove invalid frame references from the Stone and Wire database

Stone and Wire has a database inconsistency detection mechanism. If the framestore ID of a Stone filesystem or standard filesystem partition in the system is changed without reformatting the Stone and Wire database, the inconsistency check mechanism detects invalid frame references in the database, and prevents the Stone and Wire database server sw_dbd from starting.

To remove invalid frame references from the Stone and Wire database:

1. In a terminal, as root, search the /opt/Autodesk/sw/log/sw_dbd.log file for the text “Bad framestore ID” e.g.: cat /opt/Autodesk/sw/log/sw_dbd.log | grep -i bad/ framestore/ id

   • If the search returns one or more lines, the problem is related to invalid frame references. Perform the remaining steps in this procedure to clean up the Stone and Wire database.
   • If the search returns no results, the problem is not related to invalid frame references. Check the /opt/Autodesk/sw/log/sw_dbd.log file to determine the cause of the problem.

2. Stop Stone and Wire: /opt/Autodesk/sw/sw_stop all

3. Clean up the Stone and Wire database: /opt/Autodesk/sw/sw_dbd -C. A prompt offers to remove invalid entries from the Stone and Wire database.

4. Restart Stone and Wire: /opt/Autodesk/sw/sw_start

Check disk usage

To see if the filesystem is full, run /opt/Autodesk/sw/sw_df. If the filesystem is full or almost full, consider deleting or archiving older material from clip libraries to free disk space. VTR archiving in Creative Finishing applications requires free space on the framestore to generate slates, test patterns, headers, and audio encoding.

Clear the Undo Buffer to free up space. Clearing the Undo Buffer cancels any undoable operation, so use carefully. This is available in the Preferences menu, Undo section.

Solve a framestore ID conflict

All hosts on the Wire network must have a unique framestore ID. By default the ID is generated from the IP address. It is possible that there could be an ID conflict, where an ID is not unique. In this case Flame will not run, because the Wiretap server shuts down after detecting the conflict. The log file /opt/Autodesk/log/ifffsWiretapServer.log will contain an error.

Solve a partition ID conflict

Each partition must have a different partition ID. To solve a Partition ID conflict:

1. Stop Stone and Wire: /opt/Autodesk/sw/sw_stop all.
2. In /opt/Autodesk/sw/cfg/stone+wire.cfg, find the [PARTITION#] section, where # is the ID listed in the error message. Change the # for an unused ID. Partition identifiers go from 0 to 7, giving a total of 8 possible partitions. Stone filesystem and standard filesystem partitions share the same pool of identifiers, but only standard filesystem ones are listed in stone+wire.cfg. It is recommended to start numbering standard filesystem partitions at 7, decrementing with each additional standard filesystem partition.
3. Save and close the stone+wire.cfg file.
4. Start Stone and Wire: /opt/Autodesk/sw/sw_start.