Contents

• Glossary
• Flow Production Tracking At a Glance
• App module
  • Email notifier
• Database module
  • Users and roles
  • Postgres configuration
• Object caching module
• Transcoding module
  • Transcoding workflow
• References

This article gives an high level overview of the technologies and components of a Flow Production Tracking Instance.

It is intended for Flow Production Tracking Administrators.

Glossary

• DBMS. Database Management System.
• Flow Production Tracking Instance. Refers to your Flow Production Tracking site and all its components, e.g., “flow production tracking.yourstudio.com”.
• Flow Production Tracking Cluster. Generic term referring to all the servers and components in Flow Production Tracking.
• Production Cluster. Generic term referring to all the servers and components in your production cluster.
• Staging Instance. Instance dedicated for testing out new versions of Flow Production Tracking or tools you develop.
• App/Web Server. Server that receives requests from users and queries the database.
• API Server. Specialized App server that accepts only requests from the Flow Production Tracking API.

Flow Production Tracking At a Glance

Flow Production Tracking is composed of four main components.

1. App module: Receives requests from users and queries the database. The Flow Production Tracking application is powered by Apache and Phusion Passenger for Ruby on Rails. An app module can be specialized. The web module receives only requests from the browser, while the API module receives only requests from Flow Production Tracking integrations (Toolkit) and scripts using the Flow Production Tracking API.
2. Database module: Contains all your Flow Production Tracking metadata and powers your queries. PostgreSQL is Flow Production Tracking’s DBMS.
3. Object caching module: Memcached is used to cache results that are frequent and compute-intensive. Page schemas are the most heavily cached entities.
4. Transcoding module: The transcoding module is built on top of FFMPEG and ImageMagick. It converts your media into a web-friendly format.

App module

The Flow Production Tracking Application is instantiated through the Apache and Phusion Passenger stack running inside a container. Apache routes the requests to Passenger, which manages the instantiation of multiple Flow Production Tracking Application instances in order to allow the processing of multiple requests at the same time.

If no Flow Production Tracking processes are available, the request is queued. Each Flow Production Tracking process then makes one or more requests to the database, and the result is returned upon completion. If too many requests are made for the system, the queue will overflow and requests will be answered with a 503 error.

Email notifier

Another container, not driven by Apache, based on the app module is the emailnotifier.

The emailnotifier’s job is to observe Flow Production Tracking events and notify users by email on specific events. This container should always be running if email notifications are enabled in Flow Production Tracking’s Site Preferences.

Database module

At a very high level, Flow Production Tracking can be considered as a convivial layer over a PostgreSQL database. It offers off-the-shelf workflows and tools to easily fit the way you work, but it remains powered by a database. Always keep this in mind when building new pages and scripts. If your scripts are slow or making too many calls, Flow Production Tracking will feel slow!

Users and roles

Each Flow Production Tracking Instance has its own database in the DBMS. We typically configure the DBMS with three roles:

postgres. Generic PostgreSQL super-user.

shotgun. Super-user that it used for all management operations (backup, restore, etc.)

instance user. Each instance will have its related user. That role only has the permission to access that instance’s database. All transactions in the instance, including data migration on updates, are made using this user.

The password for the Shotgun superuser is stored on the app server in the home folder of the shotgun user, under .pgpass.

Postgres configuration

When running on a dedicated server, we configure PostgreSQL to achieve a high level of performance. This is why we recommend isolating PostgreSQL for heavy users. When running on the same server as the other components, PostgreSQL cannot be configured aggressively because the performance of the other components would suffer.

Among other things:

• PostgreSQL is configured to use all the memory available on the server.
• Memory settings are tuned for Flow Production Tracking patterns.
• A maximum number of simultaneous connexions is set.
• A maximum query duration is set for the instance user. This prevents performance degradation due to long queries.

It is possible to tune PostgreSQL even more aggressively, but it requires a very good knowledge of your usage patterns and your database server hardware. Please contact support before configuring if you are not a PostgreSQL expert.

Object caching module

Memcached is used to cache queries that are considered “static” and that would be expensive to compute every time.

Flow Production Tracking in mainly caching page schemas. Flow Production Tracking pages are dynamically constructed. This is because Flow Production Tracking allows for heavy customization of pages, entities, and fields. Schemas are expensive to compute and are only computed if needed, when the schema is modified.

Transcoding module

The transcoding module for a local install is a simplified version of our hosted transcoding service. A single worker, instantiated on demand by the Flow Production Tracking Application, executes all the transcoding for your site. This means that a maximum of one transcoding job is running at the same time.

ImageMagick is used to create thumbnails out of the media. FFmpeg is used to convert videos into a web streamable format. See Transcoding for more information about Flow Production Tracking transcoding.

Transcoding workflow

The Transcoding module is intended to be simple and may not be able to sustain your load. If you already have a transcoding or rendering farm in place at your studio, you may want to integrate this system with Flow Production Tracking. See Do it yourself transcoding for more details.

References

Phusion Passenger Architecture:

https://www.phusionpassenger.com/documentation/Design%20and%20Architecture.html

PostgreSQL Memory Tuning:

https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server

Memcached:

https://memcached.org

FFmpeg:

https://ffmpeg.org

ImageMagick:

http://www.imagemagick.org/script/index.php