Job Metadata

Job nodes contain the following metadata streams: info, details, xmlDetails, state, and tasks. Some streams and elements are read-only and are for the purposes of monitoring jobs already submitted. Others are read-write and can be used to create and control jobs. Like manager and server metadata, job node metadata can be obtained programmatically using the getMetaData method on the WiretapNodeHandle object for the server node, or using the wiretap_get_metadata command-line tool. It is set using the setMetaData method and wiretap_set_metadata command.

The following table summarizes the purpose of the job node metadata streams.

Metadata Stream Description
info Basic information needed to create, monitor and/or control a job.
details and xmldetails Job-specific or renderer-specific metadata for a job.
state Current state of activity for the job such as, waiting, and active.

Can be used to restart a job.

tasks Read-only information about the individual tasks making up a job.
Note: To retrieve selected metadata for all jobs on a server at once, use the joblist node instead. For all archived jobs, use the jobarchive node. See Jobarchive Metadata.

Job info Metadata

The info metadata stream contains basic job information common to all jobs. Some elements are read-only, and are automatically assigned by the system when you create the job. Others are writable, allowing you to control or modify the job.

Job node info metadata is formatted as follows: 

<info>
  <name>jobname</name>
  <id>jobid</id>
  <description>description</description>
  <tasknameList>tskname1,tskname2,...,tsknamen</tasknameList>
  <user>userid</user>
  <host>hostname</host>
  <pluginName>renderername</pluginName>
  <state>jobstate</state>
  <priority>jobpriority</priority>
  <numTasks>totaltasks</numTasks>
  <numTasksCompleted>completedtasks</numTasksCompleted>
  <percentTasksCompleted>percentage</percentTasksCompleted>
  <serverGroup>groupname</serverGroup>
  <servers>server1,server2,...,servern</servers>
  <serverCount>count</serverCount>
  <submittedTime>YYYY-MM-DD HH:MM:SS</submittedTime>
  <startTime>YYYY-MM-DD HH:MM:SS</startTime>
  <endTime>YYYY-MM-DD HH:MM:SS</endTime>
  <dependencies>jobid1,jobid2,...,jobidn</dependencies>
  <emailDest>name@host.com</emailDest>
  <emailFrom>name@ host.com</emailFrom>
  <emailCompletion>name@host.com</emailCompletion>
  <emailFailure>name@host.com</emailFailure>
  <emailProgress>name@host.com</emailProgress>
  <lastError>errorstring</lastError>
</info>

The following table explains the elements in the metadata stream.

Element Type Comments
<info> complex Container for other elements
<name> string The job display name. Must be unique within current set of Backburner jobs, and set when the job is first created.

Required. Read-only.

<id> integer The job’s ID as assigned by the Backburner Manager.

Read-only.

<description> string The description as provided by the processing client.
<tasknameList> string A comma-separated list naming each task in the job.

If you do not set this element explicitly, a default name is given to each task. The default name has the form: task X-Y

Where X is the current task number, and Y is the total number of tasks in the job.

For example, a job with only one task has a default task name of task 1-1.

Optional. Writable.

Note: The task names you set here become read-only elements in the job node’s tasks metadata stream. See Job tasks Metadata.
<user> string The name of the user that submitted the job.

This element is set automatically by Backburner when the job is first created.

This value is also returned by WireTapOS::getUserId.

Read-only.
<host> string The name of the host from which the job was submitted.

This value is also returned by WireTapOs::getHostName.

Read-only.
<pluginName> string The Backburner renderer adapter/plug-in needed for the job.

Backburner jobs are always assigned to a specific renderer type. This value is used by the Backburner Manager to determine which servers are eligible to handle a given job.

<state> string The current state of the job:
  • complete: Completed successfully.
  • active: Currently being serviced.
  • suspended: On hold.
  • idle: Not scheduled for service.
  • waiting: Ready, and waiting to be serviced.
<priority> integer The job priority from 0 to 100. Zero is the highest priority. 100 means the job is suspended. Default is 0.
<numTasks> integer Backburner jobs are broken down into tasks or sub-jobs that define the granularity of the job.

Blocks of tasks are delegated to multiple servers by the Backburner Manager in any order.

This value is set by the Wiretap client when the job is created. Default is 1.

Mandatory.
<numTasksCompleted> integer Set by the Backburner Manager as each task is completed.

Read-only.

<percentTasks Completed> integer Set by the Backburner Manager as each task is completed.

Read-only.

<serverGroup> string The server group to which the job will be submitted.

Only servers in the specified server group work on the given job, unless the group is set to use idle non-group servers.

<servers> string A comma-separated list of servers to which the job’s tasks are submitted.

When no servers are specified, the Backburner Manager is free to assign the job to any server with the appropriate adapter/plug-in.

This metadata is ignored if a server group is specified.
<submittedTime> string The time stamp at which the job is submittted (YYYY-MM-DD HH:MM:SS).
<startTime> string The time stamp at which the job is started (YYYY-MM-DD HH:MM:SS).
<endTime> string The time stamp at which the job is completed (YYYY-MM-DD HH:MM:SS).
<dependencies> string A comma-separated list of job node IDs on which the current job is dependent. The current job is started only when all the dependent jobs are complete.
<serverCount> integer The maximum number of servers that can work on this job at any one time. Set this to zero to run on all servers.
<emailDest> string Email address to which notification of job completion or failure is sent.
<emailFrom> string When email is generated by the Backburner Manager, this is the notification sender's email address.
<emailCompletion> string Turns on/off email notification on job completion:
  • 0: Email on completion is off.
  • 1 (default): Email on completion is on.
<archived> integer Turns on/off email notification on job completion:
  • 0: Email on completion is off.
  • 1 (default): Email on completion is on.
<emailProgress> integer Trigger email notification on the completion of every Nth task. Set to zero (default) to disable.
<lastError> string The last error message for the most recent task executed by the Backburner Manager.

Read-only.

Example

The following use of the wiretap_get_metadata command returns the info metadata for a job with ID 1318676208 on a Backburner Wiretap server called camiri:Backburner. As illustrated by the example, not all elements are returned for each job. For example, if you did not provide values for email-related elements when creating the job, these are not returned with the metadata stream. 

wiretap_get_metadata -h camiri:Backburner  -n 1318676208 -s info
 
<info>
  <name>Burn_camiri_070620_12.41.07</name>
  <id>1318676208</id>
  <description>Clip: COL_NOISE</description>
  <user>root</user>
  <host>camiri</host>
  <pluginName>burn 2008.0</pluginName>
  <state>suspended</state>
  <priority>50</priority>
  <numTasks>3</numTasks>
  <numTasksCompleted>3</numTasksCompleted>
  <percentTasksCompleted>100</percentTasksCompleted>
  <serverGroup></serverGroup>
  <submittedTime>2009-02-09 07:12:21</submittedTime>
  <startTime>2009-02-09 07:12:21</startTime>
  <endTime>2009-02-09 07:19:00</endTime> 			
  <dependencies><dependencies/>
  <servers>camiri</servers>
  <serverCount>4</serverCount>
  <lastError>camiri</lastError>
</info>

Job details and xmlDetails Metadata

The job node details and xmlDetails metadata streams facilitate the transmission of custom information in ASCII format from the Wiretap client to the Backburner server for processing. The details stream is converted into XML automatically. For information that is already XML compliant, there is the xmlDetails stream. When using these data streams, take care to consider the amount of data being transmitted, and backwards compatibility for future adapters/plug-ins.

An adapter is a plugin furnished by the rendering client. It receives job requests from Backburner Manager through the Backburner server and is responsible for launching a rendering engine to carry out the requests.

Both the details and xmlDetails metadata streams serve the purpose of supplying the adapter with custom information. For example, you can use the details and xmlDetails metadata streams to specify instructions specific to the renderer launched by the adapter, or to provide tailored instructions for a particular job. The job node details and xmlDetails metadata streams are free-form and there is no structure imposed. It is up to the Wiretap client and renderer to agree on a metadata format for the custom information.

The following table shows the encoding that automatically takes place in the details stream for the less-than (<), greater-than (>) and percent (%) characters. It is up to your adapter/plug-in to decode these appropriately. Null characters are not permitted in the details stream:

Character Encoding
< %3C
> %3E
% %25
null not permitted

For metadata that is already XML-compliant use the xmlDetails stream, which you can use to monitor and control a job after it is submitted. Make sure that the stream is encoded correctly. Using the xmlDetails stream with non-XML encoded data can yield unexpected results including Backburner malfunction.

Note the following points related to the details or xmlDetails metadata stream.
  • Do not use the details or xmlDetails streams to transmit binary data or large amounts of non-binary metadata (more than 1-5 MB). Send these as an attachment. See Sending Attachments to the Backburner Manager.
  • Consider identifying the metadata stream using a version number so that future renderers can accommodate submissions from older Wiretap clients.

Job State Metadata

Job nodes contain a metadata stream called state that is useful to ascertain or modify a job’s state of activity. It can be obtained using the getMetaData method on the WiretapNodeHandle object for the manager node, or using the wiretap_get_metadata command. It can be set using the setMetaData method or the wiretap_set_metadata command.

Note: State information is also included as a read-only value in the job node’s info metadata.

The job state metadata is returned as a string as shown in the following table.

String Value Meaning
waiting Ready, and waiting to be serviced.
active Currently being serviced.
idle Not scheduled for service.
suspended On hold. This is the default state for a newly created job.
complete Completed successfully.
restarting Returning the job to its initial sate; setting the state to waiting.

The following table provides examples of specific state transitions.

Start State End State Result
complete waiting/restarting Restarts the job.
suspended/waiting/active/idle restarting Restarts the job.
waiting/active/idle suspended Suspends execution of the job.
suspended waiting Resumes a suspended job.

Job Tasks Metadata

The job node tasks metadata stream contains read-only information about the individual tasks that make up a job. It can be obtained using the getMetaData method on the WiretapNodeHandle object for the manager node, or using the wiretap_get_metadata command. It is set and updated by the Backburner manager.

Job node tasks metadata is formatted as follows: 

<tasks>
  <task>
    <number>tasknumber</number>
    <name>task start-end</name>
    <startTime>YYYY-MM-DD HH:MM:SS.MS</startTime>
    <elapsedTime>HH:MM:SS.MS</elapsedTime>
    <elapsedTimeMsec>MS</elapsedTimeMsec>
    <server>servername</server>
    <state>taskstate</state>
    <lastError>message</lastError>
  </task>
</tasks>

The following table explains the elements in the metadata stream.

Element Type Comments
<number> string The task number for this task.
<name> string Name of the current task.

By default, the task name has the form:

task X-Y

Where X is the is the current task, and Y is the total number of tasks in the job. For example, task 5-7 indicates this is the 5th task in a job consisting of 7 tasks in total.

You can set task name information explicitly in the job node info metadata stream. See Job info Metadata.
<startTime> string The time stamp of the task assignment (YYYY-MM-DD HH:MM:SS.MS).
<elapsedTime> string The time duration consumed by the task (HH:MM:SS.MS).
<elapsedTimMsec> integer The time duration consumed by the task in milliseconds only.
<server> string The name of the server where the task is being executed.
<state> string The state of the task (active, complete, waiting, error).
<lastError> string The last execution error message.

Example

The following use of the wiretap_get_metadata command returns the tasks metadata for a job with ID 1318676208 on a Backburner Wiretap server called cardigan:Backburner. 

wiretap_get_metadata -h cardigan:Backburner -n 1318676208 -s tasks
 
<tasks>
  <task>
    <number>1</number>
    <name>task 1-2</name>
    <startTime>2008-11-18 14:53:25:011</startTime>
    <elapsedTime>00:16:13.721</elapsedTime>
    <server>burn03</server>
    <state>error</state>
    <lastError>Launching renderer failed </lastrror>
  </task>
  <task>
    <number>2</number>
    <name>task 2-2</name>
    <startTime>2008-11-18 14:53:27:0705</startTime>
    <elapsedTime>00:00:03.216</elapsedTime>
    <server>burn04</server>
    <state>complete</state>
    <lastError></lastError>
  </task>
</tasks>

Jobs Events

Related Command Line Tool

The client can monitor jobs using events. An event is sent for each Job node added or removed. An event is also sent every time a Job node's state changes.

To receive event you need to implement a WiretapEventHandler object and register it using WireTapNodeHandler::registerEventHandler() on the node you want to monitor.

Parent topic: Backburner Wiretap Server