Talk Protocol

mental ray provides a more advanced mechanism for frame buffer exchange using bi-directional socket communication: the talk protocol. Unlike the simple uni-directional tile protocol, the talk protocol allows to retrieve image data in its original data type, like in true 32-bit floating point format. It also supports to request any of multiple frame buffers by name for delivery to the client.

mental ray automatically selects a specific network port to communicate using this protocol, and announces it in the stub file.

Once the connection is established mental ray will listen for commands. A command is a human-readable string of characters that ends with a newline character. Unsupported commands are rejected silently. If a valid command has been sent then mental ray immediately returns the requested information through the same connection. The end of the data sent back is marked with a newline character.

The commands recognized by mental ray are : 

version
fb_list
fb_list_name
stream_begin ...
stream_end
help
stop

The commands initiate the following actions in mental ray.

version
return version information, in the form of :
version: protocol, ray, date, type
for: protocol version, ray version, ray build date, and build type. For example: 
version: 2, 3.8.0.30, Jan 1 2010, Windows NT 5.2, x86-64, 64bit
fb_list
return information about all frame buffers, in the form of :
fb_list: id, type typename, x y, ncomp bpc, stereo
for: frame buffer index, data type +/-type name, x y resolution, number of components and bits per component, stereo eye. For example:
fb_list: 0, 0 +rgba, 320 240, 4 8, 0
fb_list: 1, 0 +rgba, 320 240, 4 8, 0
fb_list_name
return information about all frame buffers, in the form of :
fb_list_name: id, type typename, x y, fbname, ncomp bpc, stereo
for: frame buffer index, type +/-typename, x y resolution, number of components and bits per component, frame buffer name, stereo eye. For example:
fb_list_name: 0, 0 +rgba, 320 240, 4 8, "userbuffer", 0, 0
fb_list_name: 1, 0 +rgba, 320 240, 4 8, "maincolor", 0, 0
stream_begin idint [,idint … ]
request to stream data of given frame buffers, identified by comma-separated list of index numbers. This initiates tile streams to be sent by mental ray which are delivered until finished, or until ended explicitly by command.
stream_end
request to stop all tile streams.
help
return help text of supported commands.
stop
request to stop progressive rendering. This does not abort any regular rendering, but will stop further sampling in progressive rendering mode and iray.

Tile Streams

The stream_begin command is special since it starts the delivery of frame buffer content as rectangular tiles. The data returned by mental ray roughly follows the idea of the tile protocol, by sending data for a rendered frame encapsulated within frame messages marking begin and end, which contain several rect messages for the individual tiles. A typical sequence of returned packets will look like this : 

frame_begin ...

rect_begin ...
rect_data ...
rect_end ...

rect_begin ...
rect_data ...
rect_end ...
   ...

frame_end

A valid data block returned by mental ray ends with a newline character. The order of the tiles is determined by mental ray task order setting. If multiple frame buffers have been requested for streaming the tiles of the different buffers are returned interspersed, in no particular order. Receiving applications should be prepared to accept tiles of any buffer at any location in the final image of the current frame. On the other hand, frames are delivered in consecutive order.

The tile stream data is delivered according to the following format.

frame_begin
marks the beginning of a new frame, in the form of :
frame_begin: frame xres yres, type
for: frame number, x y resolution, type (0 for normal frame, 1 for output shader). For example:
frame_begin: 0 1024 768, 0
rect_begin
marks the beginning of a rectangular tile, in the form of :
rect_begin: xl xh yl yh, eye
for: low high x low high y pixel position, stereo eye. For example:
rect_begin: 0 63 128 191, 1
rect_data
send the actual tile data with a small header, in the form of :
rect_data: bytes, xl yl xh yh, id type, width height ncomp bpc, eye\n
raw_data ...
for: byte size, low x y high x y pixel position, frame buffer index, data type, x y resolution of the tile, number of components and bits per component, stereo eye, followed by raw binary data. For example:
rect_data: 16384, 0 128 63 191, 0 0, 4096 192 4 8, 1
§§§§...
rect_end
marks thew end of a tile, in the form of :
rect_end: xl xh yl yh, eye
for: low high x low high y pixel position, stereo eye, matching the corresponding rect_begin message. For example:
rect_end: 0 63 128 191, 1
frame_end
marks the end of the frame, in the form of :
frame_end:

Telnet Example

It is possible to use applications like telnet to talk to a running mental ray using the talk protocol. Here is an example session connecting to a mental ray running on the local machine. The talk port is determined by inspecting the stub file. Directly entered commands are printed as red text.

> more outputimage.png

ray3.6,320,240,vibrant,49337,1.000000,17736,49338

> telnet localhost 49338

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

version

version: 2, 3.8.0.30, Jan 1 2010, Windows NT 5.2, x86-64, 64bit

fb_list_name

fb_list_name: 0, 0 +rgba, 1024 768, 4 8, "userbuffer", 0, 0
fb_list_name: 1, 0 +rgba, 1024 768, 4 8, "maincolor", 0, 0

stream_begin 1

frame_begin: 0 1024 768, 0
rect_begin: 0 63 128 191, 1
rect_data: 16384, 0 128 63 191, 0 0, 4096 192 4 8, 1
§§§§
rect_end: 0 63 128 191, 1
rect_begin: 0 63 192 255, 1
rect_data: 16384, 0 192 63 255, 0 0, 4096 192 4 8, 1
§§§§
rect_end: 0 63 192 255, 1
rect_begin: 0 63 256 319, 1
rect_data: 16384, 0 256 63 319, 0 0, 4096 192 4 8, 1
§§§§

stream_end
^]

telnet> quit

>
Copyright © 1986, 2015 NVIDIA ARC GmbH. All rights reserved.