An important aspect of the plug-in development for custom renderer, model, and locator is implementing the object picking, selection, and manipulation functionality to be consistent with the existing viewport interaction behavior.

Selection Buffer Picking

MotionBuilder traditionally used the OpenGL selection buffer mode (GL_SELECT mode) as the base technique to implement the viewport selection. However, in this mode, the OpenGL picking pipeline stops after the primitive rasterization stage, and before the pixel shading stage where the alpha channel of models’ mapped texture is accessed to define the transparent regions. Moreover, the picking routine in OpenGL driver might fall back to the CPU approach, which significantly affects the performance for dense geometry.

For more information about the OpenGL selection buffer mode, see http://www.opengl.org/archives/resources/faq/technical/selection.htm.

As shown in the following figure, you cannot pick through the transparent portion of the curtain to select the couch behind it in this mode.

Color Buffer Picking

The new color buffer based picking overcomes the disadvantages mentioned in the Selection Buffer Picking section.

The concept is quite simple as follows:

1. Draw all the visible models with the regular rendering pipeline.
2. Use the model’s unique ID instead of filling the color buffer with the regular shading result.
3. Read the pixel value back from the color buffer under the mouse pointer.
4. Translate it back to the model’s unique ID, and find the corresponding model.

The color buffer filled up with the model's unique ID is shown in the following figure.

It is possible to pick through the transparent region of the model and select an object behind it in the Transparency Selection mode. It is a good feature, but requires more complicated coding efforts.

You can set a value in the FBRendererCallback::SupportIDBufferPicking property to support this mode. During the renderer callback, query FBRenderOptions::IsIDBufferRendering() to determine whether the current call is invoked by regular rendering or picking action. The model’s unique color ID is retrieved from the FBModel::UniqueColorId property. During transparency selection, all the regular models (unlike locators) need to fill their unique color IDs into the color buffer in the FBRendererCallback::Render() function.

For example, see the ORCustomRendererCallback::RenderWithSimpleIterating() function in the CustomRenderer sample.

The sample in /OpenRealitySDK/samples/miscellaneous/marker_template shows how to implement the selection buffer and color ID buffer picking logic for the custom locators.

Geometry Consolidation and Sub Item Selection (Advanced)

Sometimes it is useful to consolidate the multiple geometries into a single entity (model) to improve the runtime performance and save memory. You can also select individual pieces in the viewport and manipulate them separately. This can be achieved with the following set of functions from the FBModel class.

bool SetAdditionalUniqueColorIDCount(unsigned int pCount);
unsigned int GetAdditionalUniqueColorIDCount() const;
FBColor GetAdditionalUniqueColorID(unsigned int pIndex) const;

For example, see the sample in /OpenRealitySDK/samples/miscellaneous/model_template.

Z Depth Selection

The color buffer selection solved the transparency picking issue effectively. However, you might still face issues in a large and crowded scene that contains many objects. A new selection interaction mode called the Z Depth Selection can select an object, and engage the Hide Front behavior (Shift+1), so you can clearly see the selected object rendered on top of all the other objects. The other objects are culled back to the selected object’s farthest point from the camera. The custom renderer needs to display this effect in the same way as the default renderer.

The following figure shows a scene with a selected object (on the left) and the same scene with the Hide Front behavior (on the right).

Following is a code snippet that shows the necessary functions to call in your custom render callback for the Z Depth selection to work properly.

void MyCustomRendererCallback::Render(FBRenderOptions* pRenderOptions)
{
    ...
    // Loop through and render each model
    int lModelsCount = lRenderer->DisplayableGeometryCount;
    while(lModelsCount--)    
    {    
        FBModel* lModel = lRenderer->GetDisplayableGeometry(lModelsCount);
        FBModelVertexData* lModelVertexData = lModel->ModelVertexData;

        // Skips model if it is not visible, or if it is not supposed to be drawn
        if(!lModel->IsVisible() || !lModelVertexData->IsDrawable())  
            continue;

        lModelVertexData->PushZDepthClipOverride();

        // Draw model,for example, using FBModelVertexData::EnableOGLVertexData(), 
        // FBModelVertexData::DrawSubRegion(), FBModelVertexData::DisableOGLVertexData(), and others. 
           ...
        lModelVertexData->PopZDepthClipOverride();
    }
}

The FBModelVertexData::IsDrawable() function is called to skip the rendering for the model. For example, when the model need not be drawn because it is hidden by the Z Depth HideFront/Isolate Selection tool. Also, enclose the drawing code in FBModelVertexData::PushZDepthClipOverride() and FBModelVertexData::PopZDepthClipOverride(), so that the model selected by the Z Depth HideFront Selection ignores the corresponding OpenGL custom clip-plane and stays visible.

Selection Highlighting

You can change the default selection highlight on a model.

By default, a green wireframe is superimposed over the regular shading on the selected model as shown in the following figure.

However, you can add additional visual hints. For example, bounding box or a semi-transparent color overlay as shown in the following figure.

You can access and modify these setting through the UI (using the property view of the global Renderer object) or through the SDK. You can also remove or replace the default wireframe selection highlight by calling the FBObject_Register() and FBObject_Unregister() functions using the parameters group name render/utilityshader/ogl and entry name SelectionShader respectively.

Viewport Manipulators

You can also extend the existing viewport interaction behavior through the SDK. For example, see the sample projects in /OpenRealitySDK/samples/manipulators.

To achieve better viewport interaction, you can combine the extension of a manipulator with other customizable objects plug-in development.