Porting Selection from Viewport 1 to 2

Introduction

Porting selection functionality in your plug-ins can be seen as a separate task from porting drawing, depending on the interfaces used and the version of Maya used.

As of Maya 2016, native Viewport 2 selection is available.

Before the 2016 release, selection could only be performed using existing Viewport 1 interfaces. The Viewport 1 interface drawing must use OpenGL even if the draw code for Viewport 2 is using a different drawing API (such as DirectX11).

The Viewport 1 interfaces are dependent on OpenGL software pick buffers for selection. Software pick buffers are not available when you are using a core profile context. If a legacy OpenGL profile is available, then this functionality may be available.

A choice of OpenGL profiles is available on the Windows and Linux platforms. On certain configurations of Mac OS X, only core profile contexts can be instantiated.

Interfaces

The following is an overview of the Viewport 2.0 selection interfaces for various objects and components. For more information regarding these interfaces, see the sections below.

Objects/components	Interface for drawing	Interface for selection
Manipulators: MPxManipContainer MPxManipulatorNode	MUIDrawManager	Drawing done using MUIDrawManager is used for hit testing. MUIDrawManager::beginDrawable(unsigned int name, bool nameIsPickable ) The bool nameIsPickable input parameter to this interface can be used to mark drawables as selectable.
Locators: MPxLocatorNode	MUIDrawManager	Drawing done using MUIDrawManager is used for hit testing.
Shapes: MPxSurfaceShape MPxComponentShape MPxLocatorNode	MPxGeometryOverride MPxSubSceneOverride	MPxSurfaceShape::getShapeSelectionMask() to specify the type of geometry, selection priority and filtering. MPxSubSceneOverride::getSelectionPath() for complex plug-ins to obtain the dag path from a render item.
Components	MPxGeometryOverride MPxSubSceneOverride	MSelectionContext to provide information about the state of selection, including selection level. MIntersection to provide hit intersection information. MSelectionInfo to provide selection state information about selected objects. MPxComponentConverter to convert selection intersection data to a list of object components (MFnComponent) by matching the index buffer positions to valid component ids. Register/de-register with MDrawRegistry::registerComponentConverter() and MDrawRegistry::deregisterComponentConverter(). MPxGeometryOverride::refineSelectionPath() to override the selected path or selected components during the hit test phase. MPxGeometryOverride::updateSelectionGranularity() and MPxSubSceneOverride::updateSelectionGranularity() to set the selection level, for example, object or component. MPxSubSceneOverride::getInstancedSelectionPath() to select the correct render item instance when the viewport is in GPU acceleration mode. MPxGeometryOverride::pointSnappingActive() and MPxSubSceneOverride::pointSnappingActive() to set the correct selection level when snapping is active. MGeometry::DrawMode kSelectionOnly to indicate draw only for selection. MRenderItem::setSelectionMask() to set the render item selection mask. MPxSurfaceShape::getComponentSelectionMask() to provide information about the shape component selection mask.

Note: Tool contexts (MPxContext) cannot be selected and are thus not included in this list.

Manipulators

Manipulators are not selectable at the object level for both Viewport 1 and Viewport 2 as they do not exist as scene elements.

The selection level that is available for MPxManipulatorNode instances is defined at the "sub-object" level, also called the "handle level".

Manipulator containers (MPxManipContainer) that are composed from existing manipulators require no additional code to support selection as this support is defined at the manipulator level. This is true for both Viewport 1 and Viewport 2. It is possible to add additional draw (and hence selection) for a container as desired.

Draw Interface Used	Viewport 2 Selection Disabled	Viewport 2 Selection Enabled	SDK Examples
MUIDrawManager (without custom selection handles)	Uses Viewport 1 draw routine for hit testing.	All drawing done using MUIDrawManager during draw will be used for a single hit test.	footprintManip (MPxManipulatorNode) swissArmyManip (MPxManipContainer)
MUIDrawManager (with custom selection handles)	Viewport 1 interfaces use OpenGL pick identifiers as the actual selection handles and explicitly relies on software pick buffers (which may not exist for all OpenGL context profiles). Interfaces that can be used: colorAndName() shouldDrawHandleAsSelected() glFirstHandle() glActiveName()	Specific drawables can be marked selectable using "handle" identifiers via arguments on the MUIDrawManager::beginDrawable() method: MUIDrawManager::beginDrawable(unsigned int name, bool nameIsPickable ) If nameIsPickable is set to be "true" then the subsequent draw code will be used for hit testing. This should apply for both 3D as well as 2D manipulators.	lineManip (MPxManipulatorNode)

Locators

Locators (MPxLocatorNode) are only selectable at the object level.

For locators that draw very simple geometry, or when there will be very few occurrences of a given locator in a scene, it is acceptable to use the MUIDrawManager interface for drawing and selection.

For "click" or closest point single selection, there is no change in the interface logic used. The members MPxLocatorNode::useClosestPointForSelection() and MPxLocatorNode::closestPoint() should be implemented for Viewport 1 selection. For Viewport 2 selection, the closest point is computed automatically from the geometry data and hence the APIs are not used.

Anything that is more complicated or requires better performance falls into the category of "shapes", covered in the next section. They should be implemented using an MPxGeometryOverride or MPxSubSceneOverride

Draw Interface Used	Viewport 2 Selection Disabled	Viewport 2 Selection Enabled	Examples
MUIDrawManager	MPxLocatorNode::boundingBox() is used for hit testing.	All drawing using MUIDrawManager is used for hit testing.	2016 version of footPrintNode
MPxDrawOverride::prepareForDraw()	MPxLocatorNode::boundingBox() is used for hit testing. Position of the hit is the position of the origin of the locator. For more precision, implement MPxLocatorNode::closestPoint() (see above). (Note: MPxDrawOverride::prepareForDraw() is not used, even if implemented).	Raw draw code cannot be supported and therefore using MPxDrawOverride is not recommended.	2015 or earlier version of footPrintNode, or 2016 version of rawFootPrintNode

Shapes

More complex object types such as MPxSurfaceShape and MPxComponentShape, (or MPxLocatorNodes that require more complex or persistent drawing) should use render item (MRenderItem) -based interfaces: MPxGeometryOverride or MPxSubSceneOverride.

Using MUIDrawManager for complex objects is not recommended, though Viewport 2 selection can still work.

Using MPxDrawOverride if Viewport 2 selection is required is not recommended as it does not support raw draw code.

MPxGeometryOverride

Draw interface used	Viewport 2 Selection Disabled	Viewport 2 Selection Enabled	Examples
MPxGeometryOverride (object level)	Use Viewport 1 selection interfaces	Selection is performed per render item. For shapes, MPxSurfaceShape::getShapeSelectionMask() can be overridden to specify the type of geometry, selection priority and filtering.	footPrintNode_GeometryOverride: for usage with a locator apiMeshShape: for usage with a surface shape gpuCacheShapeNode: for an example of using getShapeSelectionMask()
MPxGeometryOverride (component level)	Use Viewport 1 selection code	See Viewport 2 Selection Handling of Components section.	apiMeshShape

MPxSubSceneOverride

Draw interface used	Viewport 2 Selection Disabled	Viewport 2 Selection Enabled	Examples
MPxSubSceneOverride	Use Viewport 1 selection interfaces or custom selection.	Complex plugins may require implementing getSelectionPath() to get back the proper dag path (MDagPath) from a render item.	gpuCache: gpuCacheSubSceneOverride.cpp
MPxSubSceneOverride (component level)	Use Viewport 1 selection interfaces or custom selection.	See Viewport 2 Selection Handling of Components section.

Viewport 2 Selection Handling of Components

When Viewport 2 selection is enabled (as of Maya 2016) the selection interfaces on MPxSurfaceShapeUI are no longer called.

Functionality in the following Viewport 1 interfaces:

are handled by the following Viewport 2 interfaces:

MSelectionContext: Information about the state of selection including selection level.
MIntersection: Hit intersection information
MSelectionInfo: Used with MPxGeometryOverride::refineSelectionPath()
MPxComponentConverter : utility to handle MIntersections and convert index buffers to MFnComponents.

Registered / deregistered on MDrawRegistry using registerComponentConverter() / deregisterComponentConverter()
For MRenderItem:

A "selection" mode (DrawMode) allows indication of whether a render item should be used for selection.

There is the ability to set a selection mask (MSelectionMask) to indicate the selection type.
MPxSurfaceShape: has been extended with the ability to return a selection mask (MSelectionMask) to indicate that component selection is supported in Viewport 2.

Plugin Setup

A plug-in must register its own implementations of MPxComponentConverter for any given render item name. It is possible to register the same converter against multiple render item names to allow reuse between different render items. A new converter is created for each render item to process.

Regular Draw Update

Render items marked for selection are added when the draw update code is called due to any change in rendering preferences (for example: a change in the highlight list, and so forth). Starting a selection should not dirty the render items of scene elements that are not in the selection region, so the plug-in should not expect MPxGeometryOverride::updateDG() to be called for all nodes each time a selection is done.

The following describes the steps by which selection is processed:

Selection draw stage:

MPxGeometryOverride::updateSelectionGranularity() is called to find out if the geometry node wants to participate in component selection
In camera selection mode :
- All selectable render items are drawn with a special shader that uses a different color for each component
- The draw buffer is then scanned and each color found in the selection region is transformed into a MIntersection
In regular selection mode:
- The vertex positions of all selectable render items are used to compute MIntersection with the selection region.

Selection interpretation stage:

The list of hits is sorted by render item (or proximity for click selection).
All hits for a render item are then processed and MPxComponentConverter::addIntersection() is called to convert the hit information into a MFnComponent.
When all hits are processed for a render item or when the nearest hit is processed for "click" selection, MPxComponentConverter::component() and MPxComponentConverter::selectionMask() are called to gather the final component information, and then MPxGeometryOverride::refineSelectionPath() is called to add the finishing touches to the selection information before it gets added to the list of selected items.

Examples: apiMeshShape (MPxGeometryOverride, and MPxSubSceneOverride versions).

Notes for Complex Shaders

For hardware shaders that displace input geometry (for example, a mesh that is drawn at a different location, or with more details, such as tessellation), or that generate complex geometries at the shader level (for example, shaders that use a geometry stage to draw complex geometry such as a particle blob on a very basic geometry such as a point):

For the Viewport 2 selection path, when creating the selection render items, custom hardware selection effects can be added that replicate the displacement/generation of the beauty pass, but that draw flat shaded geometries using a color defined by the hardware selection protocol. These items will then be selectable using camera based selection mode in a WYSIWYG manner in the viewport.
For the non-Viewport 2.0 selection path, draw code in Viewport 1 interfaces which displaces geometry is always ignored.

Notes for Geometry

It is possible to reference or extract out the geometry used for render items (MRenderItems) and to use raw resource handle to draw with. This can help to avoid duplicating memory if using Viewport 2 draw and Viewport 1 selection.

Camera based selection

In camera based selection mode, the scene is painted using a single plain color for each distinct scene element that can be selected. The shader should use the following formula to generate the plain color used for selection:

int colorID = [HWS_PrimitiveBase];
if (![HWS_ObjectLevel])
    colorID += PrimitiveID;
if ([HWS_InstancedDraw])
    colorID += InstanceID * [HWS_PrimitiveCountPerInstance];
float4 color;
color.x = float(colorID & 0x000000FF) / 255.0;
color.y = float((colorID & 0x0000FF00) >> 8) / 255.0;
color.z = float((colorID & 0x00FF0000) >> 16) / 255.0;
color.w = float((colorID & 0xFF000000) >> 24) / 255.0;

PrimitiveID and InstanceID are system values that can be retrieved using the correct GLSL/HLSL API (SV_PrimitiveID and SV_InstanceID for DX11, and gl_PrimitiveID and gl_InstanceID for GLSL). Square brackets are used to indicate any variable that is declared using the aforementioned semantics.

Refer to the NormalView_Selection technique definition in the WaterSimulation shader example for a demonstration of how to select displaced geometry, and to draw vertex, edge, and face components at their correct displaced positions. For the GLSL version, see the WaterSimulation.ogsfx and PS_HWSelection.ogsfh files in the presets\GLSL\examples folder of the Maya installation directory. For the DX11 version, see the WaterSimulation.fx and PS_HWSelection.fxh files in the presets\HLSL11\examples folder.

Note: For more information, see camera selection mode in the Regular Draw Update section above.

Selection mask usage

MPxLocatorNode::getShapeSelectionMask() or MPxSurfaceShape::getShapeSelectionMask() can be overridden to return the desired selection mask for use in Viewport 2.0.

This can be either an existing internal mask or a custom one that is registered by using MSelectionMask::registerSelectionType(). For example, if a custom type called "my_selection_type" is registered, the getShapeSelectionMask() would return MSelectionMask("my_selection_type").

By default, custom selection types are disabled. To enable a custom selection type, do as follows:

"selectType -byName \"my_selection_type\" 1"

where "my_selection_type" is your custom type.

Also available with a custom selection type is the ability to set the priority via the selectPriority command. In this example, assume that the registration originally set the priority to be 2.

// Change it to be lower in priority 
selectPriority -byName "my_selection_type" 1;

// Change it to be higher in priority 
selectPriority -byName "my_selection_type" 3;

An example use case is when an environment encompasses the scene. The user may either not be able to or accidentally select the environment instead of the objects in the scene. To mitigate this, the environment's selection priority can be lowered.

Selection of a particular type can also be disabled to disallow selection:

"selectType -byName "my_selection_type" 0;

See the footPrintNode geometry override implementation (footPrintNode_GeometryOverride Developer Kit example) for an example use of MPxLocatorNode and a custom selection type. See apiDirectionalLightShape and apiMeshShape for example use of internal masks.