Script and right-click menu for static switching of spaces

So one of the problems inherent with switching spaces is that during animation the orientation of the limb or the position of the controller jumps as the space is changed. As mentioned before, by changing space you are effectively changing the coordinate system that the node exists in so a given set of translate/rotate values will give a different position/orientation from one space to the next. This leaves the animators having to manually re-align a joint/controller if they want to change spaces mid animation.

The way to fix this is to supplement the switch with a script solution that manages the space change and automatically sets new translate/rotate values to preserve the position/orientation of the controller. So this is easily accessible for the animators I'll also cover how to link this in to the dag menu (right-click menu) for the given controller.

I'll be using the left FK arm as an example, if you've not had a look at setting up a space switch you can do so here.

There are three parts to setting this up:

1. Modified dagMenuProc.mel that calls procedures in customDagMenu.mel depending on the node clicked on.
2. customDagMenu script that contains procedures for inserting context sensitive menu items
3. Switch and match script.

... and when it's in place, what happens is this:

1. User right clicks on node.
2. Procedure createSelectMenuItems in our modified dagMenuProc.mel is called.
3. If node is space switchable, our procedure is called to construct custom menu items - one item per space available to switch to for that node.
4. If user clicks on one of the menu items, the name of the target space and node to switch is passed to the switch script. This changes the space and re-aligns the node to maintain its global position between spaces.

 First off, however, an extra attribute spaceNode needs to be added to the joint driver_l_arm.

The purpose of this extra attribute is (a) to highlight it as a switchable node and therefore to call the switch menu procedure and (b) to act as a pointer to the node that has the space attribute. Usually the attribute is directly on the node itself as is the case here, in which case the string will have the same name as the node.
Sometimes however, the attribute is on an instanced shape such that the switch is accessible on multiple areas of the rig. In this case the space attribute isn't on the node that the animator clicked on, and so spaceNode might contain the string l_arm_spaceShape and that shape would be instanced to multiple transform nodes including driver_l_arm.
Actually, that would make a good post in itself... Sometimes it's kind of useful to have the same attribute accessible in more than one place on a rig and it's something worth covering.


Part 1. Modifying Maya's dagMenuProc.mel

So, part one is the modified script dagMenuProc.mel. This is a customised version of the standard Maya script that builds the right-click menu which, depending on your install path, can be found C:\Program Files\Autodesk\Maya2008\scripts\others. If you haven't already, check out this post to see how I go about structuring and sourcing these.

The procedure we're going to modify is called global proc createSelectMenuItems(string $parent, string $item) and it should be around line 500. The start of this procedure is defining the type of node that has been clicked on - joint, nurbs, poly etc. etc. and after that quite a few if statements defining what do do for each object type. As we want to inject some custom menu items for driver_l_arm we want to find the part that deals with joints which should be around line 630. As other parts of my control rig consist of nurbs curves these extra lines of script also reside after if ($isNurbsObject) { at around line 580.
Here is a section copied from the script, my additions highlighted:

                    -ecr false
                    -c ( "doMenuLatticeComponentSelection(\"" +
                         $item + "\", \"" +  $maskList[$i] + "\")")
                    -rp $radialPosition[$i];
            }
        }
    } else if ($isJointObject) {

// *************************************************************************************  NT CUSTOM DAG ITEMS        
        
        if (`attributeExists "spaceNode" $item`) {
            string $NT_scriptRoot = NT_getScriptRoot();
            string $sourceString = ("\""+($NT_scriptRoot)+"Interface/NT_customDagMenu.mel\"");
            eval ("source "+($sourceString));
            NT_Space_dag_menu($item);
        }
        
// ************************************************************************************* END NT CUSTOM DAG ITEMS    
        

        string $setCmd = `performSetPrefAngle 2`;
        string $assumeCmd = `performAssumePrefAngle 2`;      
        $setCmd += (" "+$item);
        $assumeCmd += (" "+$item);
        string $jts[] = `ls -sl -type joint`;
        for ($jointItem in $jts) {
            if ($jointItem != $item) {

So here you can see the added lines between the two lines of stars. (It's actually a good idea to highlight modifications to a standard script in this way as it makes your life quite a lot easier when it comes to editing later, often to understand why lots of default Maya UI stuff is mysteriously broken...)
This section is pretty simple:

• Checks for the spaceNode attribute I mentioned earlier which marks it out as being space switchable.
• Calls the procedure NT_getScriptRoot that simply returns a string of the root scripts folder and from there I define the full path to the script NT_customDagMenu which builds the custom menu items.
• Sources NT_customDagMenu.
• Calls the procedure NT_Space_dag_menu that is in the script NT_customDagMenu.mel. The string argument ($item) that is passed is the name of the node that was right-clicked on.

Part 2. Building custom menu items

This is the procedure NT_Space_dag_menu in NT_customDagMenu.mel that is called when a space switchable joint is right-clicked. 
The string variable $item is passed from NT_dagMenuProc_dag_menu and contains the name of the node that was right-clicked.
I'll go through it step by step, but here it is in its entirety.

global proc NT_Space_dag_menu(string $item) {
    string $scriptRoot = NT_getScriptRoot();
    string $namespace = "";
    string $tokenized[];
    tokenize $item ":" $tokenized;
    if (`size($tokenized)` > 1) {
        $namespace = (($tokenized[0])+":");   
        $item = $tokenized[1];
    }
    // the node which contains the space attribute
    string $spaceNode = `getAttr (($namespace)+($item)+".spaceNode")`;
    // current space in string form
    string $currentSpaceString = `getAttr -as (($namespace)+($spaceNode)+".space")`;
    // list of all spaces available on this node
    string $allSpaces[] = `attributeQuery -node (($namespace)+($spaceNode)) -le "space"`;
    string $tempString = $allSpaces[0];
    $allSpaces = stringToStringArray($tempString, ":");
   
    // build menu items
    menuItem -d true;
    for ($counter = 0; $counter < `size($allSpaces)`; $counter ++) {
        if ($allSpaces[$counter] != $currentSpaceString) {
                menuItem -tearOff 0 -allowOptionBoxes true -subMenu false -bld true -l $allSpaces[$counter]
                -c ("source \""+$scriptRoot+"Animation/NT_space_switch.mel\"; NT_space_switch(\""+($item)+"\", \""+($namespace)+"\", \""+($spaceNode)+"\", \""+($allSpaces[$counter])+"\", \""+($counter)+"\");");
            setParent ..;
        }
    } 
}
Using the tokenized command I determine the namespace of the node.

1. Grab the name of the node with the space attribute
2. ...the currently active space
3. ...and a list of all spaces in string form.
4. Generate menu items using a for loop, one for each space except the one currently active.

Each menu item's click command sources and calls the procedure NT_space_switch in NT_space_switch.mel, passing the following arguments:

• $item - the right-clicked node name.
• $namespace - the namespace of the node.
• $spaceNode - name of node that has the space attribute
• $allSpaces[$counter] - space to be switched to in string form.
• $counter - space to be switched to in int form.

(item and $allSpaces[$counter] are only used for debug prints)




Part 3. The switch and position match script

Again, here is the script:.
global proc NT_space_switch(string $item, string $namespace, string $spaceNode, string $newSpaceString, int $newSpaceInt)
{
    string $currentSelection[] = `ls -sl`;
    int $debug = 0;
    if ($debug) {
        print ("\nNT_space_switch debug:\n\t$item = "+($item)+"\n");
        print ("\t$spaceNode = "+($spaceNode)+"\n");
        print ("\t$newSpaceString = "+($newSpaceString)+"\n");
        print ("\t$newSpaceInt = "+($newSpaceInt)+"\n");
        print ("\t$namespace = "+($namespace)+"\n");
    }
   
    string $locNameArray[] = `spaceLocator -n "posStoreLoc"`;
    $tempCon = `parentConstraint (($namespace)+($item)) $locNameArray[0]`;
    delete $tempCon;
    setAttr (($namespace)+($spaceNode)+".space") $newSpaceInt;
    if (`getAttr -se (($namespace)+($item)+".translate")`) {
        $tempCon = `pointConstraint $locNameArray[0] (($namespace)+($item))`;
        vector $posVector = `getAttr (($namespace)+($item)+".translate")`;
        delete $tempCon;
        setAttr (($namespace)+($item)+".translate") ($posVector.x) ($posVector.y) ($posVector.z);
    }
    if (`getAttr -se (($namespace)+($item)+".rotate")`) {
        $tempCon = `orientConstraint $locNameArray[0] (($namespace)+($item))`;
        vector $oriVector = `getAttr (($namespace)+($item)+".rotate")`;
        delete $tempCon;
        setAttr (($namespace)+($item)+".rotate") ($oriVector.x) ($oriVector.y) ($oriVector.z);
    }
    delete $locNameArray[0];
    select -r $currentSelection;
}

1. Store current selection to later restore
2. Debug print stuff
3. Create a temporary locator and snap it to the node positon and orientation. This stores the position and orientation of the node before the space is switched.
4. Set node to new space. At this point the node position and/or orientation will jump.
5. Check that rotation and translation on the node are settable (as trying to set translate values on driver_l_arm, for example, will cause the script to error as those channels are locked).
6. Constrain the node to the temporary locator, point and/or orient depending on which channels are settable. An FK arm would just be just orient, an IK controller would be both.
7. Store translate/rotate values of node in a vector variable. So now we have the translate/rotate values that are needed to preserve the world-relative position/orientation in the new space.
8. Delete the constraint.
9. Set translate and/or rotate attribute on the node to the values stored in the vector variables.
10. Delete the temporary locator
11. Restore previous node selection.

Here is a capture showing the space switch. First I'm switching spaces directly on the space attribute, you'll notice the orientation jump. Second I'm using the custom menu which switches the space while maintaining the world-relative orientation.