Go to: Synopsis. Return value. MEL examples.

Synopsis

tokenize string [string] string[]

tokenize is NOT undoable, NOT queryable, and NOT editable.

This command will split the first string argument up according to split characters provided in the optional second argument. If this argument is not provided, tokenize will use a default split string consisting of whitespace characters.

The input string is scanned for substrings (tokens) which are separated by any of the split characters. Note: tokenize does not match the entire split string; it matches any character in the string.

The resulting token strings are put into the third argument which is a string array. The return value of this procedure is the number of tokens into which the original string is divided.

Return value

intNumber of tokens

MEL examples

string $buffer[];
$numTokens = `tokenize "A/B//C/D" "//" $buffer`;

// Buffer will contain 4 strings, not 2: "A", "B", "C", "D"
// and $numTokens will be 4.

string $buffer[];
$numTokens = tokenize("Mildred Pierce Femme Fatale", $buffer);

// Buffer will contain 4 strings: "Mildred", "Pierce", "Femme", and "Fatale."
// and $numTokens will be 4.

string $buffer[];
$numTokens = `tokenize "testing=non-default separators" "=" $buffer`;

// Buffer will contain 2 strings: "testing" and "non-default separators."
// and $numTokens will be 2.