Unicode

File Functions

String Functions

Date Functions

Conversion Functions

TSTR Strings

TSTR is a macro replaced either by WStr or CStr depending on if you have defined the Unicode symbol or not. You can still directly use CStr or WStr if you have to specifically store data in a particular format independent of the build mode. In addition, WStr and CStr implement a lazy-copying algorithm which makes it fast and cheap to copy strings inside the application.

Converting C Strings to TSTR

There are several distinct ways of storing and referencing a string in a Windows program. Examples are:

• Strings stored in ACP (active code page) format and referenced by char*.
• Strings stored in UTF-16 format and referenced by wchar*.
• Strings stored in BSTR (basic string objects that hold the size of the string in the object itself) and referenced by BSTR and so on.

The TSTR class provides some methods on converting between different types of string objects. You have to be careful, because conversion to ACP and TCHAR (in MBCS mode) can lead to losing some character information. The following static methods of TSTR will construct a TSTR object from different data types:

static TSTR TSTR::FromBSTR  (BSTR           string [, size_t length]);
static TSTR TSTR::FromACP   (const char*    string [, size_t length]);
static TSTR TSTR::FromUTF8  (const char*    string [, size_t length]);
static TSTR TSTR::FromOLESTR(LPCOLESTR      string [, size_t length]);
static TSTR TSTR::FromUTF16 (const wchar_t* string [, size_t length]);
static TSTR TSTR::FromUTF32 (const unsigned int* string [, size_t length]);
static TSTR TSTR::FromTCHAR (const TCHAR*   string [, size_t length]);
static TSTR TSTR::FromCP    (UINT cp, const char* string [, size_t length]);
static TSTR TSTR::FromCStr  (const CStr& string);
static TSTR TSTR::FromWStr  (const WStr& string);
static TSTR TSTR::FromMCHAR (const wchar_t* string [, size_t length]);
static TSTR TSTR::FromMSTR  (const WStr& string);

Converting from TSTR to Other Strings

The following is a list of TSTR member functions (methods) which return different string objects from the caller TSTR object. Except for "ToBSTR", all pointers returned by TSTR are valid as long as the TSTR object remains alive and unchanged. Note that these do not return copies of the data. Ensure you use the returned pointers promptly and avoid caching them.

C++ Generic Strings

You can also safely use the following macros to target generic C++ data types. These macros are compatible with both MBCS and Unicode configurations. These are defined in strbasic.h.

MaxString Strings

MaxString is the internal string data type for TSTR. CStr and WStr use this class to perform their internal string operations. This class uses an internal buffer that can hold a string in different encodings, and has a data member that tells which encoding is being used. Plug-in developers can use this class when they need to have string objects passed to/from 3ds Max. Some advantages of this class are:

• It is capable of holding the same string in different encodings.
• It reduces the number of copies of the same string in the memory by using a lazy copy algorithm. Several MaxString objects will point to the same physical memory location unless they need to modify the string. It is only then that an actual copy is performed.
• It prevents loss of information by performing optimized string operations. For example, concatenating a UTF-16 string with an ACP string will give a UTF-16 string.

Converting Strings From/To MaxString

The following static functions create and return a MaxString from other string types.

static MaxString MaxString::FromCP         (UINT codepage, const char*, size_t length = (size_t)-1);
static MaxString MaxString::FromACP        (const char*, size_t length = (size_t)-1);
static MaxString MaxString::FromUTF8       (const char*, size_t length = (size_t)-1);
static MaxString MaxString::FromUTF16      (const WCHAR*, size_t length = (size_t)-1);
static MaxString MaxString::FromUTF32      (const unsigned int*, size_t length = (size_t)-1);
static MaxString MaxString::FromWin32Error (DWORD err);
static MaxString MaxString::FromAnsiError  (int err);

You can use the MaxStringCast Template class to convert your MaxString objects to encoding-dependant string objects. The following functions can convert MaxStrings to MaxStringCast<typename ChType> or MaxStringCastUTF8.

MaxStringCast<char>         MaxString::ToACP  (size_t* length) const
MaxStringCastUTF8           MaxString::ToUTF8 (size_t* length) const
MaxStringCast<WCHAR>        MaxString::ToUTF16(size_t* length) const
MaxStringCast<unsigned int> MaxString::ToUTF32(size_t* length) const

MaxSDK::Util::TextFile::Reader

This class reads and interprets text files, and it was designed to perform file and stream I/O in a code page neutral way. It was designed to resolve the following problems:

• Reads and interprets correctly the Byte Order Mark (BOM, an invisible character at the beginning of Unicode files).
• Correctly detect UTF-8 and UTF-16 files, even if the file is not signed.
• Detect encoding cookies. XML files usually begin with "<?xml encoding='????'>" The detection algorithm will interpret this directive correctly.
• Prevent from splitting a character. In UTF-16, UTF-8 and some ANSI codepages, characters can be stored on 1 to 6 bytes. All the operations of this object avoid returning a partial character.

Plugin developers should consider using this class to perform File I/O to ensure that the files they generate remain compatible with previous versions of 3ds Max.

Opening Reader Files

The following lists the various methods for opening a text file using the MaxSDK::Util::TextFile::Reader class.

bool TextFile::Reader::Open(
    const MCHAR* fileName, 
    unsigned int encoding = 0, 
    LineEndMode mode = Text
);

bool TextFile::Reader::Open(
    FILE* file, 
    unsigned int encoding = 0, 
    LineEndMode mode = Text
);

bool TextFile::Reader::Open(
    HANDLE fileHandle, 
    unsigned int encoding = 0, 
    LineEndMode mode = Text 
);

The parameters of the above functions are described as follows:

• fileName/file/fileHandle - The first parameter specifies the file to open. The TextFile::Reader class will parse them to determine their format.

• encoding - A flag to give hints to the detection algorithm. Acceptable values for this parameter are all of the code page numbers recognized by MS Windows. In addition to that, you can also set one of the following flags:

  • FAVOR_UTF8 = 0x10000000 - If not used, the function assumes file encoding is ACP when the file's encoding cannot be detected. Setting this will force this function to use UTF8 when the encoding cannot be automatically detected.
  • FOUND_BOM = 0x20000000 - Tells the function that there is a BOM mask at the beginning of the file.
  • FOUND_COOKIE = 0x40000000 - Tells the function that there is a BOM mask at the beginning of the file.
  • FLIPPED = 0x80000000 - Tells the file if flipped (big endian) UTF-16.

  For example, if you specify CP_ACP | FAVOR_UTF8, the detection algorithm will treat any non-UTF8 data as ACP.

• mode - the last parameter specifies type of the end of line characters used in the file. Different arguments are:

  • Unchanged - Do not alter the end of line character sequence when reading the file.
  • Enforce_CRLF - Ensure that all lines are terminated by the CRLF character sequence.
  • Enforce_LF - Ensure that all lines are terminated by the LF character.
  • Text - Enforce_CRLF (default in Windows).

Querying Reader Encoding

You can also use the function unsigned int MaxSDK::Util::TextFile::Reader::Encoding() to inquire about the encoding of this object. For reading from the text files, you can use one of the following functions:

char TextFile::Reader::ReadChar              (bool peek = false);
MaxString TextFile::Reader::ReadChars        (size_t nchars);
unsigned int TextFile::Reader::ReadCharUTF32 (bool peek = false);
MaxString TextFile::Reader::ReadChunk        (size_t len, bool dontReturnLastEOL = false);
MaxString TextFile::Reader::ReadFull         ();
MaxString TextFile::Reader::ReadLine         (size_t nchars = (size_t)-1, bool dontReturnEOL = false);

MaxSDK::Util::TextFile::Writer

TextFile::Writer is used to write to any Unicode and MBCS text file format. It automatically performs the conversion between TCHAR and the underlying format. As for the TextFile::Reader class, it will produce a MBCS ACP file by default unless you change the "encoding" parameter to another format like CP_UTF8 or MSDE_CP_UTF16. If TextFile::Writer is set to append or overwrite a file, it will detect the file format of the existing file and continue using that particular format. This class avoids mixing different encodings within the same file, and also intelligently converts strings into binary data. Its open functions are similar to TextFile::Reader's. It has the following file writing functions:

size_t Write (const char *string, size_t nbchars=(size_t)-1);
size_t Write (const wchar_t *string, size_t nbchars=(size_t)-1);
size_t Write (const MaxString &string);

MaxSDK::Util::TextFile::ReaderWriter

While the TextFile::Reader and TextFile::Writer are used for the occasions where you only need to either read or write from/to Unicode text files, the TextFile::ReaderWriter class both reads and writes Unicode files. Please refer to the class documentation for a detailed list of its functionality.

sizeof() and _countof()

• sizeof is used in conjunction with memory allocation functions such as: malloc, memset, memcpy, etc.
• _countof is used in functions which take both a TCHAR pointer and a size parameter. This is a standard across Microsoft and 3ds Max code. Every time a function needs the length of a TCHAR pointer, it's always in number of TCHAR, and not a number of bytes. For example: LoadString.

Varying Character Memory Usage

Developers can make mistakes when they write code which processes strings. For instance, it is possible to assume that characters use a constant amount of memory. In the case of WCHAR, this is assumption is false. For example:

• MBCS characters can take 1 to 2 bytes.
• UTF-8 characters can take 1 to 6 bytes.
• UTF-16 (WCHAR) characters might take from 2 to 4 bytes.

Operations which deal with strings might also yield different string lengths. As such, do not assume that 1 character is equal to 1 byte. For example, changing the casing of a string might change its size in memory. Avoid using sizeof to determine the length of a string.

reinterpret_cast()

When adding an offset in bytes to wchar_t pointers, you will effectively add twice the value. This is why you will see many occurrences of:

reinterpret_cast<TCHAR *>(reinterpret_cast<char *>(file) + sizeof(astruct));

Note that sometimes, reinterpret_cast<char *> is missing (only the TCHAR version is present). It was decided that all occurrences of reinterpret_cast<char *> are to be replaced with reinterpet_cast<byte *>.

unsigned char*

If for any reason you must convert a char* to an unsigned char*, you have to use _TUCHAR. It's the TCHAR equivalent which guarantees that in both Unicode and ASCII versions, all characters are unsigned and non-negative.

TSTR::data() is now const

To prevent invalidating the pointers inside TSTR, the data() method now returns a const MCHAR*. Avoid casting the const string to a normal string.

GetProcAddress()

The Windows function GetProcAddress() function does not perform any kind of handling when it comes to retrieving Unicode or ASCII functions. You must explicitly handle both cases, whether you have a Unicode or an ASCII build. Also, GetProcAddress() only accepts "char" (NOT Unicode) as input. If you have a TCHAR pointer, you need to convert it before using it. For example:

#ifdef UNICODE
functionPointer = (functionProto)GetProcAddress("FunctionW");
#else
functionPointer = (functionProto)GetProcAddress("FunctionA");
#endif

Avoid calling TSTR constructors without the _T macro

When you instantiate TSTR object, you have to take attention to implicit conversion. The following code should be avoided. In ASCII mode, it's fine because the TSTR object is natively storing all its data in ACP. In Unicode mode, this source code will trigger a conversion from ACP to UTF-16.

// Avoid this:
TSTR stringObject = "some string";
TSTR stringObject2("another string");

You should enclose your string inside the _T macro. The conversion to UTF-16, if needed, is performed at compile-time instead. The following is proper use of the _T() macro to wrap string literals:

// Do this:
TSTR stringObject = _T("some string");
TSTR stringObject2(_T("another string"));

Implicit TSTRs

If an operation doesn't imply a form of convention, for example ACP to UTF16, we favor the implicit constructors and conversions of string objects.

// Avoid this:
FunctionCall(TSTR(GetString(IDS_STRING)));
TSTR stringObject = TSTR(_T("string"));

// Do this:
FunctionCall(GetString(IDS_STRING));
TSTR stringObject = _T("string");

String Concatenation

TSTR objects have concatenation operators for both TCHAR* and TSTR. If you concatenate a TCHAR* to a TSTR, you don't have to explicitly construct an intermediate TSTR object.

// Avoid this:
TSTR o1 = FunctionCall();
TSTR o2 = o1 + TSTR(FunctionCall()) + TSTR(_T("string"));

// Do this:
TSTR o1 = FunctionCall();
TSTR o2 = o1 + FunctionCall() + _T("string");

VARIANTs

Since 3ds Max is relies on Windows, many places inside 3ds Max use the OLE variant types: VARIANT, VARIANTARG, and PROPVARIANT. Inside the 3ds Max SDK, we have functions to convert intelligently between VARIANTs and TSTR objects.

namespace MaxSDK { namespace Util {
    TSTR VariantToString(const PROPVARIANT*, UINT encoding=CP_ACP, USHORT flags=0);
    bool VariantIsString(const PROPVARIANT*);
    bool VariantIsStringVector(const PROPVARIANT*);
    bool SetStringToVariant(PROPVARIANT*, const char* str, bool clear = true);
    bool SetStringToVariant(PROPVARIANT*, const wchar_t* str, bool clear = true);
}}