The P_SUBTEX parameter specification flag is used in materials or texture maps that contain other texture maps as parameters, in which these 'sub' maps are not stored in the ParamBlock2 and are not accessible as direct references on the parent map or material, but are accessible via the MtlBase::GetSubTexmap() /MtlBase::SetSubTexmap() protocol.
Specifying this P_SUBTEX flag signals this situation and looks for a p_subtexno parameter option to give the sub-texture map number for the map. For example, the Standard material stores all its maps in a separate structure that appears as a single reference in the Standard material. In this case, the individual texture maps are not direct references on the material, but are accessible on the material via MtlBase::GetSubTexmap() / MtlBase::SetSubTexmap().
Note that the parameter specification flags P_OWNERS_REF and P_SUBTEX are both primarily intended for use when re-coding existing plug-ins in which the sub-maps are already managed by the owner in some way. If you are implementing a new plug-in, you should just let the ParamBlock2 host them for you and then you don't have to bother with either of these flags.