When working with xref data, you should store the source drawing with no labels. Add the labels after you have inserted the xrefs into your current drawing. If you decide to include labels in the source drawing, they should reside on a layer that can be turned off. Xref labels from the source drawing cannot be edited. After inserting xrefs, use the labeling commands as you would for other objects.
The recommendation to add labels after inserting xrefs into your current drawing applies when working with plot xrefs. Use the source drawing as a canvas for your object data and avoid the clutter of annotation. For more information about labeling plot xrefs, see Labeling Plot External References (Xrefs) in the Plots chapter.
All xrefs transfer layers into the new drawing whether they are used or not. Components always reference layers, and all layers are transferred into the new drawing.