Modal is a command state or dialog that affects all operations. You must change the mode or dismiss the dialog before doing any operations that don't pertain to that mode.
Modeless means a nonexclusive command state or dialog. You can do other operations and commands without changing the mode or dismissing the dialog.
For example, the Track View and Material Editor dialogs are modeless, while the Open File dialog is modal.