Drag horizontally in all modes to slide the view forward and backward in time. Drag vertically to slide the view up and down in value.
If you have a middle mouse button, pressing it and dragging instantly pans in Track View, just as it does in viewports. (If you are in Maya mode, use + to pan.) Use ++ to zoom (Maya mode: +).
You can use the wheel on a wheel mouse to zoom and pan the Track View Key window. Drag with the wheel to pan, and roll the wheel to zoom.
To pan in the Key window: