If you move the mouse, the camera rotates in that direction until you move the mouse again. (The view of the panorama moves in the opposite direction from the mouse and camera.)
With the right button, you must drag the mouse to see any movement, and the view of the panorama moves in the same direction as the mouse.