Refractive materials, like glass, usually involve reflections as well. As a result, these materials are more expensive (in time and resources) to render than others. Also, the average pane of glass has 2 layers or sides, requiring multiple layers of refraction. When rendering an image, all of the layers must be calculated so you can see through the glass. For example, you need at least 6 refractions to see through 3 panes of solid glass.
When rendering an image, you can specify the number of bounces off reflective surfaces (Maximum Number of Reflections), and the number of panes of glass for refractions (Maximum Number of Refractions). Higher settings generally result in longer render times. Blurry refractions increase render time further. (In the Render Quality Settings dialog, use the Reflections and Transparency options. See Render Quality Settings.)