The bandwidth of ultrafast pulses in the UV is limited by the finite acceptance bandwidth of the nonlinear crystals used for their generation. For fundamental laser pulses it is well established that spectral broadening can be used to overcome intrinsic bandwidth limits. We show that self-phase modulation of UV pulses in bulk materials leads to large spectral broadening and allows for a significant reduction of the pulse duration. We find that for pulse energies in the range of a few mu J, a thin crystal is favorable due to the strong dispersion in the UV and the limitations set by self-focusing. In contrast to spectral broadening in gaseous media, the self-focus has to lie outside the crystal to avoid beam break up. We focus UV pulses into a 1 mm thick CaF2 crystal. For moderately short input pulses, a shortening factor up to 2.4 is achieved: the 120 fs long third harmonic output of a Ti: sapphire amplifier is compressed down to 50 fs FWHM. For a central wavelength of 315 nm, we generate pulses as short as 14.9 fs after compression with an UV pulse shaper. In both cases the resulting beam shape is close to Gaussian and fully usable for spectroscopic experiments. We use the pulses in a collinear 2D-UV experiment and clearly resolve vibronic off-diagonal peaks of the S-2 B-1(2u) vibronic progression of pyrene.