One of the most important issues facing ocean acoustic experimentalists when attempting to perform calibrated experiments is distinguishing measurement errors from environmental variabilities. These errors include accuracy of monitoring hydrophones, ship positioning (hanging sources), and, for a broadband source in particular, spectral level and phase variations of the source. Statistical analysis is used to investigate the repeatability of source and receiver signatures collected from a series of broadband experiments performed in very shallow water at the Atlantic Generating Station (AGS) site. Classification of the factors inducing signal variation, which may arise from surface and bottom roughness and time-varying volume inhomogeneities, in addition to source errors, is made for several experimental configurations. It is shown that the coherence of the received signals is directly related to the environmental effects on the acoustic waves and the shot-to-shot correlation decreases with increasing range.