Studies of the propagation and attenuation of high-frequency phonons in bulk materials have shown that they are subject to frequency down conversion and scattering due to anharmonic effects and impurities in the crystal. Recently, there have been reports on high-frequency surface phonon propagation concentrating on the issue of phonon focusing. The first time-of-flight measurements are reported of high-frequency surface acoustic phonons in (001) high-quality silicon wafers at low temperatures. Pulsed laser excitation is used for the generation of the high-frequency (>50 GHz) phonons, and aluminum edge bolometers are used as detectors. The time defined by the fast initial response of the edge bolometer as the position of the optical excitation is varied corresponding to acoustic delays in silicon. The results provide some insights on the propagation and attenuation of high-frequency surface waves, as well as on designing more efficient, high-temperature, high-frequency phonon transducers.