This paper proposes a novel neural computational model for tracking frequency-modulated (FM) tones. The model can explain various interesting phenomena related to the perception of complex FM tones. The dynamic process in perceiving the pitch of FM tones can be represented by two second-order systems having fast and slow responses [K. Aikawa et al., J. Acoust. Soc. Am. 98, 2926 (1995)]. An auto-regressive neural matrix model is newly proposed to simultaneously track multiple FM tones. The model is characterized by a novel neural network architecture called the counter-tonotopic connection and a new tracking algorithm based on the Lp-norm. Each FM tone is tracked with second-order response characteristics. Several interesting phenomena in perceiving complex FM tones have been reported. For a stimulus tone composed of crossed upward and downward sweep tones, two separate pitch streams were not perceived but a bounced pitch stream was. When both of the crossed sweeps were downward sweeps, two pitch streams were merged and then another new stream was perceived after the crossing point [Matsui, ASJ meeting, 1995-09]. When a sweep tone was followed by white noise, the sweep was perceived as being extended into the noise [Masuda, ASJ meeting, 1995-09]. The proposed neural tracking model successfully replicated these perceived images.