Abstract: In the paper, a new algorithm for estimating the optical flow of image sequences is presented, by combining Gabor filter banks with weighted least-squares. The results of Gabor filter banks are used as the new features of image for the computation of the spatial-temporal phase gradient. In order to obtain more accurate results, the optical flow of sequence image is estimated by the method of weighted least-squares. The results of the experiments indicate that more accurate results of optical estimation can be obtained compared with the methods suggested by Lucas, Fleet, et al.