Abstract: Image matching is a key problem of computer vision and is frequently used in 3D model reconstruction， object recognition，image alignment，camera sel-f calibration and so on．Feature point matching is the most common one among all kinds of image matching．Its matching result is affected greatly by many factors，such as object occlusions，lighting conditions and noises．In this paper we extend the standard assignment algorithm to solve extended assignment problem．It employs the condition that the depth of the scene is local continuous as extra constraint，and uses the method for extended assignment problem to do global optimization．Moreover，this algorithm only needs to carry out optimization twice and can be implemented almost completely with matrix computation，so its efficiency is higher than existing algorithms．Experiments show that the results of the algorithm are highly satisfactory．