Abstract: A new method of describing and matching spatial context is proposed to effectively improve the distinguishability of objects in visual words-based image retrieval.Firstly,we define an affine covariant spatial neighborhood to obtain effective spatial context description.Secondly,fuzzy- earth mover's distance metric is presented to calculate their similarity,and reduces the effect of features' false clustering.Besides,noise regions are filtrated based on entropy and self-similarity as pretreatment.Experimental results demonstrate that,compared with existing methods,the relative improvement of our method's average retrieval precision is 10.8%.