Abstract: We identify the single-layer fabric structure given as a color image, then identification of the double-layer fabric is regarded as the combination of several single-layer fabrics. Color clustering is used to segment the fabric image, and a new algorithm of warp yarn segmentation is proposed to perform the identification more precisely. Finally, with human-computer interaction, a method of semi-automatic identification of double-layer fabric based on the surface images of the fabric is proposed. Experimental outcome shows availability of the method.