Abstract: In this paper, we propose a novel multimedia cross-reference retrieval system for digital libraries. The most important feature of the system is to integrate the multi-modal data into a seamless retrieval system. The cross-reference search engine, based on the cross-reference graph, serves as a bridge in the whole system. It calculates both the semantic and content level similarity between media object and the query. It is also able to adjust the cross-reference graph model based on relevance feedback conducted by users, which progressively improves the retrieval accuracy. As demonstrated by the experiments, the system can achieve satisfactory retrieval efficiency on multi-modal datasets.