【Purpose/significance】Under the big data environment, a better multi-media information retrieval system is one of the core aspects for promoting digital libraries’ interactivities and impelling the transformation of its knowledge services.【Method/process】After investigating several famous digital libraries, it finds out two key problems still remain in the traditional multi-media retrieval system. The first is"the useful cross-modal semantic information wasn’t applied in the retriev-al procedure". The second is"the multi-media resources in digital library weren’t organized and managed systematically".To resolve the problems and improve retrieval performance in some extent, it proposes several novel ideas for optimizing thetraditional multi-media information retrieval system:"cross-modal correlation analysis","hierarchical knowledge reasoning", et al. Detailed empirical analysis is done to verify the presented novel ideas.【Result/conclusion】Retrieval performances are improved apparently. It means that several modern technologies such as deep learning and knowledge represen-tation learning actually contribute to optimize the traditional multi-media information retrieval system of digital library.More importantly, it can better satisfy users’ knowledge demands and improve the knowledge service quality of digital library.