在國內資訊檢索研究已日趨受到重視，卻缺少合適的測試評估機制之背景下，本
文參考國外各測試集的結構、特性與建構經驗，設計中文資訊檢索測試集的方法與程序，發
展華文世界第一套中文資訊檢索測試集，實際地進行測試集的規劃建置工作。本研究所建立
的測試集包括 132,207 篇新聞文件、50 個查詣主題、以及文件與查詣主題間的相關判斷，
平均每個查詣主題有 16.32 篇相關文件。 研究結果顯示，以統計抽樣的觀點，本測試集的
文件數量具有一定的效度；查詣主題呈現詳盡且多樣化的查詣需求，能反映真實的檢索情況
；由三位判斷者進行的相關判斷具有顯著的一致性，推斷它們是具有可佔度的。本測試集不
僅具有可應用性，並能作為進一步資訊檢索與評估研究之基礎。

The research and development of information retrieval has made
considerable progress recently.However, there is not any applicable test
mechanism for system evaluation in the Chinese research society. This paper
reports our research on the design and implementation of the first Chinese
information retrieval benchmark. According to the framework and contents of the
existing foreign benchmarks, we develop a methodology to establish the Chinese
IR benchmark. An IR benchmark consists of three parts： document set, topic set,
and relevance judgments. our document set contains 132,207 documents collected
from news web sites, the topic set contains 50 topics transformed from real
users' information needs, and each topic has on the average16.34 related
documents as a result of the relegance judgments. The results of our research
show that the quantity of document set is valid from the viewpoint of sampling
statistics. The topics reveal multiple kinds of information need, and they also
reflect certain real retrieval environment. Besides, the judgments given by
three judges have exhibited significant consistency, so we conclude their
reliability. Although the benchmark is in its first edition, it possesses a
complete structure and medium scale. On this basis, it is readily feasible to
expand this benchmark's current scale to a proper large one in the near future.