:::

詳目顯示

回上一頁
題名:兩種新的計算機化自適應測驗在線標定方法
書刊名:心理學報
作者:陳平
作者(外文):Chen, Ping
出版日期:2016
卷期:2016(9)
頁次:1184-1198
主題關鍵詞:全功能極大似然估計計算機化自適應測驗項目反應理論在線標定題庫建設Full functional maximum likelihood estimatorComputerized adaptive testingItem response theoryOnline calibrationConstruction of item bank
原始連結:連回原系統網址new window
相關次數:
  • 被引用次數被引用次數:期刊(0) 博士論文(0) 專書(0) 專書論文(0)
  • 排除自我引用排除自我引用:0
  • 共同引用共同引用:0
  • 點閱點閱:27
在線標定技術由于具有諸多優點而被廣泛應用于計算機化自適應測驗(CAT)的新題標定。Method A是想法最直接、算法最簡單的CAT在線標定方法,但它具有明顯的理論缺陷——在標定過程中將能力估計值視為能力真值。將全功能極大似然估計方法(FFMLE)與"利用充分性結果"估計方法(ECSE)的誤差校正思路融入Method A(新方法分別記為FFMLE-Method A和ECSE-Method A),從理論上對能力估計誤差進行校正,進而克服Method A的標定缺陷。模擬研究的結果表明:(1)在大多數實驗條件下,兩種新方法較Method A總體上可以改進標定精度,且在測驗長度為10的短測驗上的改進幅度最大;(2)當CAT測驗長度較短或中等(10或20題)時,兩種新方法的表現與性能最優的MEM已非常接近。當測驗長度較長(30題)時,ECSE-Method A的總體表現最好、優于MEM;(3)樣本量越大,各種方法的標定精度越高。
With the development of computerized adaptive testing(CAT), many new issues and challenges have been raised. For example, as the test is continuously administered, some new items should be written, calibrated, and added to the item bank periodically to replace the flawed, obsolete, and overexposed items. The new items have to be precisely calibrated because the calibration precision will directly affect the accuracy of ability estimation. The technique of online calibration has been widely used to calibrate new items on-the-fly in CAT, since it offers several advantages over the traditional offline calibration approach. As the simplest and most straightforward online calibration method, Method A(Stocking, 1988) has an obvious theoretical limitation in that it treats the estimated abilities as true values and ignores the measurement errors in ability estimation. To overcome this weakness, we combined a full functional maximum likelihood estimator(FFMLE) and an estimator which made use of the consequences of sufficiency(ECSE)(Stefanski & Carroll, 1985) with Method A respectively to correct for the estimation error of ability, and the new methods are referred to as FFMLE-Method A and ECSE-Method A. A simulation study was conducted to compare the two new methods with three other methods: the original Method A [denoted as Method A(Original)], the original Method A which plugs in the true abilities of examinees [Method A(True)], and the "multiple EM cycles" method(MEM). These five methods were evaluated in terms of item-parameter recovery and calibration efficiency under three levels of sample sizes(1000, 2000 and 3000) and three levels of CAT test lengths(10, 20 and 30), assuming the new items are randomly assigned to examinees. Under the two-parameter logistic model, the true abilities for the three groups of examinees were randomly drawn from the standard normal distribution [N(0,1)]. For all conditions, 1000 operational items were simulated to constitute the CAT item bank in which the item parameter vector were randomly generated from a multivariate normal distribution MVN(u,(50)) following the procedures of Chen and Xin(2014). Furthermore, the process of simulating and calibrating new items were replicated 100 times, and 20 new items were generated and the simulation method was the same as that of the operational items. Maximum Fisher Information method was employed to select the following items, and EAP method combined with MLE method was used to estimate the examinees’ abilities. Fixed-length rule was utilized to stop the CAT test. The results showed that the two new approaches, FFMLE-Method A and ECSE-Method A, improved the calibration precision over the Method A(Original) in almost all conditions, and the magnitude of improvement reached maximum when the test length was small(e.g., 10). Furthermore, the performance of the two new methods was very close to that of the best-performing MEM for small and medium-sized test length(i.e., 10 and 20), whereas ECSE-Method A had the best performance among all methods when the test length was relatively longer(i.e., 30). Also, larger sample size resulted in more precise item-parameter recovery for all online calibration methods. Though the simulation results are very promising, several future directions for research, such as variable-length CAT and more complex CAT conditions, merit investigation(e.g., including item exposure control, content balancing and allowing item review, etc.).
期刊論文
1.Mislevy, R. J.(1986)。Bayes modal estimation in item response models。Psychometrica,51(2),177-195。  new window
2.Chang, Y.-C. I.、Lu, H. Y.(2010)。Online calibration via variable length computerized adaptive testing。Psychometrika,75(1),140-157。  new window
3.Stefanski, L. A.、Carroll, R. J.(1985)。Covariate measurement error in logistic regression。The Annals of Statistics,13(4),1335-1351。  new window
4.Weiss, D. J.(1982)。Improving Measurement Quality and Efficiency with Adaptive Testing。Applied Psychological Measurement,6(4),473-492。  new window
5.陳平、辛濤(2011)。認知診斷計算機化自適應測驗中的項目增補。心理學報,43(7),836-850。  延伸查詢new window
6.Chen, Y. X.、Liu, J. C.、Ying, Z. L.(2015)。Online item calibration for Q-Matrix in CD-CAT。Applied Psychological Measurement,39(1),5-15。  new window
7.Chen, P.、Xin, T.、Wang, C.、Chang, H.-H.(2012)。Online calibration methods for the DINA model with independent attributes in CD-CAT。Psychometrika,77(2),201-222。  new window
8.Ban, J.-C.、Hanson, B. A.、Wang, T. Y.、Yi, Q.、Harris, D. J.(2001)。A comparative study of on-line pretest item-calibration/ scaling methods in computerized adaptive testing。Journal of Educational Measurement,38(3),191-212。  new window
9.Ban, J.-C.、Hanson, B. A.、Yi, Q.、Harris, D. J.(2002)。Data sparseness and on-line pretest item calibration-scaling methods in CAT。Journal of Educational Measurement,39(3),207-218。  new window
10.Chang, H.-H.(2015)。Psychometrics behind computerized adaptive testing。Psychometrika,80(1),1-20。  new window
11.Chang, H. H.、Qian, J. H.、Ying, Z. L.(2001)。a-stratified multistage computerized adaptive testing with b blocking。Applied Psychological Measurement,25(4),333-341。  new window
12.Chang, H. H.、Stout, W.(1993)。The asymptotic posterior normality of the latent trait in an IRT model。Psychometrika,58(1),37-52。  new window
13.陳平、辛濤(2011)。認知診斷計算機化自適應測驗中在線標定方法的開發。心理學報,43(6),710-724。  延伸查詢new window
14.陳平、張佳慧、辛濤(2013)。在線標定技術在計算機化自適應測驗中的應用。心理科學進展,21(10),1883-1892。  延伸查詢new window
15.Cheng, Y.、Yuan, K. H.(2010)。The impact of fallible item parameter estimates on latent trait recovery。Psychometrika,75(2),280-291。  new window
16.Jones, D. H.、Jin, Z. Y.(1994)。Optimal sequential designs for on-line item estimation。Psychometrika,59(1),59-75。  new window
17.Lien, D.-H. D.(1985)。Moments of truncated bivariate lognormal distributions。Economics Letters,19(3),243-247。  new window
18.Quellmalz, E. S.、Pellegrino, J. W.(2009)。Technology and Testing。Science,323(5910),75-79。  new window
19.田健全、苗丹民、楊業兵、何寧、肖瑋(2009)。應徵公民計算機自適應化拼圖測驗的編制。心理學報,41(2),167-174。  延伸查詢new window
20.van der Linden, W. J.、Ren, H.(2015)。Optimal Bayesian adaptive design for test-item calibration。Psychometrika,80(2),263-288。  new window
21.汪文義、丁樹良、游曉鋒(2011)。計算機化自適應診斷測驗中原始題的屬性標定。心理學報,43(8),964-976。  延伸查詢new window
22.游曉鋒、丁樹良、劉紅雲(2010)。計算機化自適應測驗中原始題項目參數的估計。心理學報,42(7),813-820。  延伸查詢new window
會議論文
1.Chen, P.、Xin, T.(2014)。A new online calibration approach for multidimensional computerized adaptive testing。The National Council on Measurement in Education。Philadelphia, PA。  new window
2.Guo, F. M.、Wang, L.(2003)。Online calibration and scale stability of a CAT program。The annual meeting of National Council on Measurement in Education。Chicago, IL。  new window
3.Parshall, C. G.(1998)。Item development and pretesting in a computer-based testing environment。The colloquium Computer-Based Testing: Building the Foundation for Future Assessments。Philadelphia, PA。  new window
研究報告
1.Stocking, M. L.(1988)。Scale drift in on-line calibration。Princeton, NJ:ETS。  new window
學位論文
1.陳平(2011)。認知診斷計算機化自適應測驗的項目增補--以DINA模型爲例(博士論文)。北京師範大學。  延伸查詢new window
2.Cheng, Y.(2008)。Computerized adaptive testing: new developments and applications(博士論文)。University of Illinois at Urbana-Champaign。  new window
3.Clark, R. R.(1982)。The errors-in-variables problem in the logistic regression model(博士論文)。University of North Carolina,Chapel Hill。  new window
4.Wang, C.(2012)。Semi-parametric models for response times and response accuracy in computerized testing(博士論文)。University of Illinois at Urbana-Champaign。  new window
5.Zheng, Y.(2014)。New methods of online calibration for item bank replenishment(博士論文)。University of Illinois at Urbana-Champaign。  new window
圖書
1.Wainer, H.、Dorans, N. J.、Flaugher, R.、Green, B. F.、Mislevy, R. J.、Steinberg, L.、Thissen, D.(1990)。Computerized adaptive testing: A primer。Lawrence Erlbaum Associates, Inc.。  new window
2.Baker, F. B.、Kim, S. H.(2004)。Item response theory: Parameter estimation techniques。New York:Dekker。  new window
3.Lord, Frederic M.(1980)。Applications of Item Response Theory to Practical Testing Problems。Lawrence Erlbaum Associates, Inc.。  new window
4.漆書青、戴海琦、丁樹良(2002)。現代教育與心理測量學原理。北京:高等教育出版社。  延伸查詢new window
5.Carroll, Raymond J.、Ruppert, David、Stefanski, Leonard A.、Crainiceanu, Ciprian M.(2006)。Measurement Error in Nonlinear Models: A Modern Perspective。Chapman & Hall/CRC。  new window
圖書論文
1.Chang, H. H.(2012)。Making computerized adaptive testing diagnostic tools for schools。Computers and their impact on state assessments: Recent history and predictions for the future。Charlotte, NC:Information Age。  new window
2.Flaugher, R.(2000)。Item pools。Computerized adaptive testing: A primer。Mahwah, NJ:Erlabum。  new window
3.Wainer, H.、Mislevy, R. J.(1990)。Item response theory, item calibration, and proficiency estimation。Computerized adaptive testing: A primer。Hillsdale, NJ:Erlbaum。  new window
4.Birnbaum, A. L.(1968)。Some latent trait models and their use in inferring an examinee's ability。Statistical theories of mental test scores。Addison-Wesley Publishing Company。  new window
 
 
 
 
第一頁 上一頁 下一頁 最後一頁 top