臺灣人文及社會科學引文索引資料庫系統

:::

詳目顯示

回上一頁
第 1 筆 / 總合 1 筆   第一頁 上一頁 下一頁 最後一頁
/1頁
題名：大學質性評鑑的挑戰：系所與外部專家評鑑觀點的一致性分析
書刊名：當代教育研究季刊
作者：陳慧蓉 引用關係
作者(外文)：Chen, Karen Hui-jung
出版日期：2016
卷期：24:2
頁次：頁75-109
主題關鍵詞：質性研究；外部專家；高等教育；系所評鑑；信實度；確實性；一致性；Qualitative evaluation；Peer reviewers；Higher education；Program accreditation；Trustworthiness；Credibility；Consistency
原始連結：連回原系統網址new window
相關次數：
  • 被引用次數被引用次數:期刊(1) 博士論文(0) 專書(0) 專書論文(0)
  • 排除自我引用排除自我引用:0
  • 共同引用共同引用:2908
  • 點閱點閱:16
研究目的　質性評鑑受到質性研究的理念影響，質性研究的信實度已從過去客觀中立事實的認定，轉而追求詮釋與推論的確實性與可靠性，亦即質性研究者能從被研究者的立場詮釋，使雙方意見具有一致性。我國系所評鑑採行認可制，由熟悉大學治理的專家進行質性品質的判斷，以確保高等教育品質。雖然系所評鑑運用了質性研究的理念，以資料的三角校正等方法設計評鑑流程，然而，質性評鑑的品質仍然受到各界關注。由於評鑑專家是在社會情境脈落之下進行評鑑，專家本身的社會文化及個人信念，有可能帶入評鑑過程中。為尊重受評系所的意見，我國評鑑設計了申復機制，系所可藉由申復讓專家重新考量。本研究的目的是探討系所自評與外部專家評鑑觀點之異同，從訪評意見、學校申復、及申復回覆的意見往返，探討學校與外部專家意見的一致性，了解大學質性評鑑的挑戰。研究設計／方法／取徑本研究運用內容分析法，分析2006至2010年第一週期系所評鑑1,156個班制的申復報告及其相關回覆。研究發現或結論　研究發現，系所申復報告主要說明大學提供資訊或質疑專家引用資訊的確實性（佔68.4％），次要是質疑評鑑結論與決定的公正性 （佔13.4％），第三是說明系所情境脈絡或質疑專家未考慮學校情境（佔12.0％）。而專家對於申復的回覆，主要說明評鑑報告的結論與決定的（佔48.4％），次要是說明資訊確實性的考量情形（佔31.0％），第三是詳細說明評鑑推論（佔12.7％）。交叉檢視申復及回覆的往返內容，結果顯示系所評鑑運用了增進質性研究信實性的方法設計評鑑流程，然而，專家在執行上仍然面臨了四項威脅：一、未能考慮 情境脈絡；二、檢視的是結果而非動態歷程；三、專家價值觀的涉入；四、評鑑歷程缺乏長期投入與互動。研究原創性／價值性　良好的大學質性評鑑，需要大學與專家具備相當的評鑑專業知能，才能從根本審視學校問題，促進學校改善。本研究以分析當前大學質性評鑑的執行情形及面臨的挑戰為例，提出增進質性評鑑品質的建議，以期作為改進評鑑機制及擬訂相關政策之參考。
Purpose Qualitative research is a method of inquiry employed in the qualitative evaluation. The quality of qualitative research was shifted from emphasizing on reliability and validity based on neutral facts to trustworthiness of inferences based on the consensus of qualitative researchers and participants. Taiwanese higher education adopts accreditation approach for educational quality assurance. Having experts with deeply understanding of the field being evaluated, peer review is able to make fair judgment and comments in external evaluation to assure the higher education quality. Although the accreditation approach applies the conceptions of qualitative research methods, such as triangulation for data collection during an evaluation procedure, the quality of qualitative evaluation has been challenged. It has been noticed that reviewers often conducted evaluations in a situated context and inevitably brought their own socio-cultural history and beliefs into the evaluation process. In order to remedy this problem, an objection and appeal system has been developed in Taiwanese higher education evaluation. The institution being evaluated can file an objection if they considered that the evaluation result is inconsistent with the facts. This study aims to investigate the major issues emerging from the objection reports and the corresponding replies, as well as to examine the gap between the educational quality of peer reviewers and institutional staff based on the objection reports. It provides an exploration of challenges of qualitative evaluation of higher education. Design/methodology/approach Applying content analysis methods, this study analyzed the objection reports of 1,156 programs during the 1st cycle of the program accreditation from 2006 to 2010. Findings Results showed that the institutions being evaluated tried to alter the conclusion of the on-site-visit reports by questioning the credibility of on-site visit report and providing more evidence (68.4%), weakening the justification of the review (13.4%), and asking reviewers to consider university context (12.0%). However, reviewers often repeated the same conclusion of the evaluation in the replies (48.4%), illustrated their considerations for data incredibility and inconsistency (31.0%), and provided detailed explanation for the evaluation reasoning (12.7%). Cross analysis of the opinions between university staff and reviewers revealed that the reviewers applied similar strategies to reply the objections of institutional staff. Four threats to credibility and consistency exist: 1. The university context was not fully considered; 2. The on-site visit examined only the isolated phenomena but not dynamic processes; 3. The evaluation judgments could be influenced by personal values; 4. The conclusion was not drawn based on long- term observations and interactions. Originality/value To conduct best practice of qualitative evaluation of higher education, institutional staff and external reviewers are required to have appropriate evaluation knowledge and skills. Making reflections from the suggestions of evaluation reports, institution can find out its weakness for institutional development, and make improvements for better educational quality. Through the inquiry of this study, the challenges of qualitative evaluation can be better understood, enabling future improvement of the evaluation process and making better educational policy of higher education.
期刊論文
1.Preskill, H.、Boyle, S.(2008)。A multidisciplinary model of evaluation capacity building。American Journal of Evaluation，29(4)，443-459。  new window
2.Stufflebeam, D. L.(2001)。The meta-evaluation imperative。American Journal of Evaluation，22(2)，183-209。  new window
3.Mark, M. M.、Henry, G. T.(2004)。The mechanisms and outcomes of evaluation influence。Evaluation，10(1)，35-57。  new window
4.Lather, P.(1993)。Fertile obsession: Validity after poststructuralism。The Sociological Quarterly，34(4)，673-693。  new window
5.Patton, M. Q.(1999)。Enhancing the quality and credibility of qualitative analysis。Health Services Research，34(5 Part 2)，1189-1208。  new window
6.王麗雲(20140100)。透過校務研究進行自我評鑑與自我改進。評鑑雙月刊，47，19-23。  延伸查詢new window
7.王保進(20110100)。第一週期系所評鑑結果之後設評鑑：評鑑報告內容分析。評鑑雙月刊，29，9-14。  延伸查詢new window
8.王如哲、楊瑩、劉秀曦(20120400)。臺灣高等教育評鑑的回顧與展望。臺灣教育，674，20-24。new window  延伸查詢new window
9.曾淑惠(20090900)。技職校院中評鑑倫理兩難之探析。教育研究與發展期刊，5(3)，209-239。new window  延伸查詢new window
10.顏善邦(20100500)。美國HLC機構（校務）評鑑及其評鑑倫理。評鑑雙月刊，25，9-12。  延伸查詢new window
11.鄭淑惠(20100400)。建構優質化學校：評鑑能力之探析。教育研究，192，55-66。new window  延伸查詢new window
12.Scriven, M.(1969)。An Introduction to meta-evaluation。Educational products Report，2(5)，36-38。  new window
13.黃政傑、張嘉育(20100500)。我國大學系所評鑑之問題分析與改進方向。教育政策論壇，13(2)，43-76。new window  延伸查詢new window
14.Thomas, V. G.(2004)。Building a contextually responsive evaluation framework: Lessons from working with urban school interventions。New Directions for Evaluation，101，3-23。  new window
15.Smith, J. K.(1992)。Interpretive inquiry: A practical and moral activity。Theory into Practice，31(2)，100-106。  new window
16.Kane, M. T.(2006)。Validation。Educational Measurement，4(2)，17-64。  new window
17.Kane, M. T.(2004)。Certification testing as an illustration of argument-based validation。Measurement，2(3)，135-170。  new window
18.Greene, J. C.(2011)。The construct of validity as argument。New Directions for Evaluation，130，81-91。  new window
19.House, E. R.(2011)。Conflict of interest and Campbellian validity。New Directions for Evaluation，130，69-80。  new window
20.Cousins, J. B.、Goh, S. C.、Elliott, C. J.、Bourgeois, I.(2014)。Framing the capacity to do and use evaluation。New Directions for Evaluation，141，7-23。  new window
21.劉維琪(2011)。系所對評鑑委員的期待。教育評鑑雙月刊，29，6-7。  延伸查詢new window
22.劉維琪(2010)。大學分類評鑑是限制不是鬆綁。教育評鑑雙月刊，27，4-5。  延伸查詢new window
23.楊國賜(20100500)。談評鑑委員的角色與晤談技巧。評鑑雙月刊，25，13-15。  延伸查詢new window
24.李懿芳(20121100)。加拿大評鑑人員資格認證機制及其對我國教育評鑑專業化之啟示。教育政策論壇，15(4)=44，1-21+23-24。new window  延伸查詢new window
25.Stevahn, L.、King, J. A.、Ghere, G.、Minnema, J.(2005)。Establishing essential competencies for program evaluators。American Journal of Evaluation，26(1)，43-59。  new window
26.游家政、曾祥榕(20041200)。教育評鑑的後設評鑑。教育資料集刊，29，53-94。new window  延伸查詢new window
27.吳清山、王令宜(20071000)。我國大學評鑑：挑戰、因應策略與發展方向。課程與教學，10(4)，15-30。new window  延伸查詢new window
28.王保進(20100500)。專業化評鑑之效度總體展現--評鑑報告撰寫。評鑑雙月刊，25，16-21。  延伸查詢new window
29.黃政傑(20041200)。質性教育評鑑之探討。教育資料集刊，29，95-118。new window  延伸查詢new window
30.Johnson, R. B.、Onwuegbuzie, A. J.(2004)。Mixed methods research: a research paradigm whose time has come。Educational Researcher，33(7)，14-26。  new window
31.Maxwell, J. A.(1992)。Understanding and validity in qualitative research。Harvard Educational Review，62(3)，279-300。  new window
32.Creswell, J. W.、Miller, D. L.(2000)。Determining validity in qualitative inquiry。Theory into Practice，39(3)，124-130。  new window
33.潘慧玲(20030200)。社會科學研究典範的流變。教育研究資訊，11(1)，115-143。new window  延伸查詢new window
34.宋曜廷、潘佩妤(20101200)。混合研究在教育研究的應用。教育科學研究期刊，55(4)，97-130。new window  延伸查詢new window
35.謝志偉(20070300)。教育研究典範的未來趨勢混合方法論（Mixed Methodology）介紹。屏東教育大學學報，26，175-194。new window  延伸查詢new window
36.林天祐(20041200)。教育評鑑實施過程與方法的專業化。教育資料集刊，29，27-52。new window  延伸查詢new window
37.楊瑩、楊國賜、劉秀曦、黃家凱(20140800)。100年度大學校院校務評鑑後設評鑑研究之分析。高教評鑑與發展，8(1)，1-40。new window  延伸查詢new window
38.郭秋永(20110300)。混和研究與質量爭議。東吳政治學報，29(1)，1-64。new window  延伸查詢new window
圖書
1.彭森明(2013)。高等教育校務研究的理念與應用。高等教育出版社。  延伸查詢new window
2.Silverman, D.(2013)。Doing qualitative research: A practical guide。London, England：Sage。  new window
3.Schwandt, T. A.(2002)。Evaluation practice reconsidered。New York, NY：Peter Lang。  new window
4.王文科、王智弘(2013)。教育研究法。臺北市：五南。new window  延伸查詢new window
5.Yarbrough, Donald B.、Shulha, Lyn M.、Hopson, Rodney K.、Caruthers, Flora A.(2011)。The program evaluation standards: A guide for evaluators and evaluation users。Thousand Oaks, California：Sage。  new window
6.Patton, M. Q.(2008)。Utilization-focused evaluation。Sage。  new window
7.Krippendorff, Klaus H.(2013)。Content Analysis: An Introduction to Its Methodology。Sage。  new window
8.Denzin, N. K.(1978)。Sociological Methods: A Source Book。New York, NY：McGraw-Hill。  new window
9.黃光國(2001)。社會科學的理路。臺北：心理出版社。  延伸查詢new window
10.蘇錦麗(19970000)。高等教育評鑑：理論與實際。臺北市：五南圖書出版股份有限公司。new window  延伸查詢new window
11.Denzin, N. K.、Lincoln Y. S.(2011)。The SAGE handbook of qualitative research。SAGE。  new window
12.陳向明(2009)。社會科學質的研究。五南。new window  延伸查詢new window
13.Creswell, John W.、Plano Clark, Vicki L.(2007)。Designing and conducting mixed methods research。Sage Publications。  new window
14.Teddlie, Charles B.、Tashakkori, Abbas(2009)。Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences。Sage。  new window
15.Patton, Michael Quinn、吳芝儀、李奉儒(1995)。質的評鑑與研究。桂冠圖書股份有限公司。  延伸查詢new window
其他
1.Canadian Evaluation Society(2010)。Competencies for Canadian evaluation practice，https://evaluationcanada.ca/txt/2_competencies_cdn_evaluation_practice.pdf。  new window
2.教育部(2013)。教育部試辦認定大學校院自我評鑑機制及結果審查作業原則，http://edu.law.moe.gov.tw/LawContent.aspx?id=GL000677。  延伸查詢new window
3.財團法人高等教育評鑑中心基金會(2015)。第一週期系所評鑑結果，http://www.heeact.edu.tw/sp.asp?xdurl=appraise/appraise_list.asp&ctNode=491&mp=2。  延伸查詢new window
4.教育部(2013)。大學評鑑辦法，http://edu.law.moe.gov.tw/LawContentDetails.aspx?id=FL041732&KeyWordHL=&StyleType=1。  new window
5.財團法人高等教育評鑑中心基金會(2012)。財團法人高等教育評鑑中心基金會2012年報，http://www.heeact.edu.tw/public/Data/3491672571.pdf。  延伸查詢new window
6.American Evaluation Association(2004)。American Evaluation Association guiding principles for evaluators，http://www.eval.org/p/cm/ld/fid=51， 。  new window
7.監察院(2010)。糾正案文，http://www.cy.gov.tw/AP_Home/Op_Upload/eDoc/糾正案/99/0990000790992400184.pdf， 2010/05/10。  new window
圖書論文
1.Lather, P.(2001)。Validity as an incitement to discourse: Qualitative research and the crisis of legitimation。Handbook of research on teaching。Washington, DC：American Education Research Association。  new window
2.Greene, J. C.(2005)。Evaluators as stewards of the public good。The role of culture and cultural context: A mandate for inclusion, truth, and understanding in evaluation theory and practice。Greenwich, England：Information Age Publishing。  new window
3.Abma, T. A.(2006)。The social relations of evaluation。The Sage handbook of evaluation。London, England：Sage。  new window
4.Cronbach, L. J.(1988)。Five perspectives on validity argument。Test validity。Hillsdale, NJ：Lawrence Erlbaum Associates。  new window
5.Guba, Egon G.、Lincoln, Yvonna S.(1994)。Competing paradigms in qualitative research。Handbook of qualitative research。Sage Publications。  new window
 
 
 
 
第一頁 上一頁 下一頁 最後一頁 top
:::
無相關著作
 
QR Code
QRCODE

臺灣人文及社會科學引文索引資料庫

目前上線人數：269（臺灣：251，外國：18） ／ 本年度總使用人次：1311662 ／ 訪客人次（自102年09月）：67104669
國家圖書館著作權聲明 Copyright © 2013 All rights reserved.
本館地址: 100201臺北市中山南路20號． 本館地圖及交通資訊．
總機：(02)23619132．最佳瀏覽解析度為1024x768以上